Fluid web forms using AJAX

Sometimes you want a web form that changes based on the input. The web form can give the user feedback while filling it in: We often see web forms put green check-marks behind (server side) validated fields. Some fields may turn red after you filled them in wrong. This validation can either be done client side (in JavaScript) or server side (using AJAX).

I’ve created a small application (available on Github) that looks like this:

fluid_form

When you fill in a field and it loses focus, the entire web form is submitted and updated. This includes field validation on all fields and marking them red when validation fails. Note that only after you submitted the form you want validate that the required fields are filled in. So you need to differentiate between validation before and after submit. The form has a hidden “submitted” field to facilitate that.

This demo is using the following components:

  1. Bootstrap 3 (requires jQuery)
  2. jQuery-populate (plugin by Dave Stewart)

All validation and calculation is done server side.

Why? Well, some validation can simply not be done client side. Promo codes in your web shop’s shopping cart is a good example (for security reasons). Another reason may be that sending all your validation data in a JSON object may in some cases overload your client. For instance, when you have hundreds of products with lots of product specific options. This may lead to slow rendering, due to excessive JavaScript usage. This may be especially problematic on less powerful clients, like mobile phones.

There is a downside: In order to allow for real-time server side validation you need to be able to sustain way more page loads. If the form is submitted after every changed input field, you need the web form handling to be light-weight. I managed to get the initial page load and the submit and validate to be 1 (one!) millisecond in vanilla PHP on my local machine (see screenshot). I guess that if your page loads are this cheap, you may consider using a little more server side technology. ūüôā

Play with it, feel it!

git clone https://github.com/mevdschee/bootstrap-fluid-form.git
cd bootstrap-fluid-form
php -S localhost:8888

Point your browser to http://localhost:8888 and test the application. Pay special attention to the speed and the usability. Does your application resemble this? If not, then you should ask yourself… why not?

Share

PHP-CRUD-API now has transforms!

Last week I created a new GitHub project called “PHP-CRUD-API” and it allows you to quickly setup a simple REST API with CRUD functionality by just adding a single “api.php” file to your project and configuring it’s database connection. Today I show how the relational support of this project works.

Supported table relations

There are three types of table relations supported:

  • BelongsTo
  • HasMany
  • HasAndBelongsToMany

Blog example

When you use the “list” command of the API it allows you to specify multiple tables. If these tables have relations (foreign keys) the output will be filtered in such a way that only relevant records are returned. This is what the API outputs when you list posts and comments filtered on a specific post:

{
    "posts": {
        "columns": [
            "id",
            "user_id",
            "category_id",
            "content"
        ],
        "records": [
            [
                "1",
                "1",
                "1",
                "blog started"
            ]
        ]
    },
    "comments": {
        "relations": {
            "post_id": "posts.id"
        },
        "columns": [
            "id",
            "post_id",
            "message"
        ],
        "records": [
            [
                "1",
                "1",
                "great"
            ],
            [
                "2",
                "1",
                "fantastic"
            ]
        ]
    }
}

Not so useful right? You would probably like to see something like this:

{
    "posts": [
        {
            "id": "1",
            "comments": [
                {
                    "id": "1",
                    "post_id": "1",
                    "message": "great"
                },
                {
                    "id": "2",
                    "post_id": "1",
                    "message": "fantastic"
                }
            ],
            "user_id": "1",
            "category_id": "1",
            "content": "blog started"
        }
    ]
}

That is exactly what the function “php_crud_api_transform()” does. You run this function on the client after receiving the API response. This is beneficial as it uses the CPU and RAM of the API consumer instead of that of the API server. This transformation function is implemented in PHP and JavaScript, so that you can make spiders and users with browsers equally happy!

<?php
function php_crud_api_transform(&$tables) {
	$getobjs = function(&$tables,$table_name,$where_index=false,$match_value=false) use (&$getobjs) {
		$objects = array();
		foreach($tables[$table_name]['records'] as $record) {
			if ($where_index===false || $record[$where_index]==$match_value) {
				$object = array();
				foreach ($tables[$table_name]['columns'] as $index=>$column) {
					$object[$column] = $record[$index];
					foreach ($tables as $relation=>$reltable) {
						foreach ($reltable['relations'] as $key=>$target) {
							if ($target == "$table_name.$column") {
								$columnidx = array_flip($reltable['columns']);
								$object[$relation] = $getobjs($tables,$relation,$columnidx[$key],$record[$index]);
							}
						}
					}
				}
				$objects[] = $object;
			}
		}
		return $objects;
	};
	$tree = array();
	foreach ($tables as $name=>$table) {
		if (!isset($table['relations'])) {
			$tree[$name] = $getobjs($tables,$name);
		}
	}
	return $tree;
}

And the JavaScript version:

function php_crud_api_transform(tables) {
	var array_flip = function (trans) {
		var key, tmp_ar = {};
		for (key in trans) {
			tmp_ar[trans[key]] = key;
		}
		return tmp_ar;
	};
	var get_objects = function (tables,table_name,where_index,match_value) {
		var objects = [];
		for (var record in tables[table_name]['records']) {
			record = tables[table_name]['records'][record];
			if (!where_index || record[where_index]==match_value) {
				var object = {};
				for (var index in tables[table_name]['columns']) {
					var column = tables[table_name]['columns'][index];
					object[column] = record[index];
					for (var relation in tables) {
						var reltable = tables[relation];
						for (var key in reltable['relations']) {
							var target = reltable['relations'][key];
							if (target == table_name+'.'+column) {
								column_indices = array_flip(reltable['columns']);
								object[relation] = get_objects(tables,relation,column_indices[key],record[index]);
							}
						}
					}
				}
				objects.push(object);
			}
		}
		return objects;
	};
	tree = {};
	for (var name in tables) {
		var table = tables[name];
		if (!table['relations']) {
			tree[name] = get_objects(tables,name);
		}
	}
	return tree;
}

Check out all the source code on my GitHub account: https://github.com/mevdschee/php-crud-api

Share

Please stop using pop-up windows in web applications

In the Nineties, we were writing desktop applications with pop-ups. These desktop applications consisted of multiple windows that popped up. I was programming Delphi back in these days, where windows were called forms. The naming probably came from their main purpose: data entry into the bundled Paradox database. This is comparable with the forms that we see today on the web and they were equally abused for other purposes.

The purpose of forms in a database driven application is to facilitate CRUD (Create, Read, Update, Delete) operations. That is why you need the List, Add, Edit and Delete forms. Maybe the Delete form is not needed and this can be just a conformation dialog. To simplify the application flow, it used to be possible to make pop-up windows “modal”. This meant that you could not ignore them and had to click them away before you could continue. This is typically something you would want when you want the user to confirm an action or acknowledge a critical error.

JavaScript (like Delphi back in the day) has simplified making modal pop-ups by offering us the functions “alert()” and “confirm()”. But let’s take an example of a typical company database application. Such an application may have an overview showing a listing of customers. Maybe you can search in this list. If you click on a specific customer you may be able to see a list of their¬†orders. In the Delphi days, we would have a window with customers and when you clicked on the “view orders” button, it would open up a new window with this information.

lightbox_popup
Figure 1: Example of a jQuery lightbox styled pop-up in WordPress

On the web, we first tried to copy this model by opening new browser windows in web applications. Then came the era of pop-up ads and ad blockers, and people started moving away from the multiple browser window strategy. This move was stimulated even more when browsers started having tabs. Then we saw that developers started making jQuery lightbox styled pop-ups on top of other pages. These are still used a lot, but I feel they lead to horrible user experiences wherever they are used.

In the end, most of the developers saw the light (fortunately). They probably realized that you do not need a stack of windows since the web browser already allows you to go back (and forward by opening new pages) in this stack using the back button and the links you can click. Also, the browser creators¬†acknowledged that in order to make users use the “alert()” and “confirm()” functions, they had to make sure these pop-ups rendered in much prettier way. Until then, they resembled the JavaScript error pop-ups from the Nineties.

So today, when I stumble across an HTML anchor tag with an TARGET property, I cringe. It hurts even more when I see people use jQuery lightbox styled pop-ups. Not only because they are almost always a pain to close, but also because they do not work properly on different screen sizes (like phones). However, the worst thing about this form of pop-up, is the wrong expectation that people have about the underlying page. What should happen when the pop-up is closed? Should it be reloaded, so it is updated? Or can it have old data? I don’t know, can you tell me?

To add something constructive to this rant, I will also propose some new rules for pop-up lovers that have a hard time forgetting the Nineties:

  1. Everything is a page, and your application can most probably be represented by a tree (with some jumps back).
  2. Use clickable breadcrumbs to show the current path in the tree structure of your application.
  3. The back button should work everywhere and warn when needed (about reposts or expired pages).
  4. Make sure all your pages have a single, structured, short, but descriptive URL.
  5. For confirmation, rely on the JavaScript “confirm()” function.
  6. Use top of page colored flash messages to show success or failure.

Are you in the business of making web applications that mainly do CRUD operations on a database? Have you still not sworn off pop-ups? Do you think I am wrong? Please use the comments to discuss.

Share

API first architecture or the fat vs thin server debate

API first architecture is an architecture that treats the API user as the primary user of the application. This means that API is not an alternative view in the MVC paradigm, but it has the highest priority. The main differentiators are that in “API first” the architecture enforces a complete, responsive, and well-documented API. This is especially important when targeting: mobile (Apps connect to the API), resellers (their presentation layer uses the API) and highly integrated, but decoupled, multi-product environments.

MVC reuse

The MVC architecture has been popular for a long time already. In 2004, it’s popularity skyrocketed when Ruby on Rails was released. MVC allows for high reuse in the case you have a front-end / back-end application (in the CMS sense), where customers use the front-end and employees use the back-end. This does require that you choose the same software stack for both the front-end and back-end, and make those applications as similar as possible. When the MVC strategy is executed properly, many parts of the application can be reused. Some of the parts that can be reused are: DBAL/ORM, Business Logic, Presentation and AAA. Specifically AAA (Authentication, Authorization and Accounting) can be reused by allowing employees to impersonate customers, use the same login screen and share logging facilities.

Mobile views for MVC

In 2007, Apple introduced the iPhone and from that time on the importance of web application (and websites) on small screens quickly grew. MVC applications were, and still are, very suited to facilitate small screens. All that is needed is a separate or adjusted set of views that is usable on a smartphone or tablet. The strategy of creating a single set of views that is suitable for mobile and also for desktop is called “mobile first“. This is the most cost-effective and radical approach, which requires strong leadership and decision taking, because all the software needs dramatic change, as all the views need to be adjusted. The alternative is to maintain two sets of views: one for mobile and one for desktop. The alternative views are often hosted on a “m.” subdomain. This is a simple and transparent approach.

Adding API to MVC

The dreadful “HTML5 vs Native” app development debate is going on right now and I quote Danny Brown:

Any company creating mobile apps today faces an important decision,¬†Native or HTML5? Each one has its advantages, but choosing the wrong one could be costly. –¬†Danny Brown

Choosing Native requires you to build a complete, responsive, and well-documented API, while choosing HTML5 requires you to redesign the views. There are arguments to defend either path and it depends on the situation what choice is best. There is one approach that will always fail: building an API as views on top of MVC. Let me explain why that fails and why so many people do it anyway.

Typically the (server-side) MVC approach leads to pages with 200 ms of load time. In this approach the server does three things: database abstraction, business logic, and presentation. This is why it is also referred to as “fat server”. An API is not responsible for presentation, executes smaller business logic per request and is therefor named “thin server“. A good API is highly optimized for speed and has typical load times under 20 ms. The means that when a mobile page is constructed there can be multiple (up to 10) calls made to render the full page within 300 ms.

Still when one took the MVC approach and is now lacking an API, the easiest thing to do is to add a few views that output JSON and call it a “RESTful API”. All you need to do is write some documentation and the boss will be happy. The fact that this API is totally unusable in real life, because it does not scale and is horribly slow, will only be noticed when the API is actually used and there is no way back.

architecture_v

Twitter & API first

In 2010,¬†Twitter announced their “API first” strategy. They call this a JavaScript architecture, since they created a web application in JavaScript in a similar architecture as the mobile “apps”. This allows them to have full reuse of the API they build. Where initially the API was something “extra”, next to their web application, it then became the foundation of all other development. Their API is focused on delivering optimal integration for JavaScript programmers by using a RESTful JSON API. But they are also serving their application using traditional pages:

In order to support crawlers and users without JavaScript, we needed a rendering system that runs on both server and client. – Twitter

This approach of delivering traditional pages, while still using the “API first” strategy is what I call “Hybrid”. In the diagram below I’ve tried to enumerate the different approaches.

architecture_h

Conclusion

Optimal reuse brings down costs, but optimal reuse can only be achieved when there is a strong architectural strategy to follow. Refactoring code to increase architectural compliance does not bring value to the business directly. It will bring down the cost of change eventually, but the level of trust needed for these decisions is not easily gained.

Further reading

This is not the first post about “API first”. Check out the following links if you want to learn more about it:

Slides

I created a deck of¬†11 slides about “api first” for a presentation I gave on the 26th of November at LeaseWeb.

Share