API first architecture or the fat vs thin server debate

API first architecture is an architecture that treats the API user as the primary user of the application. This means that API is not an alternative view in the MVC paradigm, but it has the highest priority. The main differentiators are that in “API first” the architecture enforces a complete, responsive, and well-documented API. This is especially important when targeting: mobile (Apps connect to the API), resellers (their presentation layer uses the API) and highly integrated, but decoupled, multi-product environments.

MVC reuse

The MVC architecture has been popular for a long time already. In 2004, it’s popularity skyrocketed when Ruby on Rails was released. MVC allows for high reuse in the case you have a front-end / back-end application (in the CMS sense), where customers use the front-end and employees use the back-end. This does require that you choose the same software stack for both the front-end and back-end, and make those applications as similar as possible. When the MVC strategy is executed properly, many parts of the application can be reused. Some of the parts that can be reused are: DBAL/ORM, Business Logic, Presentation and AAA. Specifically AAA (Authentication, Authorization and Accounting) can be reused by allowing employees to impersonate customers, use the same login screen and share logging facilities.

Mobile views for MVC

In 2007, Apple introduced the iPhone and from that time on the importance of web application (and websites) on small screens quickly grew. MVC applications were, and still are, very suited to facilitate small screens. All that is needed is a separate or adjusted set of views that is usable on a smartphone or tablet. The strategy of creating a single set of views that is suitable for mobile and also for desktop is called “mobile first“. This is the most cost-effective and radical approach, which requires strong leadership and decision taking, because all the software needs dramatic change, as all the views need to be adjusted. The alternative is to maintain two sets of views: one for mobile and one for desktop. The alternative views are often hosted on a “m.” subdomain. This is a simple and transparent approach.

Adding API to MVC

The dreadful “HTML5 vs Native” app development debate is going on right now and I quote Danny Brown:

Any company creating mobile apps today faces an important decision, Native or HTML5? Each one has its advantages, but choosing the wrong one could be costly. – Danny Brown

Choosing Native requires you to build a complete, responsive, and well-documented API, while choosing HTML5 requires you to redesign the views. There are arguments to defend either path and it depends on the situation what choice is best. There is one approach that will always fail: building an API as views on top of MVC. Let me explain why that fails and why so many people do it anyway.

Typically the (server-side) MVC approach leads to pages with 200 ms of load time. In this approach the server does three things: database abstraction, business logic, and presentation. This is why it is also referred to as “fat server”. An API is not responsible for presentation, executes smaller business logic per request and is therefor named “thin server“. A good API is highly optimized for speed and has typical load times under 20 ms. The means that when a mobile page is constructed there can be multiple (up to 10) calls made to render the full page within 300 ms.

Still when one took the MVC approach and is now lacking an API, the easiest thing to do is to add a few views that output JSON and call it a “RESTful API”. All you need to do is write some documentation and the boss will be happy. The fact that this API is totally unusable in real life, because it does not scale and is horribly slow, will only be noticed when the API is actually used and there is no way back.

architecture_v

Twitter & API first

In 2010, Twitter announced their “API first” strategy. They call this a JavaScript architecture, since they created a web application in JavaScript in a similar architecture as the mobile “apps”. This allows them to have full reuse of the API they build. Where initially the API was something “extra”, next to their web application, it then became the foundation of all other development. Their API is focused on delivering optimal integration for JavaScript programmers by using a RESTful JSON API. But they are also serving their application using traditional pages:

In order to support crawlers and users without JavaScript, we needed a rendering system that runs on both server and client. – Twitter

This approach of delivering traditional pages, while still using the “API first” strategy is what I call “Hybrid”. In the diagram below I’ve tried to enumerate the different approaches.

architecture_h

Conclusion

Optimal reuse brings down costs, but optimal reuse can only be achieved when there is a strong architectural strategy to follow. Refactoring code to increase architectural compliance does not bring value to the business directly. It will bring down the cost of change eventually, but the level of trust needed for these decisions is not easily gained.

Further reading

This is not the first post about “API first”. Check out the following links if you want to learn more about it:

Slides

I created a deck of 11 slides about “api first” for a presentation I gave on the 26th of November at LeaseWeb.

Share

The PHP floating point precision is wrong by default

Let me show you that PHP is bad at Math:

<?php
echo "0.1 + 0.2 = ". ( 0.1 + 0.2 ) ."\n";
$true = 0.1 + 0.2 == 0.3 ? "Equal" : "Not equal";
echo "0.1 + 0.2 = 0.3 => $true\n";

Output:

0.1 + 0.2 = 0.3
0.1 + 0.2 = 0.3 => Not equal

Now, to analyze what is happening here let’s first look at JavaScript:

>>> 0.1 + 0.2
0.30000000000000004
>>> 0.1 + 0.2 == 0.3
false

Okay, so JavaScript is also bad at Math, but at least it prints numbers (better) as it internally represents them, making debugging easier. We see that the answer is slightly off and that is why the test is returning false. Before explaining why there are errors, let me give you some more examples of the problem:

0.1 * .1 results in 0.010000000000000002
0.7 + .1 results in 0.7999999999999999
1.1 + .1 results in 1.2000000000000002

As you can see the problem also exists when multiplying. This is because the problem does not lie in the operation, but in the way computers internally store numbers that contain a decimal point. The internal representation is called a “floating point “. This floating point representation has accuracy problems as we have shown above, but it doesn’t only apply to PHP floating points. The reason we have these accuracy problems is described in the floating point guide:

Because internally, computers use a format (binary floating-point) that cannot accurately represent a number like 0.1, 0.2 or 0.3 at all.

When the code is compiled or interpreted, your “0.1” is already rounded to the nearest number in that format, which results in a small rounding error even before the calculation happens.  — floating point guide

The floating point guide also explains clearly that:

…binary fractions are different from decimal fractions in what numbers they can accurately represent with a given number of digits, and thus also in what numbers result in rounding errors:

Specifically, binary can only represent those numbers as a finite fraction where the denominator is a power of 2. Unfortunately, this does not include most of the numbers that can be represented as finite fraction in base 10, like 0.1.

Fraction Base Positional Notation Rounded to 4 digits Rounded value as fraction Rounding error
1/10 10 0.1 0.1 1/10 0
1/3 10 0.3 0.3333 3333/10000 1/30000
1/2 2 0.1 0.1 1/2 0
1/10 2 0.00011 0.0001 1/16 3/80

And this is how you already get a rounding error when you just write down a number like 0.1 and run it through your interpreter or compiler. It’s not as big as 3/80 and may be invisible because computers cut off after 23 or 52 binary digits rather than 4. But the error is there and will cause problems eventually if you just ignore it. — floating point guide

Now let’s go back to the PHP floating point calculation and evaluate this code:

<?php
ini_set('precision', 17);
echo "0.1 + 0.2 = ". ( 0.1 + 0.2 ) ."\n";
$true = 0.1 + 0.2 == 0.3 ? "Equal" : "Not equal";
echo "0.1 + 0.2 = 0.3 => $true\n";

Output:

0.1 + 0.2 = 0.30000000000000004
0.1 * 0.2 = 0.3 => Not equal

That is more like it. It does not solve the problem, but makes it easier to understand. Note that we have set the “precision” of the representation of floating point numbers to 17 with the “ini_set” PHP command. Gustavo Lopes explains on the php-internals mailing list why other values (like 100) do not make sense:

Given that the implicit precision of a (normal) IEEE 754 double precision number is slightly less than 16 digits [2], this is a serious overkill. Put another way, while the mantissa is composed of 52 bits plus 1 implicit bit, 100 decimal digits can carry up to 100*log2(10) =~ 332 bits of information, around 6 times more.

Given this, I propose changing the default precision to 17 (while the precision is slightly less than 16, a 17th digit is necessary because the first decimal digit carries little information when it is low). — source

So for now, let’s change the precision to from 14 to 17 in “/etc/php5/apache2/php.ini” on our servers and save ourselves some headaches when we are using PHP floating points.

; The number of significant digits displayed in floating point numbers.
; http://php.net/precision
precision = 17

If you want more background on this topic read the excellent article “What Every Computer Scientist Should Know About Floating-Point Arithmetic“.

Share

Interactive programming tutorials

Dennis Ritchie was an American computer scientist who created the C programming language and the Unix operating system. He also was one of the authors (the ‘R’) of the famous in K&R C book. The first edition of this book was published in 1978 and it was the first widely available book on the C programming language. It is one of the first books I read about programming and I still think it is one of the best.

While some people enjoy reading a book like K&R C to learn a new language (and I certainly did back in the days), I now think reading text books about new programming languages is not the most effective way of learning a new language. Although it may still be the most effective way for learning your first language, because everything is new and you need a thorough understanding of the concepts, I think there may be more effective ways for learning your second, third or fourth programming language.

It was when the world wide web became a commodity that programming books lost there use as a reference, even for code examples. This changed the nature of programming books. I notice most programmers today do not (know how to) implement from documentation. They use implementation examples that they copy/paste from code they find using Google and Koders.

Another concept in programming books is taking advantage of prior knowledge. I remember it was a delight to learn C++ from a book titled “from C to C++”. The author assumed the reader knew everything about C and only discussed the differences, therefor not wasting the readers precious time.

This idea is also the basis for a (relatively) new phenomenon: interactive web applications that behave as language tutors. It learns you programming by example and it allows you to progress at your own speed. Learning by example skips a lot of the theory and this is why I don’t know how well it will work for real beginners, but for programmers that already know a few languages it is a fun and fast way to learn. I made a list of good (free) interactive language tutorials online:

  1. Javascript: Codecademy Javascript
  2. Regular Expression: RegexOne
  3. SQL: SQLzoo
  4. Ruby: Try Ruby
  5. Python: Try Python (Runs on Linux using Moonlight 2)
  6. Haskell: Try Haskell
  7. Scala: Simply Scala
  8. PHP: W3Schools
  9. CSS: CSS 101

Even though you should try all of the above, there may be one “traditional” text-book that you should read. It is the free and very well written book Eloquent JavaScript by Marijn Haverbeke.

http://www.w3schools.com/php/default.asp

Share

Developing a mobile web application with IUI

At LeaseWeb we have a hosting control panel called “Self Service Center” or SSC. We are starting to make our first steps into making the Self Service Center available on a mobile platform.

We have a couple of options on how to implement this:
• Build native applications for each mobile platform ( mainly Android & iOS )
• Build a mobile accessible website

We had to make a list of advantages and disadvantages of the 2 options:

We prefer to keep a single code stack, for maintenance reasons, and so we can make every feature immediately available in the mobile version. This is why we decided to make a proof-of-concept of a mobile accessible website.

The first problem we encountered was to find out how to implement this mobile website the best way possible in the Symfony framework. At first, we checked if this was possible by doing only CSS changes, so without touching the templates. It did not took us long to come to the conclusion that this was not enough and we had to revise all of the templates.

By looking up the API of the Symfony framework on a mobile device we saw that they had a mobile version of it on their website. We were very curious what technology they used and how they implemented it, so we started digging. We found out that they were using the IUI framework on their mobile API page and although it didn’t work flawlessly we were impressed with the way it worked.

So what is this IUI and how does it work?
With IUI you have to think of your webpage as 1 big HTML file which you keep adding content to. So you keep updating your current HTML with new adaptations through AJAX request. When you click a button / link the new page is loaded off-screen. When the screen is loaded it slides smoothly into your screen. The old screens are not unloaded so you can smoothly slide back to the previous screen using the back button in the upper left corner.

Figure 1 – The red box shows the area that is visible on the screen, the other pages can slide in.

Using this framework you can make any webpage feel like a native iPhone application. A single IUI web page can contain multiple related iPhone screens. This makes IUI map on the Symfony MVC framework quite nicely, because all the data of a single MVC view will not fit on one iPhone screen, but it might fit on multiple.

Also for backend developers it is really nice that you don’t have to design all the screen layouts. You can just follow the layout that is provided with the framework and your screens immediately look professional. Although we created an impressive proof-of-concept there were some problems we did encounter:

A broken back button
When you post a form with validation errors, the same page – containing the form – is loaded with the errors displayed next to the input fields. This new page has the same id as the old form causing the back functionality to be broken.

The IUI back functionality works using a “stack”. This stack contains the ids of the previously loaded pages and forms. When the page loads it adds the id to the stack. When a validation error has occurred the stack contains the wrong values, because the validated form and the original form have the same id.

The possible solutions we found are:
1) To fix this we can give the validated form a different id than the original form.
2) Show the (validation) errors on a new page or popup.

An iPhone like way to handle field validation error is to show popups containing the validation errors. This can be achieved in IUI using “Dialog” type forms. First we tried using JavaScript “alert” calls but we had trouble, because these pop-ups are blocking and it is hard to use them without modifying IUI internals.

Displaying flash messages
More or less the same problem occurs when displaying “flash” messages. “Flash” messages are messages often colored yellow, red or green (for warnings, errors and confirmations) and are set on top of the screen displaying the results of the last page load. In IUI the flash messages are not working properly because pages are not reloaded on “back button” navigation. In an iPhone application these messages typically show in a popup.

Conclusion
We think IUI is a good and easy way to convert your current website to a mobile website in a MVC framework. Problems we encountered are related to error and status messages that a typical web application shows inside (and on top of) forms. Unfortunately IUI does not seem prepared for showing these. We’ve found a workaround by showing these messages in “Dialog” type forms.

LeaseWeb Self Service Center – https://secure.leaseweb.com
IUI framework – http://www.iui-js.org/
Symfony framework – http://www.symfony-project.org

Share