Why speed is more important than scalability

Software developers creating web applications like to talk about scalability as if it is totally unrelated to computing efficiency. They also like to argue that abstractions are important. They will tell you that their DBAL, Router, View Egine, Dependency Injection and even ORM do not have that much overhead (only a little). They will argue that the learning curve pays off and that the performance loss is not that bad. In this post I’ll argue that page loading speed is more important than scalability (or pretty abstractions).

Orders of magnitude of speed on the web

Just to get an idea of speed, I tried to search for a web action for every order of magnitude:

  • 0,1 ms – A simple database lookup from memory
  • 1 ms – Serving small static content from RAM
  • 10 ms – An very simple API that only does a DB lookup
  • 100 ms – A complex page load that calls multiple APIs
  • 1000 ms – Nothing should take this long… 🙂

But I’m sure that your website has pages that take a full second or more (even this site has). How can that be?

CPUs and web farms

Most severs have 1 to 4 CPUs. Each CPU has 2 to 32 cores. If you do a single web request, then you are using (at most) a single core of a single CPU. Maybe that is why people say that page loading speed is irrelevant. If you have more visitors, then they will use other cores or even other CPUs. This is true, but what if you have more concurrent requests than visitors? You can simply add machines and configure a web farm, as most people do.

At some point you may have 16 servers running your popular web application with page loads that average at 300 ms. If you could bring down the page load time to 20 ms, you could run this on a single box! Imagine how much simpler that is! The “one big box” strategy is also called the “big iron” strategy. Often programmers are not very careful with resources. This is because a software developers tend to aim for beautiful abstractions and not for fast software.

Programming languages matter

Only when hardware enthusiasts and software developers work together you may get efficient software. Nice frameworks may have to be removed. Toys like Dependency Injection that mainly bring theoretical value may have to be sacrificed. Also languages need to be chosen for execution speed. Languages that typically score good are: C, C# (Mono), Go and even Java (OpenJDK). Languages that typically score very bad: PHP, Python, Ruby and Perl (source: benchmarksgame). This is understandable, as these are all interpreted languages. But it should be considered when building a web application.

Stop using web application frameworks

But even if you use an interpreted language (which may be 10x slower), then you can still have good performance. Unless of course you build applications consisting of tens of thousands of lines of code that all need to be loaded. Normally people would not do that as it would take too much time to write. (warning: sarcasm) In order to be able to fail – against all odds – people have created frameworks. These are tens of thousands of lines of code that you can install on your server. With it your small and lean application will still be slow (/sarcasm). I think it is fair to say that frameworks will make your web application typically 10-100x slower.

Now let that be exactly the approach that is seen in the industry as “best practice”. Unbelievable right?

5 reasons your application is slow

If you are on shared hardware (VM or shared webhosting), then you need to fix that first. I’m sure switching to dedicated hardware will give you a better (and more consistent) performance pattern. Still, each of the following problems may give you an order of a magnitude of speed decrease:

  1. Not enough RAM
  2. No SSD disks (or wrong controllers)
  3. Using an interpreted programming language
  4. A bloated web framework
  5. Not using Memcache where possible

How many of these apply to you? Let me know in the comments.


This post will probably be considered offending by programmers that like VMs, frameworks and interpreted languages. Please, don’t use that negative energy in the comments. Use it to “Go” and try Gorilla, I dare you! It is not a framework and it is fast, very fast! It will not only going to be interesting and a lot of fun, it may also change your mind about this article.

NB: Go is almost as fast as the highly optimized C code of Nginx (about 10-100x faster than PHP).


Refactoring, upgrading and other software maintenance

In 1976, Lientz and Swanson categorized software maintenance activities into four classes (from Wikipedia):

  • Corrective: diagnosing and fixing errors, possibly ones found by users
  • Perfective: implementing new or changed user requirements which concern functional enhancements to the software
  • Adaptive: modifying the system to cope with changes in the software environment (DBMS, OS)
  • Preventive: increasing software maintainability or reliability to prevent problems in the future

“Corrective” and “perfective” maintenance are very visible and bring value to the customer. The need for “adaptive” and “preventive” maintenance on the other hand is harder to defend.

Customer value or waste?

The customer will demand any “corrective” change to be executed by the software developers with high priority, especially when it impacts his business. Any customer will also have a (wish) list of “perfective” enhancements to be executed and the client is willing to pay and prioritize this maintenance.

“Adaptive” maintenance on the other hand brings no (direct) value to the customer. It will be hard to get the customer to pay or even prioritize an operating system upgrade. Customers do like to argue “it works, do not touch it”. But the Internet has changed our world. We live in an all-connected world where security fixes need to be applied monthly. At some point the vendor stops providing (security) updates. At this point the product is called end-of-life (EOL). Not upgrading is considered a risk nowadays. This is why software developers can often argue that upgrading needs to get priority and the costs are justified. Examples of “adaptive” maintenance:

  1. Dependency upgrades
  2. Framework upgrades
  3. Operating system upgrades
  4. Database upgrades

“Preventive” maintenance is even harder to defend. Like “adaptive” it does not bring (direct) value to the customer. In the Agile methodology everything not adding value to the customer is called “waste”. Examples of “preventive” maintenance:

  1. Small code refactoring
  2. Big code refactoring
  3. Database refactoring

What is refactoring?

Wikipedia has a rather long definition that I tried to compact to:

Refactoring: technique for restructuring, altering internal structure without changing external behavior, undertaken to improve maintainability and extensibility

It may be trivial to conclude that refactoring is waste. Refactoring does not change behavior (functionality) and does therefor not bring value to the customer. Interestingly it is not mentioned as a waste in The Seven Wastes of Software Development by Matt Stine nor in the How to Manage the “7 Wastes” of Agile Software Development by Vijaya Kumar Bandaru. These posts are both referring to the 2006 book Implementing Lean Software Development: From Concept to Cash by Mary and Tom Poppendieck. Vijaya only says that “not refactoring” can lead to “partially done work”. From this remark we can conclude that not properly refactored code is not “done”. This is confirmed by Mayank Gupta who writes that “A common mistake is to not keep refactoring in the definition of done.”

Refactoring needs unit tests

Changing code without a functional reason, seems counter-productive at first. You spend money and take a risk to break the software, without promising any business benefits. This is why in software development the change management “change committee” should include software architects and security engineers, not only business people. The truth is that refactoring leads to lower costs for fixing bugs and extending with new features. This is mainly, because refactoring leads to software developers “touching” (and thinking about) the code. To counter the negative effects of refactoring (breaking software) one can introduce (and require) unit tests. These are programmed (automated) tests that ensure that after refactoring the code still works as before.

Selling refactoring to the business

Refactoring code that is touched by user stories in the sprint is easily done, since people are working on that code anyway. Making refactoring part of your definition-of-done encourages refactoring every time a piece of code is touched. To ensure high-quality, make sure the touched code also checks all the other marks on your definition-of-done. Some of our important definition-of-done topics:

  1. Unit tests written/ran
  2. Refactoring done
  3. Documentation updated
  4. Comitted to Git
  5. Functional tests written/ran
  6. Accepted by customer

Refactoring code unrelated to user stories in the sprint needs to be sold to the customer. Otherwise the customer will not understand why certain bugs suddenly arise. If you do not succeed to do so you must have patience. Every time you touch a part of the code for a user story, take the chance to refactor the related code or database. This way after a year or two you are able to renew your code base and data structure one feature or table at a time. Just make sure every user story is a little more expansive, so that you can squeeze in the preventive maintenance needed.