Symfony on HHVM 3 and Nginx 1.4 vs PHP 5.5 and Apache 2.4

symfony_hhvm_nginx

Installing Symfony

From symfony.com/download we get the latest (2.4.4) version of Symfony. I have unpacked it and put it in the directory “/home/maurits/public_html”. In “app/AppKernel.php” I moved the “AcmeDemoBundle” to the production section and in “routing.yml” I added the “_acme_demo” route that was originally in “routing_dev.yml”.

Test environment

I tested on my i5-2300 CPU with 16GB of RAM and an SSD. To run a benchmark, I installed on my Ubuntu 14.04 both Apache 2.4 with PHP 5.5 and Nginx 1.4 with HHVM 3. I used Apache Bench (ab) to test the “/demo” path on both web servers. In Apache, I disabled XDebug and enabled Zend OPcache.

Install and configure HHVM 3 with Nginx 1.4 for Symfony

For HHVM, we find pre-built (64-bit) packages listed on Github. This is how you install them on Ubuntu 14.04:

wget -O - http://dl.hhvm.com/conf/hhvm.gpg.key | sudo apt-key add -
echo deb http://dl.hhvm.com/ubuntu trusty main | sudo tee /etc/apt/sources.list.d/hhvm.list
sudo apt-get update
sudo apt-get install hhvm

First we install Nginx from the Ubuntu 14.04 repository using:

sudo apt-get install nginx

Now we configure Nginx using the “normal” FastCGI configuration:

server {
    listen             8080;
    server_name        sf2testproject.dev;

    root /home/maurits/public_html/web;

    location / {
        # try to serve file directly, fallback to rewrite
        try_files $uri @rewriteapp;
    }

    location @rewriteapp {
        # rewrite all to app.php
        rewrite ^(.*)$ /app.php/$1 last;
    }

    location ~ ^/(app|app_dev|config)\.php(/|$) {
        fastcgi_pass 127.0.0.1:9000;
        fastcgi_split_path_info ^(.+\.php)(/.*)$;
        include fastcgi_params;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        fastcgi_param HTTPS off;
    }
}

Single request

When doing a single request and analyzing the response times using the Firebug “Net” panel there is no noticeable difference. This is probably because the threads do not have to compete for CPU. So let’s skip this and do some load testing.

Apache Bench results (Symfony 2.4 / Apache 2.4 / PHP 5.5)

maurits@nuc:~$ ab -c 10 -n 2000 http://sf2testproject.dev/demo/
This is ApacheBench, Version 2.3 <$Revision: 1528965 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking sf2testproject.dev (be patient)
Completed 200 requests
Completed 400 requests
Completed 600 requests
Completed 800 requests
Completed 1000 requests
Completed 1200 requests
Completed 1400 requests
Completed 1600 requests
Completed 1800 requests
Completed 2000 requests
Finished 2000 requests

Server Software:        Apache/2.4.7
Server Hostname:        sf2testproject.dev
Server Port:            80

Document Path:          /demo/
Document Length:        4658 bytes

Concurrency Level:      10
Time taken for tests:   9.784 seconds
Complete requests:      2000
Failed requests:        0
Total transferred:      9982000 bytes
HTML transferred:       9316000 bytes
Requests per second:    204.42 [#/sec] (mean)
Time per request:       48.918 [ms] (mean)
Time per request:       4.892 [ms] (mean, across all concurrent requests)
Transfer rate:          996.36 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.1      0       3
Processing:    18   49  10.9     50      90
Waiting:       15   41  10.1     41      78
Total:         18   49  10.9     50      91

Percentage of the requests served within a certain time (ms)
  50%     50
  66%     54
  75%     56
  80%     58
  90%     62
  95%     65
  98%     69
  99%     73
 100%     91 (longest request)

Apache Bench results (Symfony 2.4 / Ningx 1.4 / HHVM 3)

maurits@nuc:~$ ab -c 10 -n 2000 http://sf2testproject.dev:8080/demo/
This is ApacheBench, Version 2.3 <$Revision: 1528965 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking sf2testproject.dev (be patient)
Completed 200 requests
Completed 400 requests
Completed 600 requests
Completed 800 requests
Completed 1000 requests
Completed 1200 requests
Completed 1400 requests
Completed 1600 requests
Completed 1800 requests
Completed 2000 requests
Finished 2000 requests

Server Software:        nginx/1.4.6
Server Hostname:        sf2testproject.dev
Server Port:            8080

Document Path:          /demo/
Document Length:        4658 bytes

Concurrency Level:      10
Time taken for tests:   4.678 seconds
Complete requests:      2000
Failed requests:        0
Total transferred:      9900000 bytes
HTML transferred:       9316000 bytes
Requests per second:    427.50 [#/sec] (mean)
Time per request:       23.392 [ms] (mean)
Time per request:       2.339 [ms] (mean, across all concurrent requests)
Transfer rate:          2066.52 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.1      0       1
Processing:     8   23  11.5     21      84
Waiting:        8   22  11.3     20      84
Total:          8   23  11.5     21      84

Percentage of the requests served within a certain time (ms)
  50%     21
  66%     26
  75%     30
  80%     32
  90%     39
  95%     46
  98%     54
  99%     58
 100%     84 (longest request)

Conclusion

On both setups, I did not do any optimization. I just installed and ran the benchmark. It seems that Symfony 2.4 on HHVM is about twice as fast as on PHP 5.5. In a real life setup this means you need half the machines. This seem like a good cost reduction at first, but it may not be as good as it looks. I believe that in reality most Symfony applications are doing database and/or API calls. These will not be faster when using HHVM, since HHVM only speeds up PHP execution. This is why I think that the difference will be smaller (than a factor 2) for a real life Symfony application. What do you think?

19 thoughts on “Symfony on HHVM 3 and Nginx 1.4 vs PHP 5.5 and Apache 2.4”

  1. A better benchmark would be HHVM + Nginx vs PHP + Nginx.
    Interesting anyway, thanks !

  2. @Victor: Thank you for the suggestion. I chose mod_php, since I believe it is more popular than FPM at the moment. Good idea for my next post 🙂

  3. This benchmark basically means nothing. You are testing to completely different stacks against each other. You should throw PHP-FPM under Apache *at least*. Even better if you want to throw two different web servers into the mix, test each interpreter under each web server. Because there is no “control” here, just two utterly different things being compared.

  4. @Jonathan: Thank you for your reply and I understand your reasoning. Still, I believe it will mean to some people (including me) that switching from what is standard to what “the cool kids are using” is not worth it.

  5. @Maurits you don’t think it’s worth it? I thought doubling the performance was pretty impressive.

  6. @Mike: When you have a real application, there will be database calls and API calls and such, so the PHP execution time will be a fraction of the total execution time. But even reducing half the front nodes (optimistic) in a setup at the cost of violating standard compliance and risking exotic bugs? I would choose not to.

  7. @Maurits, I’ve been using hhvm for a while now. The tests are great that you are showing however there are more to it than just a simple ab benchmark.

    Take tests for say iterating over 1M records performing a MD5 function.

    I was able to complete 1M on hhvm in less than 1 minute.
    Apache + PHP… didn’t finish.

    I agree with Mike, to double “potential” performance, why not.

  8. @Jesse: I did not experience that:

    maurits@nuc:~$ time hhvm md5.php
    real 0m0.561s
    user 0m0.539s
    sys 0m0.021s
    maurits@nuc:~$ time php md5.php
    real 0m0.871s
    user 0m0.860s
    sys 0m0.010s
    maurits@nuc:~$ cat md5.php

  9. @Maurits “Still, I believe it will mean to some people (including me) that switching from what is standard to what “the cool kids are using” is not worth it.”

    Obviously you are not one of the cool kids but one of the “I-dont-wanna-take-any-risks-in-my-life-kids” 😉

  10. @Sander: I agree, I do not like risks. But the circumstances you are in do also matter. For instance the importance of the operational costs vs the building costs of the application.

  11. @Bombos: It is enabled, as the post states, so what about it?

  12. You have to have some balls to say “This benchmark means nothing”. Maurits did all the research and showed us how to test several platform combinations (exotic or not). Two-fold performance enhancement might not be impressive -and far from the 5x promised by the HHVM people- but will certainly rise eyebrows in most Ops guys.

  13. @Alberto: Thank you for your supporting comment. As I stated before a reduction in front-end nodes, and thus operational costs, may seem good at first. Nevertheless I would prefer a more boring, flexible and predictive setup over this improved performance.

  14. Your assertion is false that db calls in HHVM have the same cost as PHP. This is because HHVM implements a db adapter in pure PHP, meaning it is too compiled into byte code. this results in most of the speed advantage you have with HHVM. they also offer async mysql as well.

  15. @Garret: Thank you for this extra information. I was not aware of this. I guess this optimization (and async feature) can make a difference in a real wold application.

  16. This really isn’t a new technology guys… As far as “the cool kids are doing it” excuse isn’t really valid. HHVM is an extremely proven platform with very little bugs. Its got a huge contributor base, and there are nightly fixes.

    It is true that database calls are now the new bottle neck of your application, but it will still speed everything else up which pretty much means the faster you make your database calls the better it gets. There is a reason Facebook can handle such high traffic with low server cost, and maintenance. Its great for doubling your server efficiency.

    Also HHVM gets much higher than just twice the RPS. Sometimes exceeding 4 times the normal response times. The JIT inside HHVM reduces machine code on each iteration therefore increasing speed with each request.

    I have used this technology with a very high traffic tech company before and I haven’t been let down so far on a WordPress setup.

Leave a Reply

Your email address will not be published. Required fields are marked *