Memcache Bundle for Symfony updated!

memcache_v2

We have released version 2 of our Symfony Memcache Bundle. The 3 biggest improvements are:

  • Added Windows support
  • PHP 5.4+ without notices and/or warnings
  • High availability support with redundancy configuration

This actually was a major refactoring. We switched from the PECL “memcached” to the PECL “memcache” (without ‘d’) extension. Only version 3 (which is still in beta) of this extension is supported currently, but this version is quite stable actually. Most of the features of the bundle have not been altered. The Memcache(Pool) object is deliberately not abstracted. This is done to allow for unrestricted access to the underlying implementation. The flags parameter is now supported and this is the third argument on a “set” instruction. You may have to alter your code for this, which may be annoying, but we promise to keep the API stable in this major version that may again last for a few years.

Some articles we have written about the bundle are:

Let us know your experience with this new version.

Download LswMemcacheBundle from Github

Heka monolog decoder

This post is about how to use heka to give your symfony 2 application logs the care they deserve.

Application logs are very important for the quality of the product or service you are offering.

They help you find out what went wrong so you can explain and fix a bug that was reported recently. Or maybe to gather statistics to see how often a certain feature is used. For example how many bare metal reinstallation requests were issued last month and how many of those failed. Valuable information that you could use to decide what feature you are going to work on next.

At LeaseWeb we use quite a lot of php and Seldaek’s monolog is our logging library of choice. Recently we open sourced a heka decoder on github here. For you who do not know heka yet, check out their documentation.

Heka is an open source stream processing software system developed by Mozilla. Heka is a “Swiss Army Knife” type tool for data processing, useful for a wide variety of different tasks, such as …

Heka runs as a system daemon just like logstash or fluentd. Heka is written in go and comes with an easy to use plugin system based on lua. It has almost no dependencies and is lightweight. James Turnbull has written a nice article on how to get started with heka.

send symfony 2 application logs to elastic search

How better to explain with an example.

Lets say you have a symfony 2 application and you want the application logs to be sent to an Elastic Search platform.

On a debian based os you can download one of the heka debian packages from github.

    $ wget https://github.com/mozilla-services/heka/releases/download/v0.9.2/heka_0.9.2_i386.deb
    $ sudo dpkg -i heka_0.9.2_i386.deb

To configure heka you are required to edit the configuration file located at /etc/hekad.toml.

    $ vim /etc/hekad.toml

Take your time to read the excellent documentation on heka. There are many ways of using heka but we will use it as a forwarder:

  1. Define an input, where messages come from.
  2. Tell heka how it should decode the monolog log line into a heka message
  3. Tell heka how to encode the message so Elastic Search will understand it
  4. Define an output, where should the messages be sent to.

First we define the input:

    [Symfony2MonologFileInput]
    type = "LogstreamerInput"
    log_directory = "/var/www/app/logs"
    file_match = 'prod\.log'
    decoder = "Symfony2MonologDecoder"

Adjust `log_directory` and `file_match` according to your setup. As you can see we alread told heka to use the `Symfony2MonologDecoder` to we will define that one next:

    [Symfony2MonologDecoder]
    type = "SandboxDecoder"
    filename = "/etc/symfony2_decoder.lua"

Change the `filename` with the path where you placed the lua script on your system.

Now we have defined the input we can tell heka where to output messages to:

    [ESJsonEncoder]
    index = "%{Hostname}"
    es_index_from_timestamp = true
    type_name = "%{Type}"

    [ElasticSearchOutput]
    message_matcher = "TRUE"
    server = "http://192.168.100.1:9200"
    flush_interval = 5000
    flush_count = 10
    encoder = "ESJsonEncoder"

In the above example we assume that your Elastic Search server is running at 192.168.100.1.

And thats it.

A simple log line in app/logs/prod.log:

    [2015-06-03 22:08:02] app.INFO: Dit is een test {"bareMetalId":123,"os":"centos"} {"token":"556f5ea25f6af"}

Is now sent to Elastic Search. You should now be able to query your Elastic Search for log messages, assuming the hostname of your server running symfony 2 is myapi:

    $ curl http://192.168.100.1:9200/myapi/_search | python -mjson.tool
    {
        "_shards": {
            "failed": 0,
            "successful": 5,
            "total": 5
        },
        "hits": {
            "hits": [
                {
                    "_id": "ZIV7ryZrQRmXDiB6thY_yQ",
                    "_index": "myapi",
                    "_score": 1.0,
                    "_source": {
                        "EnvVersion": "",
                        "Hostname": "myapi",
                        "Logger": "Symfony2MonologFileInput",
                        "Payload": "Dit is een test",
                        "Pid": 0,
                        "Severity": 7,
                        "Timestamp": "2015-06-03T20:08:02.000Z",
                        "Type": "logfile",
                        "Uuid": "344e7cae-6ab7-4fb2-a770-d2cbad6653c3",
                        "channel": "app",
                        "levelname": "INFO",
                        "bareMetalId": 123,
                        "os": "centos",
                        "token": "556f5ea25f6af"
                    },
                    "_type": "logfile"
                },
        // ...
        }
    }

What is important to notice is that the keys token, bareMetalId and os in the monolog log line end up in Elastic Search as an indexable fields. From your php code you can add this extra information to your monolog messages by supplying an associative array as a second argument to the default monolog log functions:

    <?php

    $logger = $this->logger;
    $logger->info('The server was reinstalled', array('bareMetalId' => 123, 'os' => 'centos'));

Happy logging!

Symfony on HHVM 3 and Nginx 1.4 vs PHP 5.5 and Apache 2.4

symfony_hhvm_nginx

Installing Symfony

From symfony.com/download we get the latest (2.4.4) version of Symfony. I have unpacked it and put it in the directory “/home/maurits/public_html”. In “app/AppKernel.php” I moved the “AcmeDemoBundle” to the production section and in “routing.yml” I added the “_acme_demo” route that was originally in “routing_dev.yml”.

Test environment

I tested on my i5-2300 CPU with 16GB of RAM and an SSD. To run a benchmark, I installed on my Ubuntu 14.04 both Apache 2.4 with PHP 5.5 and Nginx 1.4 with HHVM 3. I used Apache Bench (ab) to test the “/demo” path on both web servers. In Apache, I disabled XDebug and enabled Zend OPcache.

Install and configure HHVM 3 with Nginx 1.4 for Symfony

For HHVM, we find pre-built (64-bit) packages listed on Github. This is how you install them on Ubuntu 14.04:

wget -O - http://dl.hhvm.com/conf/hhvm.gpg.key | sudo apt-key add -
echo deb http://dl.hhvm.com/ubuntu trusty main | sudo tee /etc/apt/sources.list.d/hhvm.list
sudo apt-get update
sudo apt-get install hhvm

First we install Nginx from the Ubuntu 14.04 repository using:

sudo apt-get install nginx

Now we configure Nginx using the “normal” FastCGI configuration:

server {
    listen             8080;
    server_name        sf2testproject.dev;

    root /home/maurits/public_html/web;

    location / {
        # try to serve file directly, fallback to rewrite
        try_files $uri @rewriteapp;
    }

    location @rewriteapp {
        # rewrite all to app.php
        rewrite ^(.*)$ /app.php/$1 last;
    }

    location ~ ^/(app|app_dev|config)\.php(/|$) {
        fastcgi_pass 127.0.0.1:9000;
        fastcgi_split_path_info ^(.+\.php)(/.*)$;
        include fastcgi_params;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        fastcgi_param HTTPS off;
    }
}

Single request

When doing a single request and analyzing the response times using the Firebug “Net” panel there is no noticeable difference. This is probably because the threads do not have to compete for CPU. So let’s skip this and do some load testing.

Apache Bench results (Symfony 2.4 / Apache 2.4 / PHP 5.5)

maurits@nuc:~$ ab -c 10 -n 2000 http://sf2testproject.dev/demo/
This is ApacheBench, Version 2.3 <$Revision: 1528965 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking sf2testproject.dev (be patient)
Completed 200 requests
Completed 400 requests
Completed 600 requests
Completed 800 requests
Completed 1000 requests
Completed 1200 requests
Completed 1400 requests
Completed 1600 requests
Completed 1800 requests
Completed 2000 requests
Finished 2000 requests

Server Software:        Apache/2.4.7
Server Hostname:        sf2testproject.dev
Server Port:            80

Document Path:          /demo/
Document Length:        4658 bytes

Concurrency Level:      10
Time taken for tests:   9.784 seconds
Complete requests:      2000
Failed requests:        0
Total transferred:      9982000 bytes
HTML transferred:       9316000 bytes
Requests per second:    204.42 [#/sec] (mean)
Time per request:       48.918 [ms] (mean)
Time per request:       4.892 [ms] (mean, across all concurrent requests)
Transfer rate:          996.36 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.1      0       3
Processing:    18   49  10.9     50      90
Waiting:       15   41  10.1     41      78
Total:         18   49  10.9     50      91

Percentage of the requests served within a certain time (ms)
  50%     50
  66%     54
  75%     56
  80%     58
  90%     62
  95%     65
  98%     69
  99%     73
 100%     91 (longest request)

Apache Bench results (Symfony 2.4 / Ningx 1.4 / HHVM 3)

maurits@nuc:~$ ab -c 10 -n 2000 http://sf2testproject.dev:8080/demo/
This is ApacheBench, Version 2.3 <$Revision: 1528965 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking sf2testproject.dev (be patient)
Completed 200 requests
Completed 400 requests
Completed 600 requests
Completed 800 requests
Completed 1000 requests
Completed 1200 requests
Completed 1400 requests
Completed 1600 requests
Completed 1800 requests
Completed 2000 requests
Finished 2000 requests

Server Software:        nginx/1.4.6
Server Hostname:        sf2testproject.dev
Server Port:            8080

Document Path:          /demo/
Document Length:        4658 bytes

Concurrency Level:      10
Time taken for tests:   4.678 seconds
Complete requests:      2000
Failed requests:        0
Total transferred:      9900000 bytes
HTML transferred:       9316000 bytes
Requests per second:    427.50 [#/sec] (mean)
Time per request:       23.392 [ms] (mean)
Time per request:       2.339 [ms] (mean, across all concurrent requests)
Transfer rate:          2066.52 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.1      0       1
Processing:     8   23  11.5     21      84
Waiting:        8   22  11.3     20      84
Total:          8   23  11.5     21      84

Percentage of the requests served within a certain time (ms)
  50%     21
  66%     26
  75%     30
  80%     32
  90%     39
  95%     46
  98%     54
  99%     58
 100%     84 (longest request)

Conclusion

On both setups, I did not do any optimization. I just installed and ran the benchmark. It seems that Symfony 2.4 on HHVM is about twice as fast as on PHP 5.5. In a real life setup this means you need half the machines. This seem like a good cost reduction at first, but it may not be as good as it looks. I believe that in reality most Symfony applications are doing database and/or API calls. These will not be faster when using HHVM, since HHVM only speeds up PHP execution. This is why I think that the difference will be smaller (than a factor 2) for a real life Symfony application. What do you think?

Symfony2 Guzzle bundle for cURL API calling

The LswGuzzleBundle adds Guzzle API call functionality to your Symfony2 application. It is easy to use from the code and is aimed to provide full debugging capabilities. source: Github

On Packagist, you can see that in the past two years we have published 10 LeaseWeb Symfony2 bundles. The latest and tenth addition to the list is the LswGuzzleBundle. It is an alternative for the LswApiCallerBundle, so it also is a bundle that helps you to call API’s using cURL. The difference is that this bundle uses Guzzle as a wrapper for cURL.

Guzzle profiler and debugger

This bundle also adds a debug panel to your Symfony2, where you can find the cURL API calls Guzzle makes in your application. It provides profiling and extensive debugging capabilities to your Symfony2 application. We feel it is an essential bundle if you create an application that does not work with a database, but instead has an API as data store backend. Guzzle allows you to specify the parameters and and available calls in a Guzzle service description. Since this allows the bundle to understand the structure of the call, it will display the parsed parameters and output. This is an unique feature of this bundle.

guzzle_bundle

Open source

The bundle is MIT licensed. You can find it at:

As always there is a readme documentation on GitHub to get you started.

HipHopVM 2.2.0: slower, but more compatible

We wrote before about HipHopVM, Facebook’s high-performance PHP runtime, that can be found on Github. It is a production-ready virtual machine that delivers superior performance for PHP scripts. Recently version 2.2.0 was released and it now supports many frameworks. This is a big improvement. First we compare it with the previous version and see how it performs. We tested again on Ubuntu 12.04 (the latest LTS release).

hhvm220_graph

This graph shows the average execution time (of 10 runs) on PHP 5.3.10 (with APC), HipHopVM v2.1.0 and HipHopVM v2.2.0. The Y-axis shows the amount of seconds the execution took (on average) and the numbers on the X-axis represent the following benchmark scripts:

  1. bench1.php: http://www.lornajane.net/posts/2012/proof-that-php-5-4-is-twice-as-fast-as-php-5-3
  2. bench2.php: https://coderwall.com/p/il1tog
  3. bench3.php: http://php.net/manual/en/features.gc.performance-considerations.php
  4. bench4.php: http://akrabat.com/php/php-5-3-is-quicker-than-php-5-2-2/
  5. bench5.php: http://onlinephpfunctions.com/benchmarks
  6. bench6.php: http://www.synet.sk/php/en/220-php-performance-benchmark

This is the raw data:

hhvm220_data

Rewriting under HipHopVM

Symfony on Apache web server requires “mod_rewrite”. If you are running Symfony on HipHopVM you are probably looking for a “mod_rewrite” alternative. Fortunately HHVM has this feature built-in. This is how you configure the VirtualHost in the hhvm.hdf file to achieve rewriting:

VirtualHost {
    * {
        Pattern = .*
        RewriteRules {
            * {
                 pattern = ^(.*)$
                 to = index.php$1
                 qsa = true
            }
        }
    }
}

Conclusion

HipHopVM is getting more and more compatible. Still it does not run all frameworks, but is seems it is getting there. It may look like it is getting slower as it is getting more compatible. When you look at benchmark 5 & 6 you can see that it still has a good performance and that is has hardly become slower. The results from benchmark 1 are a bit worrying, but since that is just a simple object instantiation, I would not value that too much.

What do you think about these numbers? Did you get different results? Please share your thoughts in the comments below.