Limit concurrent PHP requests using Memcache

When you run a website you may want to use nginx reverse proxy to cache some of your static assets and also to limit the amount of connections per client IP to each of your applications. Some good modules for nginx are:

Many people are not running a webfarm, but they still want to protect themselves against scrapers and hackers that may slow the website (or even make it unavailable). The following script allows you to protect your PHP application from too many concurrent connections per IP address. You need to have Memcache installed and you need to be running a PHP web application that uses a front controller.

Installing Memcache for PHP

Run the following command to install Memcache for PHP on a Debian based Linux machine (e.g. Ubuntu):

sudo apt-get install php5-memcache memcached

This is easy. You can flush your Memcache data by running:

telnet 0 11211
flush_all

You may have to restart apache for the Memcache extension to become active.

sudo service apache2 restart

Modifying your front controller

It is as simple as opening up your “index.php” or “app.php” (Symfony) and then pasting in the following code in the top of the file:

<?php
function firewall($concurrency,$spinLock,$interval,$cachePrefix,$reverseProxy)
{
  $start = microtime(true);
  if ($reverseProxy && isset($_SERVER['HTTP_X_FORWARDED_FOR'])) {
    $ip = array_pop(explode(',',$_SERVER['HTTP_X_FORWARDED_FOR']));
  }
  else {
    $ip = $_SERVER['REMOTE_ADDR'];
  }
  $memcache=new Memcache();
  $memcache->connect('127.0.0.1', 11211);
  $key=$cachePrefix.'_'.$ip;
  $memcache->add($key,0,false,$interval);
  register_shutdown_function(function() use ($memcache,$key){ $memcache->decrement($key); });
  while ($memcache->increment($key)>$concurrency) {
    $memcache->decrement($key);
    if (!$spinLock || microtime(true)-$start>$interval) {
      http_response_code(429);
      die('429: Too Many Requests');
    }
    usleep($spinLock*1000000);
  }
}
firewall(10,0.15,300,'fw_concurrency_',false);

Add these lines if you want to test the script in stand-alone mode:

session_start();
session_write_close();
usleep(3000000);

With the default setting you can protect a small WordPress blog as it limits your visitors to do 10 concurrent(!) requests per IP address. Note that this is a lot more than 10 visitors per IP address. A normal visitor does not do concurrent requests to PHP as your browser tends to send only one request at a time. Even multiple users may not do concurrent requests (if you are lucky). In case concurrent requests do happen they will be delayed for “x” times 150 ms until the concurrency level (from that specific IP) is below 10. Other IP addresses are not affected/slowed down.

If you use a reverse proxy you can configure this (to get the correct IP address from the “X-Forwarded-For” header). Also if you set “$spinLock” to “false” then you will serve “429: Too Many Requests” if there are too many concurrent requests instead of stalling the connection.

This functionality is included as the “Firewall” feature of the new MindaPHP framework and also as the firewall functionality in the LeaseWeb Memcache Bundle for Symfony. Let me know what you think about it using the comments below.

Share

Open source privacy tools µBlock and µMatrix

In the past we have been giving some attention to the tools Adblock Plus and RequestPolicy when we talked about enhancing your privacy and security online.

“The user decides what web content is acceptable or not in their browser.” – µBlock manifesto

Today we want to introduce you to two alternative (open source) tools for this, made by Raymond Hill: µBlock and µMatrix

µBlock

This is a simple tool, comparable to Adblock Plus, but it is open source and light-weight and very user friendly. This tool uses the approach and block-lists as AdBlock Plus and has the main advantage of being more lightweight as you can see in the graph below:

ublock

Source code and more information: https://github.com/gorhill/uBlock

µMatrix

This tool is comparable to RequestPolicy and is aimed at blocking non first party requests, like RequestPolicy. It has a nice matrix that you can easily click to allow/disallow certain requests. I have found that it works much better as it also allows related 3rd party domains. For instance redditmedia.com is not blocked on reddit.com, where RequestPolicy would block it.

umatrix

Source code and more information: https://github.com/gorhill/uMatrix

Installation

On the following links you find the installable extensions for Chromium and Firefox:

Conclusion

Both tools enhance your security and privacy online. They are easy to use and have great integration with the free Chrome (Chromium) browser. Unfortunately µMatrix is not yet available for Firefox, but this may only be a matter of time. I would highly recommend to use both tools in your Chromium install and I would recommend µBlock also on Firefox.

Share

Ghostery lists Adobe TypeKit as privacy threat

The Internet tracker blocking program Ghostery now lists Adobe TypeKit (a very popular font service) as a privacy threat. I read about this first on WUWT:

I’ve gotten a few complaints this week from some overly paranoid people that say they can’t see WUWT anymore in Firefox, but can in Safari. The problem seems to be related solely to a browser extension called “ghostery” which is somehow flagging Adobe Typekit (used to provide custom fonts on WordPress) as some sort of malware.

Ghostery is not malware blocking software (as you can read on wikipedia). It is software that protects you against tracking while surfing the web and IMHO you are not overly paranoid when you use it. In the comments somebody explains:

Font are very seductive tracking beacons. Honest people who would never consider installing a tracking beacon have no qualms about using served fonts, and there’s no difference between them. There is a lot of ignorance out there regarding data mining.

So maybe Ghostery is not listing Adobe TypeKit by accident? We see with Google Analytics that website owners are happy to pay for analytics with their visitors privacy. The same may apply to fonts (although TypeKit is not free). But before we accuse Adobe, let’s take a look at the Adobe TypeKit privacy policy:

In order to provide the Typekit service, Adobe may collect information about the fonts being served to your website. The information is used for the purposes of billing and compliance, and may include the following: …

So, one thing is for sure: Adobe TypeKit is in fact collecting data while serving fonts. This alone may be reason for Ghostery to block it. I did some research and verified that next to the font files TypeKit is loading a 1 by 1 pixel GIF image that has an URL like this:

http://p.typekit.net/p.gif?s=1&k=sgt5tia&app=&ht=tk&h=wattsupwiththat.com&f=...

In the privacy statement Adobe says they collect data “for the purposes of billing and compliance”, which seems reasonable. Also, the privacy policy has a list of data that they collect. None of the data on the list seems to be invading the privacy of the website visitor. So is this a big fuss about nothing? I’m not sure. If you pay close attention to the wording of the sentence you see that they chose to use “may include”. AFAIK “may include” does not imply “is limited to”. Also this “compliance” is not further specified. What do they need to comply with?

Can Adobe TypeKit be trusted to respect our visitors privacy? Probably they can, but even after reading their privacy policy I’m not 100% sure. What do you think? Should I take off my tin-foil hat?

Share

Block Google and Facebook to improve Firefox privacy

firefoxFirefox, a browser built by the Mozilla foundation, is in my opinion the best browser on the web. It is available all major operating systems including Linux and Android. Unfortunately Firefox is not available for iOS. Firefox is “Committed to you, your privacy and an open Web” and on the Mozilla website they tell us that Firefox is:

  • Trusted: Designed to protect your privacy
  • Flexible: Designed to be redesigned
  • Fast: Faster than ever

On the Firefox privacy page Mozilla says:

We build Firefox with a mission to put you first, above all else.
We do it to keep you in control. We do it so you can browse without worry.
And we do it because no one else will. – Mozilla

I think it is widely accepted (and true) that your privacy is much at risk when you are surfing the Internet. Firefox will protect your privacy (to some extent) if you tell it to, but you do have to tell it to do so. You can do this by clicking the menu button and clicking “Preferences”. This screen has a privacy tab and I strongly recommend you to set the settings as strict as shown on the screenshot below:

privacy

Pay extra attention to the “Accept third-party cookies” and “Tell sites that I do not want to be tracked” options. Unfortunately this last feature just informs any third party of your preference, but it does not actually block the tracking. This is where AdBlock Plus comes into play.

ad_block_plus_logo

Download Adblock Plus here. After installing you can configure the AdBlock Plus icon (red stop sign) to be present in the toolbar (or not) by clicking the menu icon, clicking “Add-ons” and then the “Extension” tab on the left and then the “Preferences” button of AdBlock Plus. On the bottom there is a list of checkboxes and one is “Show in toolbar”.

adblockplus

It is very convenient to have the AdBlock Plus icon in the toolbar (left from the menu icon) so that you can quickly disable it if that is needed. It may for instance happen that a site no longer shows you Facebook “Like” buttons and you are very desperate to “Like” something.

ad_block_plus_settings

For the best experience I would disable “Show tabs on Flash and Java” and disable “Count filter hits”. In the filter preferences I have added three subscriptions and unchecked “Allow some non-intrusive advertising”, like this:

adblock_filter_preferences

Most people install only “EasyList”, which is easy to find and mainly blocks advertisers.  I also recommend the “Adblock warning Removal List” to avoid any warnings that may appear due to the usage of AdBlock Plus. The other subscription you should have is “Fanbox’s Annoyance List” which sounds unimportant, but actually blocks all Google and Facebook tracking (and many other “annoying” things). I also use “EasyPrivacy”, which also blocks privacy threats. These subscriptions may not be available from the user interface, but this should not stop you. You can find them on the following link:

https://easylist.adblockplus.org/en/

AdBlock Plus will block the loading of elements that match the rules that are defined in the subscriptions. These elements can be visible or invisible (scripts or transparent tracking pixels). This does not only improve your privacy online, but also makes websites load faster. It actually matters a lot as you can see from a quick experiment I did using Firebug. I executed a full page refresh on several websites with and without AdBlock Plus enabled. Below a graph showing the loading time of the website with AdBlock Plus enabled compared to the loading time of the website without AdBlock Plus enabled. You can see that all sites load faster with AdBlock Plus enabled (<100%), since the browser has to load less elements from the website:

website_loading_times

This is the data I collected in my (single) run along some popular websites, which is used to draw the above graph:

           website   total   onload   total ABP   onload ABP   total   onload
washingtonpost.com   12.06     6.98        5.12         4.72     42%      68%
       nytimes.com   11.35     5.72        6.84         4.28     60%      75%
             nu.nl    5.17     4.07        2.29         1.63     44%      40%
     microsoft.com    3.41     2.85        2.69         2.15     79%      75%
   mail.google.com   10.19     1.15        8.47         1.12     83%      97%
        google.com    1.58     1.06        0.89         0.84     56%      79%
           cnn.com    9.48     5.45        3.09         2.17     33%      40%
           bbc.com    3.42     3.05        2.09         1.82     61%      60%

So the bottomline is this: by protecting your privacy better, surfing the Internet will go faster. This is a well-kept secret that I share with you “because no one else will.”

About privacy and the ethics of blocking ads

Some people argue that you should not install ad blocking software, because blogs can exist because banners bring income to the writers. Although I doubt that this is true (direct advertising and editorials pay a magnitude better and cannot be blocked), I want to focus on the opposite: websites stealing from their visitors. By using “free analytics”, “like buttons”, “JavaScript-driven ad engines” and “web-shop tracking” many, if not most, websites are sharing very sensitive (privacy related) information about their visitors with third parties (without the visitors consent). This information can be stored and used to identify and profile visitors. The bad thing is that many site owners do not even realize their behavior is unethical (and in some cases even forbidden by law). In my opinion this unethical behavior makes using blocking software ethical.

Share

Browse safer by disabling SSLv3 in Firefox

vulnerable poodle sslv3

You may be at risk! A man-in-the-middle attack may be effective between you and any site that runs on HTTPS. This is explained two days ago by Google in their publication about the POODLE attack. It explains that SSLv3 has a vulnerability and negotiation of this protocol can be enforced by a man-in-the-middle. That man-in-the-middle is able to read (part of) the plaintext of your secure communication with the server. You can click the above image (that links to https://www.poodletest.com/) and if you are vulnerable you will see a poodle.

Fixing the vulnerability is also very easy. If you run a server you may want to check out my post on fixing the POODLE issue in Nginx and Apache. Even transfers from browsers that are not fixed can then no longer be intercepted  and decoded by a man-in-the-middle.

firefox poodle fix

But you should also fix this issue in your browser right now! In Firefox you simply type “about:config” in the address bar and then “tls” in the search bar. Change the value of “security.tls.version.min” from “0” to “1” as the above screenshot illustrates:

Mozilla says that it is making Firefox 34 safe from POODLE by disabling SSLv3 by default. – betanews.com

This change is so easy (only costs a few seconds and requires a browser restart) that I would not wait for Mozilla to release Firefox 34. If you run another browser, and you are looking for a guide, you may want to check out tomsguide.com.

Share