“HTTP secure link” using OpenResty Nginx module

Whenever you generate a link on your website to let your visitor download some content, you run into the risk that people will start sharing that link with others. If you want to discourage that you can require the user to login before downloading. If this is not an option for you then you can generate links that are only valid for a few seconds. This is a feature that Nginx supports using the Nginx module HttpSecureLinkModule.

Standard HttpSecureLinkModule

You can specify a Unix timestamp on which the link should no longer be valid; you add this timestamp to the query string like this:


In this URL (that uses ‘GET’ parameters) the ‘e’ parameter is the expiry timestamp and the ‘s’ parameter is the signature. This signature is calculated by taking a secret and appending the expiry timestamp to it and calculating a hash over that.

$signature = md5( $secret . $path . $expiry );

OpenResty Nginx modules

But what if you want to use a variant on that? For instance, if you want to restrict on certain networks or use a different hashing algorithm for the signature? My idea was to use the excellent building blocks from the set-misc-nginx-module that is made by Yichun Zhang (a.k.a. “agentzh”). This Nginx module is not included in Nginx by default, but is a part of the OpenResty Nginx distribution or as Yichun Zhang explaines:

By taking advantage of various well-designed Nginx modules, OpenResty effectively turns the nginx server into a powerful web app server. OpenResty is not an Nginx fork. It is just a software bundle. – openresty.org

We did 3 pull requests to the “set-misc-nginx-module” that allow for the needed functionality. You can compile this Nginx module into your Nginx by pointing the “–add-module” configure option to your checkout of this module. We have added the “set_encode_base64url”, “set_decode_base64url”, “set_expired” and “set_ip_matches” directives for building a custom secure link Nginx module.

Installation instructions

The configuration for the Nginx module would be like this:

# GET /?e=1893452400&n=MC4wLjAuMC8w&s=QVvNrcn-j4smvOFhuPTUU7EgoI8
# returns 403 when signature on the arguments is not correct OR
# when expire time is passed or network does not match.
# It is an alternative to the HttpSecureLinkModule in Nginx.
# This example has expiry "2030-01-01" and network "".
location / {
  set_hmac_sha1 $signature 'secret-key' "e=$arg_e&n=$arg_n";
  set_encode_base64url $signature;
  if ($signature != $arg_s) {
    return 403;
  set_expired $expired $arg_e;
  if ($expired) {
    return 403;
  set_decode_base64url $network $arg_n;
  set_ip_matches $ip_matches $network $remote_addr;
  if ($ip_matches = 0) {
    return 403;
  root /var/www;

Where the PHP code to generate the link would be:

function base64url_encode($s)
{ return rtrim(strtr(base64_encode($s),"+/", "-_"),"=");
$secret = "secret-key";
$expiry = strtotime("2030-01-01 00:00:00");
$network = base64url_encode("");
$query = "e=$expiry&n=$network";
$signature = base64url_encode(hash_hmac('sha1',$query,$secret,true));
$url = "http://localhost/?$query&s=$signature";
echo "$url\n";

Output would be:


Applying a SHA 256 HMAC to an AES 256 cipher in PHP

In a previous post we have looked at encrypting data using AES in PHP. Today we look at how to ensure integrity of the message when encrypting. Doing AES one could simply send the following over the wire (using a randomized IV for every message):

Message = IV || AES-256-Encrypt(Plain-Text)

But this scheme does not allow you to see if there is a corruption in the message. Only if the plaintext is human language, you can see that it does not make sense. But then you have to rely on meta-knowledge about the structure of the plaintext. If you want to automatically detect tampering you need to add a cryptographic hash. A naive way of doing this would be to add a plain SHA-256 like this:

Message = IV || AES-256-Encrypt(Plain-Text || SHA-256(Plain-Text))

But if you have read a little bit about cryptography, you would probably know that you should be using a MAC (Message Authentication Code). MAC algorithms require two parties to share a secret key. MAC algorithms based on cryptographic hash functions (like SHA-256) are called HMAC and are defined in rfc2104. In PHP you can calculate a HMAC using the function “hash_hmac“. It would be convenient, but wrong, to use the same key for the AES and the HMAC and do:

Message = IV || AES-256-Encrypt(Plain-Text || SHA-256-HMAC(Plain-Text))

In the above scheme you take then plaintext and calculate a SHA-256-HMAC over it. You add the result to the end of the message. You encrypt the entire thing with AES-256 using a random IV that you prepend to the message.

When using AES in CBC (Cipher-block chaining) mode each ciphertext block depends on all plaintext blocks processed up to that point. That is why you also need a random IV (Initialization vector) for the first block.

If you tamper with data it has an effect on all the next blocks. When adding a HMAC with the size of a block to the end of the message one would think that this would provide good security. Or as Nick is saying on Scientopia:

Couldn’t you simply append a secure hash of the preceding plaintext to each message? Unless the attacker can predict the effect of his changes (which shouldn’t be possible), he shouldn’t be able to modify the message _and_ the hash in such a way that it remains undetectable, right?

On the other hand Nate from Root Labs is saying:

For schemes like CBC where there is some redundancy, there are a number of cases where the propagation behavior of CBC does not prevent an attack.

I am not a cryptographer, so I cannot argue why the above schemes are wrong (this article might). I did learn that the correct way of applying the HMAC is in fact very simple: First you apply the AES encryption and then you apply the HMAC independently on the IV and crypted message to ensure neither has been tampered with. It looks like this:

Cipher-Text = IV || AES-256-Encrypt(Plain-Text)
Message = Cipher-Text || SHA-256-HMAC(Cipher-Text)

This scheme has the downside that you will need two keys: one for the HMAC and one for the AES encryption. Note that those keys should be unrelated. To illustrate how the correct scheme works I implemented it (in PHP) below:

// define your key,use length 32,48 or 64 bytes for AES-128,AES-192 and AES-256
$key1 = pack("H*","7206ea04845d1f740401a6e6b2e52d22ac2d314a3d138eb8f61ebe5fb1d6aa5d");
// define the key for the SHA-256 HMAC (both keys should be random shared secrets)
$key2 = pack("H*","dee5cb4a2c4b5b61a729337fc9153daf6387a54b765877752ca36b44a721f469");

// hash comparison function to avoid timing (side channel) attacks
function hash_equals($a, $b) {
  if (!is_string($a) || !is_string($b)) return false;
  if (strlen($a) !== strlen($b)) return false;
  $match = 0;
  for ($i = 0; $i < strlen($a); $i++) {
    $match |= ord($a[$i]) ^ ord($b[$i]);
  return $match===0;

// define the plain text
$text = "123";
// get the block size for the padding
$block_size = mcrypt_get_block_size(MCRYPT_RIJNDAEL_128,MCRYPT_MODE_CBC);
// get the length of the padding needed
$pad = $block_size - (strlen($text) % $block_size);
// pad the plain text with the padding character using PKCS7 padding
$text .= str_repeat(chr($pad),$pad);
// get the size of the initialization vector
$iv_size = mcrypt_get_iv_size(MCRYPT_RIJNDAEL_128,MCRYPT_MODE_CBC);
// create a random initialization vector
$iv = mcrypt_create_iv($iv_size,MCRYPT_DEV_URANDOM);
// encode the plain text to get the cipher text
$crypt = mcrypt_encrypt(MCRYPT_RIJNDAEL_128,$key1,$text,MCRYPT_MODE_CBC,$iv);
// calculate the SHA-256 HMAC over the initialization vector + cipher text
$hmac = hash_hmac("sha256",$iv.$crypt,$key2,true);
// add the SHA-256 HMAC to the message
$output = bin2hex($iv.$crypt.$hmac);
// print output displaying the process
echo "$text\n";

// tamper with the data
echo "$output\n";
$output = bin2hex(chr(ord(pack("H*",substr($output,0,2))) ^ 1<<3)).substr($output,2);
echo "$output\n";

// read the data back
$message = pack("H*",$output);
// get the size of the initialization vector
$iv_size = mcrypt_get_iv_size(MCRYPT_RIJNDAEL_128,MCRYPT_MODE_CBC);
// extract the initialization vector from the message
$iv = substr($message,0,$iv_size);
// get the size of the SHA-256 HMAC
$hmac_size = strlen(hash_hmac("sha256","","",true));
// extract the SHA-256 HMAC from the message
$hmac = substr($message,-$hmac_size);
// extract the cipher text from the message
$crypt = substr($message,$iv_size,-$hmac_size);
// calculate the correct SHA-256 HMAC
$crypt_hmac = hash_hmac("sha256",$iv.$crypt,$key2,true);
// check whether the HMACs are equal
if (!hash_equals($hmac,$crypt_hmac)) die("HMAC ERROR\n");
// decode the cipher text to get the plain text
$text = mcrypt_decrypt(MCRYPT_RIJNDAEL_128,$key1,$crypt,MCRYPT_MODE_CBC,$iv);
// get the length of the plain text padding
$pad = ord($text[strlen($text)-1]);
// remove the PKCS7 padding from the plain text
$text = substr($text,0,-1*$pad);
// print output displaying the process
echo "$text\n";

The output of the above code is:


Or if you comment out the tamper line (41):


Lessons learned implementing AES in PHP using Mcrypt

The Advanced Encryption Standard (AES) is the successor of triple DES. When you need a standardized, secure, high performance symmetric cipher it seems like a good choice. Wi-Fi network traffic is encrypted with AES for instance. Also when you want to securely store data in a database or on disk you could choose AES. Many SSDs store data internally using AES encryption. PHP supports AES through “mcrypt”. On Debian based systems (like Ubuntu and Mint) you can install it using “sudo apt-get install php5-mcrypt”.

Rijndael-256 is not AES-256

In Mcrypt there is no cipher called “AES”, but Mcrypt supports “Rijndael”. Note that AES is based on the Rijndael cipher. So can we just use Rijndael for AES-128, AES-192 and AES-256? It seems straight forward since Mcrypt supports Rijndael-128, Rijndael-192 and Rijndael-256. But be aware of this pitfall:

AES-256 is different from RIJNDAEL-256. The 256 in AES refers to the key size, where the 256 in RIJNDAEL refers to block size. AES-256 is RIJNDAEL-128 when used with a 256 bit key. – source

So remember that only “Rijndael-128” in “Cipher-block chaining” (CBC) mode is defined as the Advanced Encryption Standard (AES).

You can see this in effect in the following code:

echo "RIJNDAEL 128 CBC\n";
$max_key_size = mcrypt_get_key_size(MCRYPT_RIJNDAEL_128, MCRYPT_MODE_CBC)*8;
echo "Max key size: $max_key_size\n";
$block_size = mcrypt_get_block_size(MCRYPT_RIJNDAEL_128, MCRYPT_MODE_CBC)*8;
echo "Block size: $block_size\n";

This is the output:

Max key size: 256
Block size: 128

Key padding woes

PHP is a very forgiving language. It allows you to define a variable containing a string and use it as an integer without a single warning. The Mcrypt functions handle different key sizes in the same forgiving way. Only 128, 192, and 256 bit keys are accepted in Rijndael-128, but if you feed Mcrypt a smaller key it will automatically pad it to an acceptable size using “zero” bytes. This is shown in the example below:

$text = "http://xkcd.com/1323/";
$iv = mcrypt_create_iv(mcrypt_get_iv_size($enc, $mode), MCRYPT_DEV_URANDOM);
$keys = array();
// smaller than 128 bit key gets padded to 128 bit
$keys[] = pack('H*', "00112233445566778899aabbccddee");
$keys[] = pack('H*', "00112233445566778899aabbccddee00");
// larger than 128 bit key gets padded to 192 bit
$keys[] = pack('H*', "00112233445566778899aabbccddeeff00");
$keys[] = pack('H*', "00112233445566778899aabbccddeeff0000000000000000");
// larger than 192 bit key gets padded to 256 bit
$keys[] = pack('H*', "00112233445566778899aabbccddeeff001122334455667700");
$keys[] = pack('H*', "00112233445566778899aabbccddeeff00112233445566770000000000000000");
// larger than 256 bit key will be truncated (with a warning)
$keys[] = pack('H*', "00112233445566778899aabbccddeeff00112233445566778899aabbccddeeff00");
foreach ($keys as $key)
{ echo bin2hex(mcrypt_encrypt($enc, $key, $text, $mode, $iv))."\n";

The output of the above script:

PHP Warning:  mcrypt_encrypt(): Size of key is too large for this algorithm on line 19

Null padded strings

Because AES is a block cipher it cannot encrypt and decrypt a three byte plain text. Any plain text should be a multiple of the block size (which is 16). Again Mcrypt accepts any size of plain text and solves this “issue” for you by automatically padding it with zero bytes. Just like it does with the key. When decoding the plain text this may lead to some confusion. You can see this illustrated in the code below:

$key = "secret";
$iv = mcrypt_create_iv(mcrypt_get_iv_size($enc, $mode), MCRYPT_DEV_URANDOM);
$text = "123";
$crypt = mcrypt_encrypt($enc, $key, $text, $mode, $iv);
$text = mcrypt_decrypt($enc, $key, $crypt, $mode, $iv);

That produces:

string(16) "123"
string(32) "31323300000000000000000000000000"
string(3) "123"
string(6) "313233"

In the above example the decrypted plain text seems to be padded with zero bytes. This is not true. It is actually the plain text, when fed to the cipher, that was padded with zero bytes. This makes the plain text after decrypting equal to the plain text before encrypting.

As you can see above a simple “strtok” call can remove everything from the first zero byte on from the string. NB: You should only do this if you are sure the original plain text does not contain any zero bytes. If it can contain zero bytes then make sure your plain text length is always a multiple of 16 and no padding occurs.

Ignored initialization vector

If the initialization vector does not have the right size (16 bytes), then Mcrypt does not pad it. Instead it outputs a warning and uses an empty initialization vector. This empty initialization vector is a series of 16 zero bytes. This behavior is shown in the code example below:

$key = "secret";
$iv = "short_iv";
$text = "123";
$crypt = mcrypt_encrypt($enc, $key, $text, $mode, $iv);
$iv = pack('H*', "00000000000000000000000000000000");
$text = mcrypt_decrypt($enc, $key, $crypt, $mode, $iv);

The output of the above code is:

PHP Warning:  mcrypt_encrypt(): The IV parameter must be as long as the blocksize on line 7
string(16) "123"

So make sure you turn on warnings when using Mcrypt, it might hold some very valuable information.