5 crucial optimizations for SSD usage in Ubuntu Linux

msataI bought my first SSD more than 5 years ago (late 2007), for my white MacBook Core2Duo 2.0 Ghz. It may be needless to say that my MacBook ran faster than many MacBook Pro’s at that time. The look on peoples faces when they saw my MacBook boot faster than their MacBook Pro was unforgettable.

By that time I was also experimenting with running 2 x 8GB CompactFlash with SATA converters in RAID-0 mode in my completely silent (but old and under-clocked) Pentium 3 Linux desktop. When people would visit my home they couldn’t believe my PC was actually turned on: it was dead silent.

Even today I still enjoy building low-power, silent and ultra-fast (Linux) computers with an SSD as the primary drive. Nowadays the new mSATA standard (see picture) is being used. During the past years I learned a thing or two about configuring Linux on a SSD drive. Today I will share those things with you:

1) Buy more than enough RAM

Yep, you should have at least 16GB of RAM. I know you’ll say that you don’t use that, but that is exactly the point. If you would be using it, it would not be enough. And why not? RAM is relatively cheap these days and you don’t want your system to start swapping, since this will wear out your expensive SSD.

2) Use the default partition layout

Contrary to popular belief, the default (guided) partitioning will do. If you partition manually, keep in mind that you do need a swap partition that is bigger than your RAM size.

3) Avoid using swap

You have plenty of RAM, so you could turn off swap or even create a system without swap. I have been running without swap for a year or so and never experienced significant problems (other people say the same). The most annoying side-effect is that the hibernate function is not available, since it uses the swap partition. This is why I recommend to not completely disable swap, but to reduce the swappiness of Linux. This way Linux will use all available RAM before starting to write to your precious SSD, while still allowing you to hibernate. Still, be aware when it comes to using hibernate, because it uses a lot of writes. Run this one-liner to add a permanent swappiness setting:

echo -e "vm.swappiness=0" | sudo tee -a /etc/sysctl.conf

This will only be effective after reboot.

4) Disable access time logging

Remove the access time logging on the file system. Without the “noatime” flag on your file system every read will cause a write, because the file system will update the access time. This is bad for the life-time of your SSD, since it supports a limited number of writes and this is causing significantly more writes. Edit the “fstab” to add the “noatime” flag:

sudo nano /etc/fstab

Now change “errors=remount-ro” to “noatime,errors=remount-ro”. Save the file and reboot.

5) Enable TRIM

TRIM will stop your SSD from slowing down after using it for a while. If you do not enable TRIM the writes may become slower due to the erasing of the deleted blocks. By running this one-liner you can add TRIM as a daily cron job:

echo -e "#\x21/bin/sh\\nfstrim -v /" | sudo tee /etc/cron.daily/trim

You have to make the cron job executable using:

sudo chmod +x /etc/cron.daily/trim

It will be run every day to avoid slowing down your writes and you will hardly notice it.

More advice: monitor the expected life-time

Your disk will not run forever. It has an expected life-time and you should probably monitor it using:

sudo smartctl -data -A /dev/sda

This will output something like this:

maurits@nuc:~$ sudo smartctl -data -A /dev/sda
smartctl 5.41 2011-06-09 r3365 [x86_64-linux-3.8.0-26-generic] (local build)
Copyright (C) 2002-11 by Bruce Allen, http://smartmontools.sourceforge.net

=== START OF READ SMART DATA SECTION ===
SMART Attributes Data Structure revision number: 18
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
1   Raw_Read_Error_Rate     0x0000   006   000   000    Old_age   Offline      -       6
3   Spin_Up_Time            0x0000   100   100   000    Old_age   Offline      -       0
4   Start_Stop_Count        0x0000   100   100   000    Old_age   Offline      -       0
5   Reallocated_Sector_Ct   0x0000   100   100   000    Old_age   Offline      -       0
9   Power_On_Hours          0x0000   100   100   000    Old_age   Offline      -       2592
12  Power_Cycle_Count       0x0000   100   100   000    Old_age   Offline      -       258
232 Available_Reservd_Space 0x0000   100   100   000    Old_age   Offline      -       4914564640
233 Media_Wearout_Indicator 0x0000   100   000   000    Old_age   Offline      -       100

maurits@nuc:~$

At ID# 233 you see the MWI. This is a value starting at 100 and when it reaches values below 10 you should start to worry. Note that different manufacturers may have different names and numbers for this indicator.

Final advice: avoid buying TLC drives

Triple Level Cell (TLC) drives are making their entry in the market. These drives have a dramatically lower expected life-time. The life-time of a drive is measured in Program/Erase (PE) cycles or in Peta Bytes Written (PBW). Check the specs carefully to avoid buying a cheap drive, thinking you made a good deal, while actually buying a model with a lower expected life-time.


Share

Simple firewall in Ubuntu with UFW

The default firewall configuration tool for Ubuntu is UFW. Developed to ease iptables firewall configuration, UFW provides a user friendly way to create an IPv4 or IPv6 host-based firewall. — Ubuntu community documentation

By default UFW is disabled. Although it might not be strictly needed to run a firewall in all cases, it is good advice for most users. UFW can be configured to allow all outgoing traffic and deny all incoming traffic. This is the “normal” operation mode for desktop PC’s. In general, it is advisable to run a firewall, so that if you are mis-configuring and opening up a port, your firewall will protect you. This is especially relevant when your machine is a laptop that you use on other people’s WiFi networks or when your network supports IPv6. If you run a web server and you want to open up ports to allow incoming traffic you can configure UFW using either a GUI (graphical user interface) or the CLI (command line interface).

Using the GUI to configure UFW

The Gufw GUI for UFW can be installed by executing the following simple command:

sudo apt-get install gufw

In the GUI you can go to “Edit” and “Preferences” to turn off or adjust the logging levels. The preferences pane also allows you to toggle the listening applications list. This is a nice overview, but not as powerful as the output of the commands “netstat -plant” and “ps aux”.

Gufw_1
Figure 1: A screenshot from the Gufw GUI for “ufw”

Configure UFW using the CLI

Normally, I advise desktop users to use a GUI for configuring the software, but the “ufw” CLI is so easy-to-use (or uncomplicated) that you might prefer it. The commands you typically have to type at the prompt are:

sudo ufw enable
sudo ufw logging off
sudo ufw status verbose

First we execute the “ufw enable” command to enable the firewall. Second we issue the “ufw logging off” command to prevent log lines in “/var/log/syslog” when connections are denied. Last we run the status command to check whether the firewall is running with the right configuration. Note that if you want to start all over again and wish to throw the configuration away you can run the “ufw reset” command. By default the enabled UFW will deny incoming and allow outgoing traffic.

ufw2
Figure 2: A sample of ufw log lines that show up in /var/log/syslog

Check firewall status

If you want to make sure the effective firewall rules are correct you can run the following command:

maurits@nuc:~$ sudo ufw status verbose
Status: active
Logging: off
Default: deny (incoming), allow (outgoing)
New profiles: skip
maurits@nuc:~$

Allow some (incoming) traffic

If you are running Apache (or Nginx) to serve HTTP (port 80) traffic from your box to your network or even the Internet, then allow it like this:

maurits@nuc:~$ sudo ufw allow 80
Rule added
Rule added (v6)
maurits@nuc:~$

Remove a rule

If you want to delete a rule, just prefix the rule with the word “delete” like this:

maurits@nuc:~$ sudo ufw delete allow 80
Rule deleted
Rule deleted (v6)
maurits@nuc:~$

Remove a rule by number

You can also identify and delete a rule using a (sequence) number. First use the “numbered” suffix on the “status” command to list the rules with their numbers, like this:

maurits@nuc:~$ sudo ufw status numbered
Status: active

To                         Action      From
--                         ------      ----
[ 1] 80                         ALLOW IN    Anywhere
[ 2] 80                         ALLOW IN    Anywhere (v6)

maurits@nuc:~$

Then execute the command for deletion, like this:

maurits@nuc:~$ sudo ufw delete 2
Deleting:
 allow 80
Proceed with operation (y|n)? y
Rule deleted (v6)
maurits@nuc:~$

Note that every time you delete a rule, all other sequence numbers might change.

Share

Upgrade Ubuntu 12.04 LTS kernel to 3.11

I have various high-end PC’s, because I am really fond of lightning-fast hardware. But my development machines sometimes have issues with older kernels, because the drivers in the Ubuntu 12.04 LTS kernels that I run are outdated and do not support the kick-ass and brand-new hardware.

The reason I run 12.04 LTS is that most servers (I work with) run that version. And by running the latest long-term-stable on your development environment you can avoid writing software that cannot run on your production system, because it would simply not work there as well.

Even when you are limited to Ubuntu 12.04 LTS there are still many things you can choose. One thing you can tweak is your window manager. You can either run Ubuntu (using Unity), Xubuntu (with XFCE), Lubuntu (LXDE) or Kubuntu (KDE). You may run whatever variant you prefer, but Linus uses XFCE. My preference also goes to XFCE, and more particular Xubuntu, since it is lightweight and traditional in its layout.

The other thing you can tweak is the kernel version. You can upgrade to newer kernels if needed, even though these newer kernels are officially not supported on Ubuntu 12.04 LTS. The kind of issues that may be caused by having an old kernel include:

  • Not able to recover from standby (outdated power management support)
  • High power usage or noisy fan (also due to outdated power management support)
  • WiFi card not detected or other wireless issues (outdated WiFi chipsets support)
  • Random freezes or other stability issues (outdated motherboard chipsets support)
  • Crashing window manager or other video issues (outdated video chipsets support)

NB: If your USB device or PCI card is not recognized it may be sufficient to update your PCI and USB id’s with the following command:

sudo update-pciids && sudo update-usbids

The reason to not update to the latest (13.10) Ubuntu is that apart from the kernel, also the software packages will be updated. To get the best of both worlds, you would want to update your kernel, but not update the software packages. The good news is that you can. To get the latest kernel you can either “use the package manager”, “download and install” or “compile it yourself”. This post will cover the easiest of all the options, which is “use the package manager”.

Use the package manager to get a new kernel

To list all Linux kernel meta packages in Ubuntu, execute:

maurits@nuc:~$ apt-cache search kernel | grep linux-image | grep -v "\-3\."
linux-image - Generic Linux kernel image.
linux-image-extra-virtual - Linux kernel extra modules for virtual machines
linux-image-generic - Generic Linux kernel image
linux-image-server - Linux kernel image on Server Equipment.
linux-image-virtual - Linux kernel image for virtual machines
linux-image-generic-pae - Generic Linux kernel image
linux-image-lowlatency - lowlatency Linux kernel image
linux-image-lowlatency-pae - lowlatency Linux kernel image
linux-image-current-generic - Depends on the most recently released generic kernel image.
linux-image-generic-lts-quantal - Generic Linux kernel image
linux-image-generic-lts-raring - Generic Linux kernel image
linux-image-generic-lts-saucy - Generic Linux kernel image
linux-image-hwe-generic - Depends on the generic hardware enablement kernel image and headers.
maurits@nuc:~$ 

To remove all previously installed kernels:

sudo apt-get purge linux-image-* linux-headers-*

To install the latest 3.2 kernel:

sudo apt-get install linux-image-generic linux-headers-generic

To install the latest 3.5 kernel:

sudo apt-get install linux-image-generic-lts-quantal linux-headers-generic-lts-quantal

To install the latest 3.8 kernel:

sudo apt-get install linux-image-generic-lts-raring linux-headers-generic-lts-raring

To install the latest 3.11 kernel:

sudo apt-get install linux-image-generic-lts-saucy linux-headers-generic-lts-saucy

After rebooting, kernel 3.11 should be loaded. To check the kernel version after rebooting, open a terminal and type “uname -a”.

maurits@nuc:~$ uname -a
Linux nuc 3.11.0-13-generic #20~precise2-Ubuntu SMP Thu Oct 24 21:04:34 UTC 2013 x86_64 GNU/Linux
maurits@nuc:~$ 

That was easy right? Updates should automatically be installed. I hope it solves your Linux hardware issues as well.

Share

Daily MySQL off-site backups using logrotate and rsync

For daily MySQL off-site backups I was looking for a simple approach. Scott Miller’s minimal, but concise post on MySQL Backups With Logrotate  lead me to the unusual choice of logrotate to backup MySQL. What Scott was not doing, and I did need, was off-site backup. For this I had set up rsync access to the machine with public key authentication.

After setting up the daily rsync with some help from this post of Uwe Hermann I found out that the logrotate files were all transfered every time rsync ran, so not only the new ones. This turned out to be due to the default naming scheme of logrotate (appending sequential numbers). Fortunately I found a post by Eric Thern titled logrotate tweaks to enhance rsync backups that explains that one can use the “dateext” option to make the logrotate files “rsync friendly”.

The last thing I had to find out is when the logrotate task was fired to choose a good time for my cronjob that fired the rsync. I found a post on serverfault that showed me how to find out when the daily cron tasks were started. It turned out they were started at 6:25 AM on my Debian based machine. I scheduled the cronjob on the remote server to rsync at 9:00 AM to be sure the backup was finished.

Instructions

I will go through all the files I modified on the production server and the (off-site) backup server. On the production server I created the file “/etc/logrotate.d/mysql-bkup”:

/home/maurits/bak/project.sql.gz {
daily
dateext
rotate 8
nocompress
create
postrotate
mysqldump -uproject -pproject project > /home/maurits/bak/project.sql --single-transaction
gzip -9f /home/maurits/bak/project.sql
chown maurits:maurits /home/maurits/bak/project.sql.gz
chmod 640 /home/maurits/bak/project.sql.gz
endscript
}

As you can see the logrotate is run daily, with the dateext option for better rsync efficiency. It keeps 8 backups (7 days) and does not apply compression (we do that manually). The create option will make sure new files are created with the same permissions and owner as the previous ones. We finish with the “postrotate” script that does a mysqldump, gzip and then sets a proper owner and permissions. To test run the script you can execute:

sudo logrotate -f /etc/logrotate.d/mysql-bkup

After running it we can check the created files:

maurits@nuc:~$ ls bak -al
total 18192
drwxr-xr-x  2 maurits maurits    4096 Jun 22 15:17 .
drwxr-xr-x 10 maurits maurits    4096 Jun 22 14:54 ..
-rw-r-----  1 maurits maurits 9291115 Jun 22 15:17 project.sql.gz
-rw-r-----  1 maurits maurits 9291114 Jun 22 15:06 project.sql.gz-20130622

Now we login to the (off-site) backup server. Here we need to configure the cronjob to retrieve the logrotated MySQL backups using rsync. We add the cronjob by editing the crontab with:

maurits@nuc:~$ crontab -e

Then we add the rsync command to the crontab. Mine looks like this:

# m h  dom mon dow   command
0 9 * * * rsync -a --delete-during www.project.nl:bak ~

As you can see we configured that at minute 0 of hour 9 on every day of the month, of every month, on any day of the week we run the rsync command. This is all there is to it. If you have any questions, remarks or improvements, please use the comments section below.

Share

Stream Spotify via Airplay to your Linux HTPC

Today we will show you how to setup your Linux HTPC as AirPlay target so that you can stream music services like Spotify from your iPhone or iPad to it. A Home Theater PC (HTPC) or Media Center computer is a PC connected to a TV and/or audio system that supports video, photo, audio playback, and sometimes video recording (PVR) functionality.

Spotify is a commercial music streaming service providing DRM-protected content from a range of major and independent record labels, including Sony, EMI, Warner Music Group and Universal. Today (June 21st 2013) Spotify is available in: Australia, Austria, Belgium, Denmark, Finland, France, Germany, Netherlands, New Zealand, Norway, Spain, Sweden, Switzerland, UK, US, Ireland, Andorra, Liechtenstein, Monaco, Luxemburg. It has very nice “app” for the iPad or iPhone to select and play music.

Alternatives to Spotify are:


If you have both a Linux HTPC and a (paid) Spotify premium account you can combine them into an awesome entertainment system. The convenient part is that you will use your iPhone/iPad to choose music that will be streamed over the Wifi using the AirPlay protocol to your HTPC and then to your sound system. I assume your home network/living room setup looks like this:

shairtunes_network

What we are going to do is add a service to the HTPC that allows us to play music on the stereo system. We are going to be using the AirPlay protocol that is created by Apple and allows you to play stream audio from an Apple device towards speakers or a stereo line-out. The first device to support AirPlay protocol was the AirPort express home Wifi router by Apple. Nowadays you can get a really cool portable speaker or regular speaker with Wifi support that allow you to stream directly to them using the AirPlay protocol.

Even if you are very Linux minded it could happen that you get an iPhone from your boss. In that case you might not be willing to invest in Apple (related) products like the speakers mentioned above. What you can do is install “shairport”: a software package that allows your free Linux machine to behave like an AirPlay target. This way you can enjoy music from your iPhone on your speakers via your Linux HTPC. You need to install the dependencies, download the code, compile and install it. This sounds like a lot of work, but it is actually not that hard:

sudo apt-get install build-essential libssl-dev libcrypt-openssl-rsa-perl libao-dev libio-socket-inet6-perl libwww-perl avahi-utils pkg-config
sudo cpan install HTTP::Request HTTP::Message Crypt::OpenSSL::RSA IO::Socket::INET6 Net::SDP
wget https://github.com/abrasive/shairport/archive/master.zip
unzip master.zip
cd shairport-master/
make
perl shairport.pl
sudo make install

Now all you need to do is start the “shairport” service and you are ready to enjoy your new audio setup:

sudo /etc/init.d/shairport restart

Enjoy the music!

Share