FreeBSD Patch Accepted!

It was an emotional moment last week when I had a patch (code change) accepted by the FreeBSD operating system. It was a relatively simple change to the dns/noip port’s post-install message to clarify the port’s configuration and use (a port is the scaffolding code that integrates a third-party application into FreeBSD, and provided as part of FreeBSD).

FreeBSD is a free Unix-like operating system, most commonly found on servers in backrooms and data centers. It is similar to Linux, except that FreeBSD is a direct open source derivative of the original AT&T Research UNIX® (Linux is a re-implementation or clone). FreeBSD is more enterprise friendly than either Microsoft Windows or Linux, because of its permissive license allowing use and rework without cost or legal restrictions (Linux imposes the restrictive copy-left GNU Public License, or GPL, limiting a business’s ability to profit from its own innovations), and because FreeBSD has a well-defined release engineering process that is visible, accessible, and emphasizes stability and trust.

The motivation for the patch came when I was re-building several systems in which the dns/noip port was used – the client for No-IP’s DDNS service. Each time I searched which files had been installed by noip to find the README file with configuration details. The problem would have been avoided entirely if the missing information had only been included in noip‘s post-install message.

I informally discussed my proposed patch with Stanislaw Halik, the dns/noip port maintainer on record, who indicated his approval. I then submitted an official PR (problem report) to the FreeBSD project, and included a patch file giving the specific changes I was proposing.

The PR was reviewed by the FreeBSD maintenance team, Stanislaw gave his official approval, and my provided patch file had been merged into the FreeBSD codebase in less than 24 hours – visible, traceable and trustable!

2016-09-16-noip-pr-212637-dalescottshaw-ca

How cool was that! Although I submitted the patch to scratch my own itch, it was accepted based on its benefit to others.

Dale
Transparency with Trust

P.S. Stanislaw also introduced me to POLA, or the Principle of Least Astonishment. POLA applies to user interface and software design, and can be taken to mean that software should behave in a way that users will anticipate. Since the dns/noip port can now be installed as a binary package (not only compiling from source as has been traditional), users installing the binary package will expect instructions to be meaningful to their situation.

Load Testing a Home Internet Server

The “new” basement server hosting dalescott.net has been rock solid now for a couple days, so it was time for some load testing.

The server is an HP M7690Y media center with Intel Core2 2.40GHz CPU, 3G of RAM, and connected to the internet through a residential “internet-over-cable” service. I’m using the Apache pre-fork MPM with default configuration (no need to tune for reduced RAM with 3GB).

LoadImpact

I ran LoadImpact’s free account-required 50 user / 12 minute test, and monitored server resources while the test was running.

top/htop while while running LoadImpact test

CPU utilization spiked to maximum, but never ran out of RAM, let alone getting into the cache. Increased CPU performance means that http requests aren’t getting queued, resulting in less demand on RAM compared to a single-core CPU with 512MB RAM.

Here is the test summary. The number of VUs, or virtual users, is on the left Y-axis, the VU Page Load Time is on the right Y-axis, and time is on the X-axis.

LoadImpact results

WebPageTest

Next, I checked to see if WebPageTest liked the new server any better than the old one.

WegPageTest results

 

Compared to previous testing on the 1 CPU 512MB vps, the First Byte Time has gone from an F to a D. However, it’s not clear why Compress Images has gone from a B to a D, the servers should have identical WordPress, Apache and PHP configurations.

ISP Speed Test

Finally, I ran my ISP’s Speed Test.

ISP Speed Test

I ran the test from my laptop on the LAN side of a Hitron DOCSIS (“internet over cable”) interface adapter, but the results should apply equally to the server. The server is also connected to the Hitron, but configured on a pass-through to get its own external IP address via DHCP from my ISP.

Conclusion

Performance from the new server far exceeds that of the previous minimal vps droplet, but that is to be expected given the hardware performance. However, it seems performance on a residential ISP service is much more variable than the vps was. I ran the LoadImpact test several times from mid-morning to mid-afternoon, with worst-case VU load times in tens of seconds occurring after lunch with 40+ VUs. Obviously there will need to be changes again when the site starts drawing significant traffic.

Evaluating FAMP server performance

I recently moved dalescott.net from a bare metal server to a $5/month DigitalOcean droplet (512MB Memory, 1 Core Processor, 20GB SSD Disk). Afterwards, I wanted to get a measure of server performance. A quick web search came up with some candidates:

  1. Load Impact, claiming “The leading on-demand load and performance testing software-as-a-service.”
  2. Neustar, who claims “We started it all. And we continue to shape the industry.”
  3. WebPageTest, created by AOL for internal use before being open-sourced and now primarily developed and supported by Google.

There are more, but these will be enough to get started with. I had wanted to include Blitz.io, which was referenced in a blog post by Ryan Frankell on using Apache on a small DigitalOcean droplet . However, I couldn’t find any free evaluation capability, and Blitz’s demo is hard-coded to use a Blitz demo site, which eliminated Blitz from consideration.

Tuning Apache

The first step was to tune Apache. 512MB of RAM is low by server standards and also Apache’s default configuration. Some tuning is needed to keep Apache operating in the available RAM, otherwise Apache will gradually consume everything, including swap, and eventually the kernel will start thrashing.

KeepAlive

KeepAlive On
MaxKeepAliveRequests 100
KeepAliveTimeout 5

The FreeBSD apache24 port already sets these as defaults in /usr/local/etc/apache24/extra/httpd-default.conf  (the default MaxKeepAliveRequests is even lower than the recommended 200).

References

MPM-Prefork

Although Event is reportedly the default MPM in Apache 2.4, Prefork is still the default on FreeBSD 10.x for compatibility with non-thread-safe php/perl/python modules.

MaxRequestWorkers and MaxConnectionsPerChild must be lowered in /usr/local/etc/apache24/extra/httpd-mpm.conf, and the Include line for httpd-mpm.conf in usr/local/etc/apache24/httpd.conf must be un-commented (and Apache restarted).

# prefork MPM
# StartServers: number of server processes to start
# MinSpareServers: min number of server processes which are kept spare
# MaxSpareServers: max number of server processes which are kept spare
# MaxRequestWorkers: max number of server processes allowed to start
# MaxConnectionsPerChild: max num of connections a server process serves
# before terminating
#
#<IfModule mpm_prefork_module>
#    StartServers 5
#    MinSpareServers 5
#    MaxSpareServers 10
#    MaxRequestWorkers 250
#    MaxConnectionsPerChild 0
#</IfModule>

<IfModule mpm_prefork_module>
 StartServers 5
 MinSpareServers 5
 MaxSpareServers 10
# reduce max number of server processes to not exceed physical memory
# called "MaxClients" prior to v2.3.13 (MaxClients is still supported)
# - uses cache under load when 25
# - appears to not use cache under load when 12
 MaxRequestWorkers 12
# reduce max connections per child to avoid idle processes from holding onto memory
# - memory use seems stable when 200 but could be lower than optimal
# MaxConnectionsPerChild 350
 MaxConnectionsPerChild 200
</IfModule>

References

Load Impact

I first used the free no-login-required test, which loads 25 Virtual Users in 5 minutes, but by creating a free login profile you are allowed 5 (QTY)-five minute tests (executions) up to 100 VUs per month. Here are the results of the Load Impact test with Apache tuned, showing load times in the low seconds for up to 15 users. This can likely be further improved, but the server is stable and adequate for the expected amount of traffic.

2016-08-12 LoadImpact test results

 

Neustar

Neustar queued three servers for testing, but the Washington server never ran. Here are the results when I gave up waiting.

image

WebPageTest

WebPageTest concentrates on measuring how fast it takes a page to load, and gives lots of information for digging into the load times for each aspect of the page. I was looking for more of a general load or stress test to see if my server will fail under load, but the information can be valuable none the less.

image

Server Performance

Here are the result of top while the Load Impact no-login (20 VU) test is running. Although the server can by no means be considered fast, it also is not exhausting swap and stalling. I’m pleased for now, and will be back to review when time permits.

top-load-test-during

 

Dynamic DNS on FreeBSD and NoIP.com

no-ip is the only free dynamic IP address provider that provides a client for FreeBSD (or to give credit where it is due, that some kind volunteer ported to FreeBSD).

Regardless, the service works and I have no complaints, other than forgetting how to set it up when I build a new server. So here, for my reference, is how to set up the client on FreeBSD.

  • Create an account on no-ip, then create a domain (either pay to register a domain, or use one of the free ones available).
  • Update your ports tree, then build and install the client. I found you can install the binary using pkg, but the only way to create the binary configuration file seems to with the port. 
    % cd /usr/ports/dns/noip/
    % sudo make install
  • Create the client configuration file.
    % sudo make conf
  • You will be prompted for your no-ip username and password, network interface and hostname (say No when asked if you want to update all your hosts, you’ll then be prompted for each of the hostnames you have registered (fwiw, the configu file is /usr/local/etc/no-ip2.conf).
  • Edit /etc/rc.conf to start noip at boot.
    noip_enable=”YES”
  • Start the noip service.
    % sudo service start noip service status noip