Rotating Apache log files on FreeBSD

I needed to review an Apache httpd server error log file recently and even though the server had been rebooted only a month ago, I had to scroll through 95,000 lines before I got to the interesting part.

To make Apache’s log files more manageable, I configured them to roll every week using the FreeBSD standard newsyslog utility. newsyslog is run from cron, and in the default configuration runs every hour. This limits rolling logfiles to at most once an hour, but this is typically more than adequate for system log files.

I could have hacked newsyslog’s main config file /etc/newsyslog.conf, which would kept all configuration in one place for convenience. However this can lead to trouble when updating the system (because any new version would have to be manually merged with the edited old version), and keeping updating simple is generally a good idea. To deal with this, there are directives in the main config file to read additional optional configuration:

...
<include> /etc/newsyslog.conf.d/*
<include> /usr/local/etc/newsyslog.conf.d/*

Since the Apache server is third-party software, I created /usr/local/etc/newsyslog.conf.d/apache.conf containing the following:

# Apache
/var/log/httpd-access.log www:www 440 9 * $W1D4 J /var/run/httpd.pid 30
/var/log/httpd-error.log www:www 440 9 * $W1D4 J /var/run/httpd.pid 30

To understand what this means, the fields are: [logfile name] [owner-group] [mode] [count] [size] [when] [flags] [path to pid file] [signal]. Apache’s access and error log files will be rolled every Monday at 4am (system time), a total of 9 weekly archives will be kept (providing up to 10 weeks of logs counting the current log), and log file archives will be compressed using bzip2. Some other interesting points are:

  • For security, ownership of archived logs is set to www and the file mode is set to read-only for user and group, with no access by anyone else.
  • A SIGUSR1 signal (30) is sent to Apache to perform a graceful restart after rolling the log file.

For more information, see the System Logging section of the FreeBSD Manual as well as the man pages for newsyslog and newsyslog.conf.

To read a compressed log file, uncompress the file and pipe to less:

% sudo bzcat httpd-error.log.0.bz2 | less

Which is also equivalent to the simpler:

% sudo bzless httpd-error.log.0.bz2

Cheers!

5S Program

The efficiency of a plant or production facility is determined in part by the cleanliness and order of the facility. The 5S program was developed in Japan to educate employees and improve production plant order and cleanliness. The program has also been widely adopted in the west.

The components of the 5S program are:

  • Sort (Seiri)
  • Set In Order (Seiton)
  • Shine (Seiso)
  • Standardize (Seiketsu)
  • Sustain (Shitsuke)

One popular variation of the program uses six components by adding Safety (although In Order could easily be interpreted to include Safety), while another variation only uses four components by combining Set In Order and Shine.

Henry Ford’s CANDO program pre-dates 5S and includes Cleaning Up, Arranging, Neatness, Discipline and Ongoing Improvement, although the two are likely unrelated.

Now you can start your own 5S program!

FreeBSD on a BBG

Here’s the situation after installing FreeBSD on my BBG (BeagleBone Green), using an image published by the raspBSD project. No custom configuration or installing additional software has been done, although I have updated the package database. There’s more information on the install in a previous post.

For background, a BeagleBone Green (BBG) has a TI Sitara AM335x (1GHz ARM Cortex-A8) with 512MB DDR3 and 4GB eMMC (primary boot device), a micro SD socket (alternate boot device and additional storage), two USB connectors (one client and one host), ethernet, two Grove 4-pin connectors and two 46-pin 2×23 0.100″ pin headers with GPIO, SPI, I2C and other signals. 

Identification & Disk Use

Active Processes

I have two remote ssh sessions open.

Memory use

BeagleBone and FreeBSD

I recently purchased a BeagleBone Green (BBG) to experiment with FreeBSD on an embedded platform. The BBG has been available for a couple of years, and while I was tempted to get a BBG Wireless (BBGW), it would have meant ordering on-line and waiting for delivery. At least for initial development work I prefer a hard-wired connection, but also prefer to support local when possible.

BeagleBone Green

A BeagleBone Green (BBG) is a TI Sitara AM335x (1GHz ARM Cortex-A8 processor) with 512MB DDR3 and 4GB eMMC (which is the standard boot device), a micro SD socket (the alternate boot device and data storage memory), two USB connectors (one client and one host), ethernet, two Grove 4-pin connectors and two 46-pin 2×23 0.100″ pin headers. The original BeagleBoard emerged around 2010, and in 2013 was a winner in Embedded Computing Design’s “2013 Top Embedded Innovator award” in the Top Products Silicon category. The BeagleBone Black (BBB) was launched in 2013 as a lower-cost barebones BeagleBoard, and the BBG was launched in 2015 with two Grove connectors replacing the BBB’s HDMI connector.

Open-Source Hardware

A significant advantage of the BBG (and the other BeagleBoards and BeagleBones) for that the physical design (the schematics and pcb layout files) are provided under an open-source license. The BBG files are in a GitHub project using the MIT license. For someone designing a similar-but-different product, this can be a significant time saver.  The openness reportedly continues to other technical details of the design, such as the low-level details of power management.

FreeBSD

The RaspBSD project provides pre-built images for the BBG. The project originally provided FreeBSD images for the Raspberry Pi, but has expanded their scope to include the BBG (and BBGW). Once I sorted out how to select the alternate boot device (to boot from the micro SD card I had copied the RaspBSD image to) everything started falling into place.

The RaspBSD image is based on Head, which is new for me as I’m running 10.3-RELEASE on my web server. However, I’m looking forward to experiencing life on the edge.

I’m not sure if this is correct, but it seemed to work consistently. To boot from the micro SD card instead of eMMC, disconnect power then hold down the switch beside the micro SD slot, apply power and continue holding the switch for a count of three. After some testing (and being sure I could re-load the bundled Ubuntu-based system if I wanted), I used the script provided with RaspBSD to copy the FreeBSD image to the eMMC. This significantly improved boot time and also freed up the micro SD card for data storage.

Conclusion

For me, the BBG is the superior single-board unix computer for basing a new product design on. The openly provided design files and technical documentation, as well as easy access to GPIOs and other hardware resources on the two pin headers, provide a significant head start compared to having to start from scratch.