Author: dale

I started as a technician, first setting up sound reinforcement and projection equipment at a convention facility, co-founded a DJ business and constructed our PA system from scrach, then got into electronics maintenance and repair at a community college. After obtaining a university degree in engineering, I designed integrated circuits for the first generation of 900 MHZ analog cellphones, the first generation of CA-code-phase tracking GPS receivers and digital pro-audio. I then progressed through a variety of leadership roles in product management, marketing, engineering product development and manufacturing, in fields such as industrial GPS-based positioning and navigation, pipeline integrity inspection and industrial process control.
Posted on

Calgary Tech Groups

A colleague asked about tech groups in the Calgary Canada region.

The local ‘nix groups are CUUG (Calgary Unix Users Group, www.cuug.ab.ca) and Calgary LUG (Linux Users Group, http://clug.ca/). CUUG tends to focus more on server-side topics such as networking and security, whereas CLUG (at least from meeting topics and mailing list conversation) tends to focus more on distros, desktop apps and device interconnectivity.

CAMUG (Calgary Agile Method’s Users Group, www.calgaryagile.com). They cover a broad spectrum of software topics in their meetings and workshops, with a small-a-agile hook (e.g. developing software in an agile way using Git and GitHub, developing software in an agile way using Ruby…). CAMUG can draw 50-100 people for a popular topic, and their session last night on Appsec with Jim Bird was great!

The ASQ (American Society of Quality). The Calgary chapter (http://www.asqcalgary.org/ee/) seems mostly interested in manufacturing quality systems (i.e. enterprise stuff) and meets at night monthly in the NE. The topics have been hit and miss for my interests, but they’ve had excellent topics/speakers when I’ve attended.

The SQDG (Software Quality Discussion Group, http://www.sqdg.ca/) meets monthly downtown during lunch, and focuses on software testing practices. They’ve had some good topics, but I haven’t been to a session in over a year now. Again, the topics are hit and miss for my interests but the ones I’ve attended have been excellent with good networking opportunities.

Calgary Protospace (http://protospace.ca/) seem most interested in low-level hardware and firmware hacking, but the system drive failed in a NAS array a few years ago and all the family photos were temporarily unavailable (my wife said I lost them). A unix networking specialist I met through protospace helped me reconstruct the array and save my marriage.

Mobile Mondays (https://twitter.com/momoyyc) host a mobile-related hackathon the first Monday night of every month.

Startup Calgary (http://startupcalgary.ca/) is a government/industry coalition I like to follow not only for their own events, but also because they aggregate what everyone else is doing into a nice regular email they send me.

Calgary democamp (http://democampcalgary.com/) organize DemoCamps periodically for tech entrepreneurs to demo shiny new apps that will change the world <wink>. I’ve only been to one, but it was a packed house and very interesting set of presentations.

There are also a number of Calgary-area tech MeetUp groups, such as the Calgary WordPress MeetUp Group, the Calgary PHP AND NOT PHP Meetup Group, Pixels and Pints, PyYYC (Python) and more. Do some poking around for your interests, and contribute to the success of a community.

Posted on

Removing Ports No Longer Required

I replaced Postfix with ssmtp recently, and wanted to check my FreeBSD server for any installed ports that are no longer needed. Thanks to Chris on the FreeBSD mail list, I have added some new portmaster flags to my tookit.

First, you may want verify your ports index by rebuilding it:

# cd /usr/ports
# make index

Next, list the ports installed, grouped by:

  • Root ports (No dependencies, not depended on)
  • Trunk ports (No dependencies, are depended on)
  • Branch ports (Have dependencies, are depended on)
  • Leaf ports (Have dependencies, not depended on)
# portmaster -l

Finally, delete the leaf ports that you know you don’t need on their own:

# portmaster -e portname

Thanks, Chris.

 

Posted on

Installing mdbtools, MariaDB, Nginx, and PHP on FreeBSD 9.1

Update 2014-07-06 mdbtools is now available as a package for installing using pkg. It can also be compiled from source if desired.

Sometimes building a new server is easier than upgrading an old one, with the added bonus of staying up to date with current install procedures. Here’s the procedure I followed to build a new FreeBSD 9.1 server recently. I’ll be installing everything on the bare server (no jails).

This is a unfinished work in process!

The major applications being installed are:

  • FreeBSD 9.1-RELEASE
  • mdbtools v0.7 (project head from GitHub)
  • MariaDB 5.5.31
  • Nginx 1.4.2
  • PHP 5.5.1
  • phpMyAdmin 4.0.5

Create a virtual machine

You can build a bare-metal server, but a Virtual Machine (vm) can be more convenient to work with, and a dump from the vm can be easily restored on a bare-metal server if needed. The virtualizing environment I use is VirtualBox. Start by creating a basic virtual machine for BSD (FreeBSD), with 256 MB memory, a 20G IDE primary master drive, and a CD/DVD drive IDE secondary master.

Next, consider the network interface and how the vm will connect to the internet, and how you will connect to it. My default configuration is to bridge my laptop and the vm network interfaces. This gives me access to the vm from the host and from any other devices on my LAN, but requires a DHCP server on your LAN, and port 22 routed (not blocked). If you don’t have internet access (e.g. if you are in someone else’s conference room), or if you do but port 22 is blocked (e.g. you’re in a certain popular fast-food restaurant with free WiFi), you will need to use a NAT or Host Only network connection in VirtualBox. If you don’t, the IP address Windows choses for itself will typically not be in the same subnet as the IP address VirtualBox choses for the client, and the two will not be able to communicate. If you use the NAT connection, you will need to forward the following ports in order to communicate with the vm from the host:

ssh               host port 2222        client port 22
HTTP              host port 8880        client port 80
MySQL TCP         host port 3336        client port 3306
MySQL UDP         host port 3336        client port 3306
ftp               host port 2221        client port 21

Install FreeBSD OS

Install FreeBSD base system. Perform a standard install using FreeBSD-9.1-RELEASE-i386-dvd1.iso (or -bootonly.iso), with the following configuration:

hostname: firefly.scc.local
root password: secret
daemons to start at boot: sshd
user: dale (group wheel)

After installing the base system and rebooting, login as root and update the FreeBSD OS:

# freebsd-update fetch
# freebsd-update install

If freebsd-update reports that /usr/src/crypto/ssl/s3_cbc.c is missing, create the directory path (e.g. “# mkidir -p /usr/src/crypto/ssl/”) and fetch/extract again.

Update the ports tree:

# portsnap fetch
# portsnap extract

Edit /etc/hosts to specify a FQDN (fully qualified domain name) for the server:

::1                     localhost firefly.scc.local
127.0.0.1               localhost firefly.scc.local
#10.0.2.15               firefly.scc.local # default VBOX NAT IP address

Install portmaster:

# cd /usr/ports/ports-mgmt/portmaster/
# make install clean

If desired, install and run portaudit to monitor port security notices (you may want to omit this if you won’t be updating the system):

# cd /usr/ports/ports-mgmt/portaudit/
# make install clean
# portaudit -Fda

Edit /etc/ssh/sshd_config to allow remote ssh login by root and user dale, and restart sshd. I will keep OpenSSL and OpenSSH from the base system. The root user will be allowed remote ssh access for convenience, but this is not advised for systems accessible from the internet.

# vi /etc/ssh/sshd_config
# add following
AllowUsers root dale
PermitRootLogin yes
#
# /etc/rc.d/sshd restart

Copy your ssh public key to ~/.ssh/authorized_keys (e.g. using WinSCP).

You may want to define a mail alias for the root user and have local system mail forwarded to the system administrator.

# vi /etc/mail/aliases
add following alias:
root: <a href="mailto:realuser@realdomain.com">realuser@realdomain.com</a>

Install utility applications

Install some basic utility apps that usually come in handy eventually:

# cd /usr/ports/archivers/p7zip/
# make install clean
#
# cd /usr/ports/ftp/curl/
# make install clean
#
# cd /usr/ports/textproc/flip
# make install clean
#
# cd /usr/usr/ports/devel/git
# make install clean
#
# cd /usr/ports/www/lynx
# make install clean
#
# cd /usr/ports/ftp/wget
# make install clean
#
# cd /usr/ports/archivers/unzip   # might already be installed
# make install clean

Install mdbtools

mdbtools is a suite of utilities for working with data from an MS Jet database on a Unix system. First, install the GNU build toolchain needed to compile mdbtools.

# cd /usr/ports/devel/libtool   # may already be installed
# make install clean

# cd /usr/ports/devel/automake
# make install clean

# cd /usr/ports/devel/autoconf    # may already be installed
# make install clean

# cd /usr/ports/textproc/flex/
# make install clean

# cd /usr/ports/devel/bison/    # may already be installed
# make install clean

# cd /usr/ports/textproc/txt2man/
# make install clean

# cd /usr/ports/devel/glib20  # undocumented dependency
# make install clean

# rehash

Next, clone the mdbtools Github repo locally:

> mkdir ~/src/
> cd ~/src/
> git clone https://github.com/brianb/mdbtools.git

And finally, build and install mdbtools:

> cd ~/src/mdbtools/
> autoreconf -i -f
> ./configure
> gmake
> su - 
# gmake install

Add the installed mdbtools man pages to manpath (the install uses the Linux-typical /usr/local/share/man/man1/) by creating /usr/local/etc/man.d/mdbtools.conf and rebuilding the whatis database:

# vi /usr/local/etc/man.d/mdbtools.conf
# add MANPATH
MANPATH /usr/local/share/man
#
# /etc/periodic/weekly/320.whatis
# exit
> apropos mdb

Install Web App Stack

My goal is a simple common stack using MariaDB and Nginx for hosting PHP and Python-based web applications with MySQL back-ends. I’m installing MariaDB here to learn about using it, as MySQL appears to be loosing favor in the opensource community, and MariaDB is fully compatible with MySQL. I’m installing Nginx for similar reasons, it has become popular recently for its asynchronous event-driven approach to request handling.

Install MariaDB Database Server

Install MariaDB:

# cd /usr/ports/databases/mariadb55-server/   # includes client
# make config ; make install clean
# rehash

Edit rc.conf to start MariaDB at boot:

# vi /etc/rc.conf
...
# add mysql_enable
mysql_enable="YES"

Manually start MariaDB:

# service mysql-server start

Setup grant tables:

# cd /usr/local/   # mysql_install_db assumes its running from here
# mysql_install_db --user=mysql

Configure root password:

> mysqladmin -u root password 'appleton'
> mysqladmin -u root -p -h firefly.scc.local password 'appleton'

Grant root permission to connect remotely:

> mysql -u root -p
> grant all privileges on *.* to 'root'@'%' identified by 'appleton' with grant option;
> exit;

Use the provided my-medium.cnf config file and edit for using InnoDB tables:

The output from “my_print_defaults –help” implies my-medium.cnf should copied to /etc/my.cnf, but I’ll use the MySQL convention I’ve learned until I know different.

# cp /usr/local/share/mysql/my-medium.cnf /var/db/mysql/my.cnf
# vi /var/db/mysql/my.cnf
...
# Uncomment the following if you are using InnoDB tables
innodb_data_home_dir = /var/db/mysql
innodb_data_file_path = ibdata1:10M:autoextend
innodb_log_group_home_dir = /var/db/mysql
# You can set .._buffer_pool_size up to 50 - 80 %
# of RAM but beware of setting memory usage too high
innodb_buffer_pool_size = 16M
innodb_additional_mem_pool_size = 2M
# Set .._log_file_size to 25 % of buffer pool size
innodb_log_file_size = 5M
innodb_log_buffer_size = 8M
innodb_flush_log_at_trx_commit = 1
innodb_lock_wait_timeout = 50

Restart MariaDB:

# service mysql-server restart

and finally do a basic test to confirm things are basically working:

> mysql -u root -p
...
MariaDB [(none)]> show databases;
...
MariaDB [(none)]> use test;
...
MariaDB [(test)]&gt; exit;
Bye
>

Install Nginx web server

Nginx will interoperate with PHP via FastCGI and PHP-FPM (FastCGI Process Manager), and with Python via FastCGI and via the flup library (py27-flup). Install Nginx with appropriate options (note I’m not enabling SSL, which I don’t need at the moment, but I may wish later I had included it also):

# cd /usr/ports/www/nginx/
# make config
Use the default configuration options:
<span style="color: #666666; font-family: Consolas;">IPV6 IPv6 protocol support
HTTP Enable HTTP module 
HTTP_CACHE Enable http_cache module 
HTTP_REWRITE Enable http_rewrite module 
HTTP_STATUS Enable http_stub_status module</span>
WWW Enable html status files

# make install clean

The Nginx config file is:

/usr/local/etc/nginx/nginx.conf

Edit rc.conf to start Nginx at boot:

# vi /etc/rc.conf
...
nginx_enable="YES"

Manually start Nginx:

# service nginx start

Test that Nginx is running by browsing to the vm (e.g. http://localhost:8880). Nginx will be configured later.

Install PHP processor

Install PHP:

# cd /usr/ports/lang/php55/
# make config
...
enable additional options:
FPM Build FPM version

# make install clean

Edit PHP php.ini configuration file:

# cp /usr/local/etc/php.ini-production /usr/local/etc/php.ini
# vi /usr/local/etc/php.ini
...
# uncomment session.save_path
session.save_path = "/tmp"
# add default date timezone
date.timezone = "America/Edmonton"

Install a motley assortment of PHP extensions (essentially the current requirements for phpMyAdmin, MediaWiki and the Yii framework).

# cd /usr/ports/lang/php55-extensions
# make config
...
enable additional extensions to install:
BZ2
CTYPE
CURL
DOM
FILTE
GD
ICONV
JSON
MBSTRING
MCRYPT
MYSQL
MYSQLI
OPENSSL
PDO
PDO_MYSQL (why PDO if not PDO_MYSQL ??)
READLINE
SESSION
SOAP
XML
ZIP
ZLIB

# make install clean

Configure PHP-FPM

I will be using the fastCGI process manager PHP-FPM (included with PHP starting with release 5.3.3) with Nginx. Configure PHP-FPM by editing /usr/local/etc/php-fpm.conf:

# vi /usr/local/etc/php-fpm.conf
...
make following changes ("-" means delete, "+" means add):

-; events.mechanism = epoll
+events.mechanism = kqueue
...
-listen = 127.0.0.1:9000
+listen = /var/run/php-fpm.sock
...
-;listen.owner = www
-;listen.group = www
-;listen.mode = 0666
+listen.owner = www
+listen.group = www
+listen.mode = 0666

Edit rc.conf to start PHP-FPM at boot:

# vi /etc/rc.conf
...
php_fpm_enable="YES"

Manually start PHP-FPM:

# service php-fpm start

Nginx will need to be configured further for specific to use PHP-FPM.

Install phpMyAdmin

phpMyAdmin provides convenient management of the MariaDB database server, without requiring any client-side software. First, install phpmyadmin from ports:

# cd /usr/ports/databases/phpmyadmin/
# make config
...
disable options:
APC PHP APC (animated progress bar) support
# make install clean

It seems that pecl-APC (the APC option in the phpMyAdmin config) can’t be compiled with PHP 5.5 (see FreeBSD forum post). I’d rather not downgrade to PHP 5.4, and I suspect I can make do without an “animated progress bar”, so I’m unselecting it for now. Load pma tables:

# cd /usr/local/www/phpMyAdmin
# mysql -u root -p < ./examples/create_tables.sql

Configure Nginx:

# command
# command

Create the phpmyadmin configuration using the setup wizard (and copy to config.inc.php). Access http://hostname/phpmyadmin/setup, specify connection type: socket (instead of tcp) and use suggested names for all tables. Increase max session before auto logout to 9 hrs (from 3 min):

# vi /usr/local/www/phpMyAdmin/config.inc.php
...
$cfg['LoginCookieValidity'] = 3600 * 9; // 3600 sec/hr * 9 hrs
...

Also edit session.gc_maxlifetime in php.ini:

# vi /usr/local/etc/php.ini
...
; increase max session time for phpMyAdmin. Max session time for phpMyAdmin
; set to 9 hrs in phpMyAdmin config.inc.php ((LoginCookieValidity), which
; requires increasing php garbage collection to greater than 9 hrs
; E.g. 32500 sec = (3600 sec/hr * 9 hrs) + 100 sec
session.gc_maxlifetime = 32500

Configure Nginx:

# command
# command
# command

Install requirements for PHP unit and functional testing (optional)

If you’re going to use the server for PHP unit and functional testing, you will likely want to install xdebug and testing frameworks. Install php-xdebug:

# cd /usr/ports/devel/php-xdebug
# make install clean

Edit /usr/local/etc/php/extensions.ini and comment loading xdebug as std extension. Edit /usr/local/etc/php.ini to add loading xdebug as zend_extension.

[xdebug]
; load xdebug as zend_extension (loading as std extension commented in php/extensions.ini)
zend_extension=/usr/local/lib/php/20100525/xdebug.so
; enable profiling
xdebug.profiler_enable = 1
xdebug.profiler_output_dir = /tmp/profiler
; remote settings
xdebug.remote_autostart=off
xdebug.remote_enable=on
xdebug.remote_handler=dbgp
xdebug.remote_mode=req
xdebug.remote_host=localhost
xdebug.remote_port=9000

Install pear and pear PHPUnit for unit testing Yii-based projects.

# cd /usr/ports/devel/pear
# make install clean
# pear config-set auto_discover 1
# pear install pear.phpunit.de/PHPUnit

Install the PHP pear package Selenium, which is required for Yii PHPUnit testing and used also for functional testing. Install curl and PHP curl extension first if not already installed.

# cd /usr/ports/ftp/curl; make install clean
# cd /usr/ports/ftp/php5-curl; make install clean
# pear install phpunit/PHPUnit_Selenium

And that’s that!