It turns out that fortunately only the boot and system drive failed, and the three drives in the software-RAID5 array are presumably still good. In a best-case scenario, I would have a backup of the system configuration and after re-installing FreeNAS on a new system drive could simply mount and configure the array using the backup configuration and life would be good. I didn’t have a backup of the system configuration. In the second-best scenario, I would have a recent backup of the file system contents and after bringing up FreeNAS with a new boot/system drive, I would re-create a clean file system and then recover the contents from the backup. I didn’t make backups because I implicitly tusted the inherent redundancy of a RAID system. Wrong, Wrong, WRONG! RAID systems can fail in all sorts of ways that make life much more difficult than swaping a drive and watching the array rebuild itself.
I promptly made full images of the drives and am learning everything I can about the FreeBSD OS (operating system) that FreeNAS is based on, disks, disk partitions, RAID, file systems, GEOM, etc., in order to figure out the old configuration and how to safely recover the array and file system. I tried looking for commercial support, but not finding any turned to the manuals and DIY. The community has been a big support, but understandably, are also pushing me to figure it out for myself (they are giving general guidance and hints).
The take-away is that a RAID system still needs off-line backups for full protection!
Along the path of educating myself, I’ve happily discovered FreeBSD. FreeBSD is an advanced operating system derived from BSD, the version of UNIX® developed at the University of California, Berkeley. The FreeBSD code license, the BSD license, allows developers to use the code pretty much without restriction – including selling it. A Linux distribution, another UNIX-like OS, is licensed under the GPL. The GPL requires developers to share new developments back to the community and prevents any selling of the code. Taking into account the development and release process, and the philosophies of the two camps, for me FreeBSD is better defined, more robust, more stable, and more viable than Linux for embedded systems development as well as a darn fine desktop OS (e.g. Apple’s OS X is based on FreeBSD).
When this is all over, I’ll be using FreeBSD throughout my home IT system for desktops (using the excellent PC-BSD distribution), servers, and for my embedded machine control project (switching from Linux). I may even convince a friend that switching to a FreeBSD makes good cents, it’s almost as “good” as a Mac, less painful than switching to Vista, and will leave enough change in his pocket for a new bike!