ZFS

Before finalizing our SSD storage I really wanted to give ZFS, and more specifically, FreeNAS a go. ZFS is particularly interesting because of its awesome performancy with its parity implementation dubbed RAID-Z. Basically, the Sun (now Oracle) guys came up with some clever ideas that greatly eliminate the performance hit that kills the regular parity RAID levels like RAID5 and RAID6. One might even go as far to say that traditional parity levels are fundamentally broken and after seeing the numbers (I’ll post them sometime later), I tend to agree.

However, there are some drawbacks, of course. Let’s just say that the use of ECC RAM is strongly encouraged. That’s no problem when we’re talking about servers, but there are many people building SOHO NASes who can’t really afford such equipment. Also, with RAID-Z rebuild times are incredibly long, so long that with big disks they can take weeks to complete, making you highly vulnerable to other disks failing during the process. In fact, this even led to the construction of RAID-Z3 – that is exactly what it suggests, triple parity and which is highly recommended for arrays with big capacities or big numbers of drives.

Despite all this, ZFS is super awesome. It totally blew Storage Spaces away in parity performance. I’ll give you some numbers sometime later but it’ll take some time to crunch the Iozone numbers. It’s so great that I’ll even make a feature request for Storage Spaces to incorporate it in some form for an upcoming release of Windows Server. Now that OpenZFS exists as a meeting ground for all people involved, it should be quite possible. Even the Linux guys, who are totally notoriously pedantic about licensing, are working on including ZFS in a later release (make sure to also read the 1st comment for some insight). It’s available on Solaris, FreeBSD, Linux and OS X, meaning that the only major player is Windows that still lacks support.

Rest assured though, I’m not saying Microsoft should simply add ZFS to Windows. That’d be a very bold statement, I know, although with their recent moves I’d say it’s much more possible than ever before. Anyway, they can make any changes they see fit, implement the ideas on their own, introduce a new FS or add it to ReFS or whatever, it’s all fine by me – the point is that they need to fix the horrendous performance they currently provide with parity spaces because it’s nearly unusable, even for archiving purposes.

FreeNAS

FreeNAS is a long-standing storage-specific distribution based on FreeBSD that almost got converted to Linux but then they made a split and thus the Debian-based OpenMediaVault was born. Thankfully FreeNAS remains to be based on top of FreeBSD which means we get full-blown ZFS support out of the box. Additionally, FreeNAS is now owned by iXsystems, a company who’s also selling hardware storage appliances, even coping with big players like Dell. So the dogfooding is certainly there.

The great thing about FreeNAS is that you can install it onto a USB drive and just give it a spin without having to mess around with disks. In fact, that is the recommended way to do it. I’m not sure though what happens if the drive goes crap, maybe you’re supposed to back it up every day or something. Yes, I should read the manual, but for a quick test, it’s not feasible.

Anyway, I can’t speak of FreeNAS in such superlatives as I did with ZFS though. I’ve already explained how braindamaged multipath on FreeNAS is so I’ll spare that now. Due to this, I prolly won’t ditch Storage Spaces for SSD (SAN) storage in the foreseeable future, but for NAS purposes it may be a real candidate.

Another, much more minor example is that I could only make it boot in legacy mode (i.e. no UEFI), although it may be because I’ve applied the ISO on the thumb drive using Windows (FYI you need Win32 Disk Imager). Then the startup wizard failed to create my first array and gave me 3 pages of Python stack trace instead. Then I couldn’t create it manually either. Then the individual disk wipe also failed with permission denied error. Then trying to nuke the disks via SSH also said the resource is busy. Then I checked for updates and also rebooted after. Magically, my problems were solved – maybe it was the update, maybe the reboot. Who knows, who cares.

Now it’s time to create a RAID10 array. Conceptually, you need to create RAID1 vdisks, then iteratively add these to the same zpool, so yes, you literally build RAID1+0 on your own. Now what’s puzzling me is why FreeNAS doesn’t give me an easy way to do this. For example, there’s MegaRAID Storage Manager, an LSI/Avago GUI util for managing LSI cards that also works with any other card using LSI chips inside (including Intel). MSM also lacks a separate RAID10 option but if you select 4 drives and pick RAID1 it’ll be smart enough to create a RAID10 array. Not with FreeNAS though! If you select 24 disks and RAID1, it’ll create a 12-way mirror – how clever. So you need to create twelve RAID1 volumes but of course if you want to use the same name (so that it gets in the same zpool) it’ll error out because you already have that. Of course. For this reason, and because an action that takes 1 second in the console can take 1 minute on the web UI for no clear reason, I had to create the RAID10 array via SSH. Obviously, the web UI didn’t notice, and I’ve found no other way to let it know other than maybe rebooting.

Another issue is that the “Volume Manager”, used to dealing with disks and arrays on the web UI, is anything but intuitive. Additionally, it only shows like 15 or 16 disks and if I try to drag them further it shows some NaN (JavaScript not a number) error. And the slowness mentioned above. The good news is that iX is busy rewriting the web UI from scratch with FreeNAS 10, which already has an alpha release. Good luck, you’ll need it!

Final Thoughts

Even with all its shortcomings, FreeNAS seems to be an alternative definitely worth considering. I’d like to say that’s merely because of ZFS but it really isn’t.

Our current NAS (SMB) server runs on Windows Server and Storage Spaces, but because of the lower than low performance of parity schemes I had to stick to RAID10 and thus our capacity is halved. With FreeNAS I could migrate to RAID-Z3 and get a big boost of storage space while also maintaining throughput. FreeNAS also supports Active Directory integration and SMB (Samba) OOB, which means you can replace a Windows SMB box altogether with stock FreeNAS.

The best possible window for such a move will be the point when we need to expand our SMB server with another JBOD full of disks. Then I can set up a FreeNAS instance on a thumb drive, set up the ZFS storage and do some stress tests with the clients. If it works out, I’ll migrate all the existing Storage Spaces data to ZFS and kiss goodbye to Storage Spaces without a doubt.