Open Source Storage Technology Crafters

The conversation often starts with a challenge. “What’s so great about open source storage technology?

For the casual end users of storage systems, regardless of SAN (definitely not Fibre Channel) or NAS on-premises, or getting “files” from the personal cloud storage like Dropbox, OneDrive et al., there is a strong presumption that open source storage technology is cheap and flaky. This is not helped with the diet of consumer brands of NAS in the market, where the price is cheap, but the storage offering with capabilities, reliability and performance are found to be wanting. Thus this notion floats its way to the business and enterprise users, and often ended up with a negative perception of open source storage technology.

Highway Signpost with Open Source wording

Storage Assemblers

Anybody can “build” a storage system with open source storage software. Put the software together with any commodity x86 server, and it can function with the basic storage services. Most open source storage software can do the job pretty well. However, once the completed storage technology is put together, can it do the job well enough to serve a business critical end user? I have plenty of sob stories from end users I have spoken to in these many years in the industry related to so-called “enterprise” storage vendors. I wrote a few blogs in the past that related to these sad situations:

We have such storage offerings rigged with cybersecurity risks and holes too. In a recent Unit 42 report, 250,000 NAS devices are vulnerable and exposed to the public Internet. The brands in question are mentioned in the report.

I would categorize these as storage assemblers.

Continue reading

A FreeNAS Compression Tale

David vs Goliath Credit: Miguel Robledo of https://www.artstation.com/miguel_robledo

David vs Goliath

It was an underdog tale worthy of the biblical book of Samuel. When I first caught wind of how FreeNAS™ compression prowess was going against NetApp® compression and deduplication in one use case, I had to find out more. And the results in this use case was quite impressive considering that FreeNAS™ (now known as TrueNAS® CORE) is the free, open source storage operating system and NetApp® Data ONTAP, is the industry leading, enterprise, “king of the hill” storage data management software.

Certainly a David vs Goliath story.

Compression in FreeNAS

Ah, Compression! That technology that is often hidden, hardly seen and often forgotten.

Compression is a feature within FreeNAS™ that seldom gets the attention. It works, and certainly is a mature form of data footprint reduction (DFR) technology, along with data deduplication. It is switched on by default, and is the setting when creating a dataset, as shown below:

Dataset creation with Compression (lz4) turned on

The default compression algorithm is lz4 which is fast but poor in compression ratio compared to gzip and bzip2. However, lz4 uses less CPU cycles to perform its compression and decompression processing, and thus the impact on FreeNAS™ and TrueNAS® is very low.

NetApp® ONTAP, if I am not wrong, uses lzopro as default – a commercial and optimized version of the open source LZO compression library. In addition, NetApp also has their data deduplication technology as well, something OpenZFS has to improve upon in the future.

The DFR report

This brings us to the use case at one of iXsystems™ customers in Taiwan. The data to be reduced are mostly log files at the end user, and the version of FreeNAS™ is 11.2u7. There are, of course, many factors that affect the data reduction ratio, but in this case of 4 scenarios,  the end user has been running this in production for over 2 months. The results:

FreeNAS vs NetApp Data Footprint Reduction

In 2 of the 4 scenarios, FreeNAS™ performed admirably with just the default lz4 compression alone, compared to NetApp® which was running both their inline compression and deduplication.

The intention to post this report is not to show that FreeNAS™ is better in every case. It won’t be, and there are superior data footprint reduction tech out there which can outperform it. But I would expect potential and existing end users to leverage on the compression capability of FreeNAS™ which is getting better all the time.

A better compression algorithm

Followers of OpenZFS are aware of the changing of times with OpenZFS version 2.0. One exciting update is the introduction of the zstd compression algorithm into OpenZFS late last year, and is already in TrueNAS® CORE and Enterprise version 12.x.

What is zstd? zstd is a fast compression algorithm that aims to be as efficient (or better) than gzip, but with better speed closer to lz4, relatively. For a long time, the gzip compression algorithm, from levels 1-9, has been serving very good compression ratio compared to many compression algorithms, lz4 included.

However, the efficiency came at a higher processing price and thus took a longer time. At the other end, lz4 is fast and lightweight, but its reduction ratio efficiency is very poor. zstd intends to be the in-between of gzip and lz4. In the latest results published by Facebook’s github page,

zstd performance benchmark against other compression algorithms

For comparison, zstd (level -1) performed very well against zlib, the data compression library in gzip. It was made known there are 22 levels of compression in zstd but I do not know how many levels are accepted in the OpenZFS development.

At the same time, compression takes advantage of multi-core processing, and actually can speed up disk I/O response because the original dataset to be processed is smaller after the compression reduction.

While TrueNAS® still defaults lz4 compression as of now, you can probably change the default compression with a command

# zfs set compression=zstd-6 pool/dataset

Your choice

TrueNAS® and FreeNAS™ support multiple compression algorithms. lz4, gzip and now zstd. That gives the administrator a choice to assign the right compression algorithm based on processing power, storage savings, and time to get the best out of the data stored in the datasets.

As far as the David vs Goliath tale goes, this real life use case was indeed a good one to share.

 

Discovering OpenZFS Fusion Pool

Fusion Pool excites me, but unfortunately this new key feature of OpenZFS is hardly talked about. I would like to introduce the Fusion Pool feature as iXsystems™ expands the TrueNAS® Enterprise storage conversations.

I would not say that this technology is revolutionary. Other vendors already have the similar concept of Fusion Pool. The most notable (to me) is NetApp® Flash Pool, and I am sure other enterprise storage vendors have the same. But this is a big deal (for me) for an open source file system in OpenZFS.

What is Fusion Pool  (aka ZFS Allocation Classes)?

To understand Fusion Pool, we have to understand the basics of the ZFS zpool. A zpool is the aggregation (borrowing the NetApp® terminology) of vdevs (virtual devices), and vdevs are a collection of physical drives configured with the OpenZFS RAID levels (RAID-0, RAID-1, RAID-Z1, RAID-Z2, RAID-Z3 and a few nested RAID permutations). A zpool can start with one vdev, and new vdevs can be added on-the-fly, expanding the capacity of the zpool online.

There are several types of vdevs prior to Fusion Pool, and this is as of pre-TrueNAS® version 12.0. As shown below, these are the types of vdevs available to the zpool at present.

OpenZFS zpool and vdev types – Credit: Jim Salter and Arstechnica

Fusion Pool is a zpool that integrates with a new, special type of vdev, alongside other normal vdevs. This special vdev is designed to work with small data blocks between 4-16K, and is highly efficient in handling random reading and writing of these small blocks. This bodes well with the OpenZFS file system metadata blocks and other blocks of small files. And the random nature of the Read/Write I/Os works best with SSDs (can be read or write intensive SSDs).

Continue reading

OpenZFS 2.0 exciting new future

The OpenZFS (virtual) Developer Summit ended over a weekend ago. I stayed up a bit (not much) to listen to some of the talks because it started midnight my time, and ran till 5am on the first day, and 2am on the second day. Like a giddy schoolboy, I was excited, not because I am working for iXsystems™ now, but I have been a fan and a follower of the ZFS file system for a long time.

History wise, ZFS was conceived at Sun Microsystems in 2005. I started working on ZFS reselling Nexenta in 2009 (my first venture into business with my company nextIQ) after I was professionally released by EMC early that year. I bought a Sun X4150 from one of Sun’s distributors, and started creating a lab server. I didn’t like the workings of NexentaStor (and NexentaCore) very much, and it was priced at 8TB per increment. Later, I started my second company with a partner and it was him who showed me the elegance and beauty of ZFS through the command lines. The creed of ZFS as a volume and a file system at the same time with the CLI had an effect on me. I was in love.

OpenZFS Developer Summit 2020 Logo

OpenZFS Developer Summit 2020 Logo

Exciting developments

Among the many talks shared in the OpenZFS Developer Summit 2020 , there were a few ideas and developments which were exciting to me. Here are 3 which I liked and I provide some commentary about them.

  • Block Reference Table
  • dRAID (declustered RAID)
  • Persistent L2ARC

Continue reading