Exploring the venerable NFS Ganesha

As TrueNAS® SCALE approaches its General Availability date in less than 10 days time, one of the technology pieces I am extremely excited about in TrueNAS® SCALE is the NFS Ganesha server. It is still early days to see the full prowess of NFS Ganesha in TrueNAS® SCALE, but the potential of Ganesha’s capabilities in iXsystems™ new scale-out storage technology is very, very promising.

NFS Ganesha

I love Network File System (NFS). It was one of the main reasons I was so attracted to Sun Microsystems® SunOS in the first place. 6 months before I graduated, I took a Unix systems programming course in C in the university. The labs were on Sun 3/60 workstations. Coming from a background of a VAX/VMS system administrator in the school’s lab, Unix became a revelation for me. It completely (and blissfully) opened my eyes to open technology, and NFS was the main catalyst. Till this day, my devotion to Unix remained sacrosanct because of the NFS spark aeons ago.

I don’t know NFS Ganesha. I knew of its existence for almost a decade, but I have never used it. Most of the NFS daemons/servers I worked with were kernel NFS, and these included NFS services in Sun SunOS/Solaris, several Linux flavours – Red Hat®, SuSE®, Ubuntu, BSD variants in FreeBSD and MacOS, the older Unices of the 90s – HP-UX, Ultrix, AIX and Irix along with SCO Unix and Microsoft® Xenix,  NetApp® ONTAP™, EMC® Isilon (very briefly), Hitachi® HNAS (née BlueArc) and of course, in these past 5-6 years FreeNAS®/TrueNAS™.

So, as TrueNAS® SCALE beckons, I took to this weekend to learn a bit about NFS Ganesha. Here are what I have learned.

What is NFS Ganesha? 

NFS Ganesha is a user land open source NFS server. It is developed in the user space instead of the operating system’s kernel space. This meant that Ganesha was easier to develop and scale because it did not have to be involved very much with many constraints of systems programming of an operating system’s kernel. Also, this also contributed to the well adoption of the technology and is cited often in the integration of high performance NFS services in Red Hat® CephFS, Red Hat® GlusterFS, Nutanix™ Files, Panasas®, LustreFS and the now defunct IBM® SONAS (Scale Out NAS).

NFS Ganesha supports all the current versions of the NFS protocol. These are NFSv3, NFSv4, 4.1, 4.2 and pNFS (parallel NFS). At the integration end with a backend storage, NFS Ganesha provides the FSAL (File System Abstraction Layer) that has multiple FSAL modules developed for many storage filesystems such as GlusterFS, ZFS, ext4, XFS and Lustre filesystem. An architecture view from the Usenix FAST’07 presentation (with FSAL highlighted) is shown below:

Original NFS Ganesha layered architecture

Despite its name sounding like the Hindu deity, Ganesh, its roots came from France’s CEA (French Commission for Atomic Energy and Alternative Energies). In the 2007 Usenix FAST Conference, the presented NFS Ganesha paper was under the names of Philippe Deniel, Thomas Leibovici and Jacques-Charles Lafoucrière, whom all developed the NFS Ganesha at the High Performance Computing Center at CEA.

Exciting possibilities with TrueNAS® SCALE

The things I have learned so far are general knowledge and do not reflect the roadmap of what TrueNAS® SCALE when it is released. However, looking at the past and present use cases of the many implementations of NFS Ganesha with other companies, I am very eager to see several exciting possibilities in TrueNAS® SCALE. Please note that this is based on my own opinions and my own research and does not reflect the technology roadmap planned for TrueNAS® SCALE by iXsystems™.

I list them briefly here.

• Scale out NFS – NFS Ganesha support multi-node NFS services with a single export namespace with NFS v4
• Distributed Clustered NFS – Its scale out architecture in TrueNAS® SCALE GlusterFS enabled multiple NFS Ganesha servers to be clustered for load balanced high performance NFS service
• High availability NFS – In a scale out and distributed architecture of TrueNAS® SCALE, nodes can fail in the cluster but the NFS service continues to run either as an active-active or active-passive capacity because of the high availability capability of NFS Ganesha.

Still exploring

This NFS Ganesha blog is still a learning and exploratory journey for me. I am merely scratching the surface of NFS Ganesha prowess and it has opened up plenty of new tunnels and rabbit holes for me to tour. But as with every weekend’s project, I love the deep diver learning of storage and filesystem architectures and NFS Ganesha certainly felt like a breath of fresh air.

Lastly, check out some of the NFS related articles I have written in the past.

• [ 2020 ] A paean to NFS
• [ 2012 ] NFS-phobic in Malaysia
• [ 2011 ] NFSv4 – Your filesystem librarian
• [ 2011 ] NFS version 4
• [ 2011 ] NFS deserves more credit from guys doing virtualization
• [ 2017 ] Of Object Storage, Filesystems and Multicloud
• [ 2015 ] The reverse wars – DAS vs NAS vs SAN