iXsystems™ released second iteration of TrueNAS® SCALE software just over a week ago. It is known as version 22.02.2 or Anglefish.2, with the most notable upgrades to HA (High Availability) for SCALE and Clustered SMB capabilities. This is the perfect excuse for me to learn about Clustered SMB and share what I have learned.
TrueNAS SCALEFor the
For the uninformed, Clustered SMB brings highly available SMB file sharing services to mission critical environments. More importantly, Clustered SMB is high availability in a scale-out clustered architecture.
My view beyond HA SMB
I am not familiar with Clustered SMB in a NAS (Network Attached Storage). The world I am more familiar with is either having CIFS/SMB file services on a dual controller storage appliance or running Windows File Sharing on an Microsoft® Clustered Service (MSCS). Typically in these 2 types of HA SMB services, the scale up architecture require a shared access to a consolidated storage volume. Behind the scenes, there are many mechanisms at play to ensure that one, and only one, storage controller or HA host can have write access capabilities at one time. The most common mechanism is the SCSI-3 Persistent Reservation or sometimes known as SCSI fencing, using the SPC-3 (SCSI Primary Command) primitives. The whole objective is to prevent 2 nodes or hosts to writing to the shared storage volume at the same time and other issues like split-brain.
Diving into Clustered SMB’s CTDB
Clustered SMB is providing SMB (Server Message Block) file sharing services with high availability. I also want to point out that what I am writing here is generic Clustered SMB, because each vendor (like Microsoft®, SuSE®, Red Hat® and even iXsystems™) has a different approach on how to design and deliver the solution. Most of these deep dive details are from Samba, the open source implementation of the SMB protocol, notably from the Samba website and the implementation of CTDB (Cluster Trivial Data Base) in the CTDB website. Also note that the implementation of Clustered SMB is using Samba version 4.
Clustered Samba
As with most clustered systems, where storage resources are shared among the nodes of the cluster, the most important fundamental requirement is to prevent several nodes to write to the same storage segment in the shared storage simultaneously. A fencing capability is required because without this “fencing” capability, the storage bits committed by one node could easily be overwritten by another node. This could lead to data integrity problems, significantly data corruption. (Go back to the SCSI-3 reservation mechanisms I brought up earlier in the previous section of this blog).
The mechanism to establish the “fencing” in Clustered SMB is using the Clustered Trivial Database (CTDB). Samba uses Trivial Database (TDB) as the key registry to handle the metadata, the coordination and the management of information related to SMB file sharing, access control and others. This TDB is local to the Samba server. CTDB extends the capability in a clustered environment.
CTDB provides a transient database-like (hence trivial) to keep the information and metadata related to cluster operations. It enables HA capabilities to function within the cluster, monitors nodes, and handles cluster resiliency. Most importantly, it provides a messaging subsystems that coordinates and harmonize the file locking mechanisms such as byte-range locking, opportunistic locking and to prevent the different nodes to write and overwrite to same record location at the same time. This is the critical fencing part, and execute with speed and precision that its functions are totally seamless and transparent in a clustered SMB file service environment.
To be clear, CTDB in itself is not the clustered filesystem. CTDB plays the central role to enable Clustered SMB but in its documentation, CTDB also supports other cluster-aware filesystems such as NFS, RedHat® GFS (nee Sistina), IBM® GPFS, OCFS2 (Oracle® Cluster File System) and Lustre®.
[ Note: The scale-out design I described for Clustered SMB above is a shared storage architecture. There are other school of thoughts using the shared nothing architecture for distributed clusters, and federated clustering. TrueNAS® SCALE is very flexible, and would be able to address these architectures. Gluster in TrueNAS® SCALE is a use case of a shared nothing architecture using an on-the-fly metadata hashing method.
Clustered SMB getting the TrueCommand power soon
It is widely known that the TrueCommand software will be the central nervous system of the clustered, distributed and scale-out storage service. The few versions of TrueCommand was to provide NAS fleet management, where as many as 500 TrueNAS® systems (CORE, Enterprise and SCALE) can be monitored through a single pane of glass. TrueCommand provides single sign-on (SSO), RBAC (Role-based Access Control), an enhanced statistics, analytics and reporting engine and many more features.
When TrueCommand 2.2 comes out in this month of July 2022, scale-out storage administrators will be able to create and deploy Clustered SMB services via the intelligent wizards in the GUI (graphical user interface). As TrueNAS® SCALE matures with newer releases and iXsystems™’ next-gen technologies (hush hush for now), TrueCommand will be the ultimate C&C (Command and Control) for all things TrueNAS®.
Why Clustered SMB?
It is natural to provide scale-out capabilities when TrueNAS® SCALE was conceived. GlusterFS and Clustered SMB are the first out of the gates, and we will see more scale-out file services coming soon. As the world demands more and more file storage capacity, the requirements of both performance and file services resiliency grow with it as well. And Clustered SMB is provided to meet with this demand.
The Power to Scale
There is a whole new world coming into the world Open Source storage and TrueNAS® SCALE is leading the way. That is because we have niched open source storage technology dedicated to a niched segment of the industry such as the venerable Lustre® filesystem, that is often implemented on COTS x86. But not many storage vendors that can claim to run their open source storage on small scale VMs with minimal processing footprint to the largest of clusters. TrueNAS® SCALE can do that in my books because I have deployed TrueNAS® SCALE in 4GB, 1 virtual CPU Virtualbox® VMs in my homelab. I deployed a decent 6-node Gluster cluster on TrueNAS® SCALE running at home a few months back.
The flexibility and the elasticity in TrueNAS® SCALE with Clustered SMB allows end users to start small, and gradually, grow to the a scale-out SMB file service that can serve thousands, and even tens of thousands of SMB clients. This is the power of SCALE.
iXsystems TrueNAS has both scale-up and scale-out technologies
The next release, nicknamed Bluefin, will be even more exciting. I am hyper-optimistic of the future of TrueNAS®.
