I was up at about 1am several nights ago listening to the SNIA update of NFSv4 and it was worth losing sleep. I was both intrigued and interested to see a full scale NFSv4 adoption in the coming few years.
We all know that NFS has been around for more than 20 years already. Version 2 was released way back in 1989, with version 3 being around since 1995. That’s a long time, and it is beginning to show its age. NFS version 4 is not new as well. Believe or not, it was released in 2000, 11 years ago. But there was a significant update in 2010, with NFS version 4.1 and this came with parallel NFS (pNFS) support to address new requirements such as scale-out NAS and file striping across clusters of nodes.
I am doing my responsible bit to disseminate NFS version 4.x updates and moving the storage networking and filesystem community towards its imminent wide scaled adoption. This is one of the many entries I intend to share about NFS version 4.
So, why this librarian thingy? First of all, for those folks working on NFSv3, you would probably encounter issues about file locking and also high availability.
Don’t you just hate it when the server reboots, and your NFS mount point hangs? Depending if it was a hard mount or soft mount, the NFS retries could take forever or sometimes, the NFS clients just freezes. In additional to that, another frequent complaint is that NFSv3 has lousy file locking.
However, in the beginning years of NFS, the world was a very different place. Such issues about file locking and HA were very well addressed by NFSv2/v3 because the demands of the previous client-server world were lesser. As the world progressed in the 2nd millenium, NFS v2/v3 started to sputter.
In this lesson#1, I would like to share about 2 key features of NFSv4 to address the 2 issues I brought up – which is NFS HA and file locking. The 2 features are
- Lease
- Delegation
As I said, NFS high availability in version 3 was simplistic. In version 3, if an NFS client fails, the NFS server has little knowledge that the client has failed. Remember, NFSv3 is stateless. This can cause complications and issues such as ambiguity about file locking. Likewise, if an NFS server fails, the client could freeze and if recovered, could get stale NFS handles and have all sorts of problems related to file locking. Many locks have to be released before an application can restart properly.
In NFSv4, things related to either the NFS server or client failing with regards to file locking are much more simplified. The mechanism is leasing and both the client and server will know what happening to each other. NFSv4 is a stateful protocol. This is how leasing works:
- The NFS client leases a file lock from the NFS server for a certain period of time, eg. N seconds. It renews the lock with after the N seconds period has expired.
- If the NFS client fails, the lock is reclaimed by the server and released by the NFS server to other clients after a grace period
- If the NFS server fails and rebooted, all the files are locked for M seconds for the incumbent NFS clients to reclaim the locks. If they are not reclaimed by the respective client, the file lock is released.
Such an agreement with the stateful communication between the clients and the NFS server, makes file locking in an high availability environment much simpler and more robust.
Another new feature of NFSv4 is delegation. Part of the exported filesystem from the NFS server can be delegated or “loaned out” to the NFS client. The NFS client which had “borrowed” this piece of the filesystem can then work on it in its local cache with little communication to the NFS server (performance gain and reduced chattiness of the previous version). Here’s a step-by-step guide of this delegation process.
- NFS client request a piece of the filesystem. The NFS server “lends” this piece to the client if it is not already locked, as a lease.
- NFS client works on this piece in its local cache
- Once the NFS client has completed the writes and commits to the piece of filesystem on the local cache, it releases the “borrowing” lease back to the NFS server
- If other NFS clients requests for the same piece of filesystem while it out on loan, the NFS server would say “Sorry, I loaned it out”
- If there is a high order authority requesting for the piece of filesystem, the NFS server would say to the NFS client, “I need this back” and will send an order to recall the filesystem
In many ways, both file locking lease and delegation works like a librarian. Pieces of the filesystem are loaned to the requester for a lease period much like books are loaned to the borrower for a period of time.
Enjoy your weekend!
Pingback: NFSv4 – Your filesystem librarian | Storage news | Scoop.it
Hi,
Thanks for posting such a great article.
>3.If the NFS server fails and rebooted, all the files are locked for M seconds for the >incumbent NFS clients to reclaim the locks. If they are not reclaimed by the respective >client, the file lock is released.
1) How to define “M” sec ? Is this a grace period configuration at NFS server side ?
2) What happened if “M” sec is shorter than “90” sec which is very common value of
Grace period ?
If this Value is small, let’s say 10 sec, NFS clients fail to reclaim (get) the lock after
NFS server rebooting and what’s happened ?
Appreciate your comments.
Best Regards,
Hello,
Thanks for sending in your comments.
1) The RFC3530 of NFSv4 specs does not specify a default grace period and left the definition open to OS and apps implementers. Yes, this grace period value is defined on the NFS server end, not at the client to allow the NFS server to dictate how it wants to control its resources.
2) If I understand your question correctly, you are saying that the NFS server has a defined number of seconds but the NFS client failed to claim it. The NFS server will allow the lease to expire, and release the lock in that defined number of seconds and allow other NFS clients to grab the lease and lock the file or portions of it, depending on its respective implementation.
There are a lot of flexibility of how NFS lock grace period can be implemented because each vendor has specific reasons to define it longer or shorter.
Hope this helps.
Thanks
/Chin-Fah