Storage Facebook likes

There is a mini revolution going on, and Facebook is the main force driving it.

It is the Open Compute Project (OCP), and its mission is to redesign the modern-day data centers and drive open hardware and architectural designs and specifications, including storage. The overall goals are to drive greater data center efficiency, flexibility, energy savings and cost effectiveness in a new class of “hyperscale” datacenters. Facebook, Google and Amazon are some of the examples of hyperscale datacenters, where their businesses relies on massive computing power, exponential storage performance and racks and racks of computing infrastructure to drive their web-computing or cloud-computing services.

Some of the cool technology innovations in mind includes having systems that support any CPUs from any vendors including Intel and AMD. We may even see both processor brands running on the same motherboard. The Open Common Slots component for processors is based on PCIe. Intel has pledged their Decathlete motherboard specifications for OCP and likewise AMD has produced its Roadrunner mobo series specification for the project as well. The ARM processor could also be supported in the near future in this “mix-and-match” OCP ideals.

Other proposed changes include OpenRack specifications, “sleds”, and of course, the Open Vault project for storage (aka “Knox”). Continue reading

AoE – All about Ethernet!

This is long overdue.

A reader of my blog asked if I could do a piece on Coraid. Coraid who?

This name is probably a name not many people heard of in Malaysia. Even most the storage guys that I talk to never heard of it.

I have known about Coraid for a few years now (thanks to my incessant reading habits), looking at it from nonchalant point of view.  But when the reader asked about Coraid, I contacted Kevin Brown, CEO of Coraid, whom I am not exactly sure how I was connected through LinkedIn. Kevin was very responsive and got one of their Directors to contact me. Kaushik Shirhatti was his name and he was very passionate to share their Coraid technology with me. Thanks Kevin and Kaushik!

That was months ago but the thought of writing this blog post has been lingering. I had to scratch the itch. 😉

So, what’s up with Coraid? I can tell that they are different but seems to me that their entire storage architecture is so simple that it takes a bit of time for even storage guys to wrap their head around it. Why do I say that?

For storage guys (like me), we are used to layers. One of the memorable movie quotes I recalled was from Shrek: “Orges are like onions! Onions have layers!“.

Continue reading

Say VDI very fast

This one bugs me.

All the talk about Virtual Desktop Infrastructure (VDI) and how VDI is the next IN thing is beginning to look like hulla baloo to me. Every storage vendor in town is packaging their VDI messaging in the best gift wrapping paper possible, trying to win the hearts of potential customers. But I have a creeping feeling that the customers in Malaysia and even perhaps some in the region are going to be disappointed when all the fluff and huff of VDI meets reality.

I have to admit that I have no experience with VDI. I have no implementation experience, and I have no selling experience of VDI, but having gone through the years looking and observing at the centralized computing and thin client space, history could be repeating itself (again!). Many previous pre-VDI experiences have fallen flat on the face.

Remember the days of X-terminals, early versions of thin clients? Remember the names such as NCD (Network Computing Devices), Wyse Technologies (they were recently acquired by Dell), SCO Tarantella and the infamous Javastation? I don’t know about you, but that Javastation design was one ugly motherf****r.

So, it is my pleasure to remind you again and hopefully give you some nightmares too 😉

Back to VDI. Yes, the thin-client/zero-client/remote desktop/VDI concept is a great idea! I would have love VDI to be successful. It will be the implementation and the continuous user complaints that will be the bane of its problems. Ultimately, it’s the user’s experience that counts. Continue reading

Can VSA help NetApp?

Almost a year ago, I had an interview with VMware Malaysia for a Senior SE position. They wanted a pre-sales guy who knows Oil & Gas and a strong technology background. I had a strong storage background, and I was involved in Oil & Gas upstream since my NetApp and EMC days.

I thought I was their guy having being led to believe (mostly by my own self-belief) to be so. I didn’t get the job but I did not find out the reason why I lost the opportunity. But I remembered well that I brashly mentioned to the Australian interviewer over the phone that VMware could become the next “storage technology” company. At that time, VMware just launched their VMware 5.0 and along with it, their vSphere Storage Appliance (VSA). This was a turning point of the virtual storage appliance space.

My friend, whose company is a VMware partner, said that the list price for the vSphere VSA was USD5,000.00 a pop. The price wasn’t too bad to the small-medium-enterprise businesses in Malaysia, minus the hardware and storage capacity cost. But what intrigued me back then was this virtual storage appliance concept was disruptive.

VMware could potentially take large JBOD farms, each for the minimum of 3 physical ESXi nodes and build a shared storage using the vSphere Storage Appliance (VSA). Who needs shared iSCSI or Fibre Channel LUNs anymore if VMware had its way?

But VMware still pretty much depended on their storage partners, especially its master, EMC and so I believe VMware held back pushing VSA for the reason of allowing its storage partner ecosystem to thrive. And for that reason, the vSphere Storage API such as VAAI and VASA were developed since vSphere 4 to enhance the deeper integration of these storage vendor’s technology into the VMware world.

But of course, long before the VMware’s VSA venture, HP LeftHand already had one on the cards. The LeftHand Virtual SAN Appliance (also VSA) was already getting rave comments from their partners and customers, impressed with how they were able to showcase HP LeftHand storage solution and technology brilliantly. Eventually, HP recognized the prowess of the LeftHand VSA and started marketing it as HP StoreVirtual VSA. I don’t hear much about the HP LeftHand (since has been renamed as P4000) VSA nowadays, seeing the HP guys in Malaysia preferring to pitch the physical storage than the virtual storage software.

NetApp, back in Q1 of 2012, also decided to go down the path of virtual storage appliance, announcing the ONTAP-v to the world here. It was initially resold through the Fujitsu partnership, but the Q1 announcement expands the ONTAP-v to a larger set of server vendors as shown below. The key component is to have a qualified RAID controller in each of the server vendors.

Continue reading

The beginning of the end of FCoE

Never bet against Ethernet!

I am sure many IT experts and practitioners would agree. In the past 30 years or so, Ethernet has fought and won against many so-called would be “Ethernet killers”. The one that stood out for me was ATM (Asynchronous Transfer Mode) because in my past job, I implemented NFS over ATM, running in LANE (LAN Emulation) mode in a NetApp filer setup in Sarawak Shell.

That was more than 10 years ago. And 10 years ago, ATM was hot technology. It was touted as the next generation network technology and supposed to unify the voice, data and network together. ATM also had better framing and QOS (Quality-of-Service) control and offers several modes of traffic shaping and policies. And today, ATM is reduced to a niche telecommunication protocol, and do not participate much in the LAN technology space.

That was the networking space. The storage networking space is dominated by Fibre Channel for almost 15 years. Fibre Channel is a serial technology that replaced the channel-based technology of SCSI in the enterprise. And Fibre Channel has also grown leaps and bounds, dominating the SAN (Storage Area Network) landscape with speeds up to 16Gbit/sec today.

When the networking world and storage networking world collided (I mean combined) with Fibre Channel over Ethernet (FCoE) technology some years back, one has got to give some time soon. Yup, FCoE was really hot 2 years ago, but where is it today? Is Cisco still singing about FCoE like it used to? What about the other storage vendors that used to have at least 1 FCoE slide in their product presentation?

Welcome to the world of IT hypes! FCoE benefits? Ability to carry LAN and SAN traffic with one piece of wire. 10 Gigabit-style, baby!

Continue reading

“I want to put in my own hard disk”

I want to put in my own hard disk“.

If a customer ever utter that sentence, it will trigger a storage vendor meltdown. Panic buttons, alarm bells, and everything else that will lead a salesman to go berserk. That’s a big NO, NO!

For decades, storage vendors have relied on proprietary hardware to keep customers in line, and have customers continue to sign hefty maintenance contracts until the next tech refresh. The maintenance contract, with support, software upgrades and hardware spares replacement, defines the storage networking industry that we are in. Even as some vendors have commoditized their hardware on the x86 platforms, and on standard enterprise hard disk drives (HDDs), NICs and HBAs, that openness and convenience of commodity hardware savings are usually not passed on the customers.

It is easy to explain to customers that keeping their enterprise data in reliable and high performance storage hardware with performance optimization and special firmware is paramount, and any unwarranted and unvalidated hardware would put the customer’s data at high risk.

There is a choice now. The ripple of enterprise-grade, open storage kernel and file system has just started its first ring, and we hope that this small ripple will reverberate across the storage industry in the next few years.

Continue reading

SMP than VMware

VMware is not a panacea for all your server virtualization requirements but because they do fantastic marketing (not to mention doing 1 small seminar every 1.5-2 months here in Malaysia last year), everyone thinks they are the only choice for server virtualization.

Efforts from Citrix Xen, Microsoft Hyper-V and RedHat Virtualization do not seem to make a dent into VMware’s armour and it is beginning to feel that VMware is the only choice for server virtualization. However, every new server virtualization proposal would end up with the customer buying a brand new, much more powerful server. More CPUs, more cores, and more RAM (I am not going into VMware vRAM licensing issues here but customers know they are caged-in).

You see, VMware’s style of server virtualization is a in-system virtualization. The amount of physical resources within the system are being pooled, virtualized and shared with the virtual machines (VMs) in the physical chassis. With exception to the concept of distributed vSwitches (dvSwitch), CPUs, processing CPU cores and RAM are pretty much confined within what’s available in the physical box in most server virtualization environment. You can envision the concept of VMware’s in-system virtualization in the diagram below:

So, the consolidation (and virtualization) phase of older physical servers would involve packing tons of CPU cores and tons of RAMs in a newer, high end server.

I just visited a prospect a few days ago. For about 30 users for an ERP system and perhaps 100 users of Zimbra mailboxes, he lamented that he had to invest into 2 Dell R710 servers with 64GB of RAM each and sporting 2 x 8-core Intel Xeon. That sounded to like an overkill but that is what is happening here in this part of the world. The customer is given the perception and the doubt of inadequacy when they virtualize their servers. “What if I don’t have enough cores?; what if I don’t have enough RAM?” That in itself is the typical Malaysian (and Singaporean) kiasu mentality. Check out the Wikipedia definition of kiasu here.

Such a high-end server costs a lot of moolahs. And furthermore, the scalability and performance of the virtualized servers in the VMs are trapped within how much these servers can scale physically. If the server is maxed out at 16-cores and 128GB of RAM, then the customer to upgrade again with a server forklift. That’s not good.

And one more thing. VMware server virtualization is not ready for High Performance Computing (HPC) …yet.

Let’s look at this in another way. Let’s assume that you can look the server virtualization approach in an outward manner rather than the inward within kind of thinking, like the VMware in-system method.

What if you can invest in lower-end x86 servers with 1 x quad-core CPUs, with 8GB of RAM? What if you can put aggregate many of these lower-end servers together and build a large cluster of lower-end x86 servers into a huge symmetric multiprocessing server farm that supports 1,024 CPUs of 16,384 cores, 64TB of RAM? Have a look at this video that explains what I just mentioned:

ScaleMP video

Yeah, yeah .. it’s a marketing video from ScaleMP. But I am looking beyond the company and looking at the possibility of this out-system type of server virtualization. The ability to pool together all the CPU processing power of many physical servers and the aggregation of physical RAMs of all the combined servers into a single shared memory architecture unleashes the true power of server virtualization. This is THE next generation symmetric multiprocessing (SMP) architecture, and it breaks free from the limitations and scalability the in-ward virtualization of physical servers.

In the past, SMP system rely on heavy programmability of the applications to scale with SMP systems. Applications didn’t necessary scale on-the-fly with SMP systems, and some level of configuration and programming have to be applied to address the proprietary  SMP methods and interconnects. ScaleMP’s vSMP Foundation hypervisor solution removes the proprietary nature of SMP and bringing x86 server virtualization to meet the demands of HPC.

Here’s a look at the high level architecture of ScaleMP vSMP:

This type architecture brings similarity to RNA Networks solutions that I blogged some time ago. RNA Network, which was acquired by Dell late last year, based their solution on the RDMA technology and protocol, and was more about enhancing scalability and performance with memory pooling via Memory Cloud. ScaleMP’s patent-pending technology is more than that. It pools both memory and processing cores as well, giving it greater scalability and performance, the much needed resources for the demands of HPC environments.

The folks at ScaleMP contacted me a couple of weeks back and shared some of their marketing datasheets and whitepapers. While the information passed to me were OK, I wish the information could have a deeper dive into the technology and implementation as well. I hope they could share it, and I don’t mind signing an NDA.

Well, this is done pro bono, because I want everyone to know the choices and possibilities out there. It is my worldly cause to have people educated because only by being informed, we make better choices. The server virtualization world isn’t always about VMware, you know.

Feed us with Filr

I have checked about the progress of Novell Filr, which has generated a lot of buzz on the web. I blogged about it here and here and I was hoping to get a job to review the product. But I didn’t get the job yet, because the product will only be available in Q4 of 2012.

That’s a long time to come to market, considering that from the time it was announced in Novell BrainShare in November 2011. And the competitors are gearing up as well. There is Dropbox for the enterprise called Dropbox for Teams, and VMware is doing something along the same lines called Project Octopus. I am pretty sure there are plenty of other vendors who are already offering something equivalent to what the Novell Filr can do.

I browsed around the web for more info about the Novell Filr, hoping that it won’t be like a blackhole after Novell’s announcement. Fortunately, I found more details which I thought was interesting but it took a while after 5 or 6 Google pages.

The set of slides I found belonged to Anthony Priestman of Novell. The slides started with the Novell Filr ease of installation.

  • Local Administrator/Password
  • Server Name
  • IP Address
  • Finalize the configuration with a browser

In a nutshell Novell Filr is a virtualization service. It virtualizes the backend NTFS shares, CIFS shares, identity management through Active Directory or Novell eDirectory, as well as access control and security to present a “merged-view” of files and folders from different disparate sources.

Going deeper, the Novell Filr becomes the orchestrator and broker, linking up the backend to the front end with ease and simplicity. Even though it sounds easy, the entire architecture and its implementation is complicated because there are so many components and services involved.

Therefore, to make file services and authentication services matters easy and simple should be considered genius, and we shall see how Novell Filr pans out when it is released. I have no doubt in my mind that it will be easy and simple.

Here’s a deeper look at the architecture:

The Filr integrates with both Novell eDirectory and Windows Active Directory for authentication and file access control. I

One of the new concepts is called File Spaces. This is great, because this is going to do away with drive letters that we are so used to in mapped drives concept in Windows. There is a running joke that the number of mapped drives in Windows will run out after the letter Z:.

File Spaces allows a simple folder to represent any Windows shares, NAS CIFS shares or Novell NSS volumes. This is based on UNC (Universal Naming Convention), so it is straightforward. File Spaces also allows users to right-click to share their files easily, probably similar to how you share Google Docs files when you want to invite team members to collaborate on files. And it will update you on notifications after files have been updated or modified. This ease-of-sharing, of course, is governed with higher, company-wide policies about file sharing, both internally and externally (across firewalls as well)

Both the most powerful feature of File Spaces is the ability to have a single, “merged-view” of all files and folders with all types of devices, from tablets, to smartphones to laptops. The slide below explains a bit of File Spaces “Merged Views”:

The view of files and folders will look like the following below:

The Novell Filr concept and technology is going to define the new file sharing, home and user directories landscape in IT. The archaic concept of mapped drives will slowly fade away, and the Filr will bring forth new frontiers of tight and secure enterprise user and resource management, but with the ease of use and simplicity of sharing concepts of social media. 

Some older implementations from Novell will eventually be replaced by Filr. iFolder, Netstorage, and QuickFinder will go the way of the dodos, while the next generation Filr will become the flagship of the new Novell.

This sounds dandy. Unfortunately, I personally am worried about how Novell’s new owner, Attachmate will be good to Novell. Right now, the future of Novell seems like business as usual but that’s not good. Novell has been losing mindshare and they had better make their stand with a strong product like Novell Filr.

Like I said earlier, the product is shipping in late of 2012. It was announced in late 2011. That’s a whole 12 months in which Novell could do much better by feeding the minds of followers of Novell Filr. Let them and people like me, learn more about the technology. Let us help spread the idea and word and keep the Novell Filr interests up and the fire burning until the launch date.

NFS-phobic in Malaysia

I taught the EMC Cloud Infrastructure and Services (CIS) class last week and naturally, a few students came from the VMware space. I asked how they were implementing their storage and everyone said Fibre Channel.

I have spoken to a lot of people about this as well in the past, whether they are using SAN or NAS storage for VMware environments. And almost 99% would say SAN, either FC-SAN or iSCSI-based SAN. Why???

When I ask these people about deploying NFS, the usual reply would be related to performance.

NFS version 3 won the file sharing protocol race during its early days where Unix variants were prevalent, but no thanks to the Balkanization of Unices in the 90s. Furthermore, NFS lost quite a bit of ground between NFSv3 in 1995 and the coming out party of NFSv4.1 just 2 years ago. The in-between years were barren and NFS become quite a bit of a joke with “Need For Speed” or “No F*king Security“. That also could be a contributing factor to the NFS-phobia we see here in Malaysia.

I have experiences with both SAN and NAS and understood the respective protocols of Fibre Channel, iSCSI, NFS and CIFS, and I felt that NFS has been given unfair treatment by people in this country. For the uninformed, NFS is the only NAS protocol supported by VMware. CIFS, the Windows file sharing protocol, is not supported, probably for performance and latency reasons. However, if you catch up with high performance computing (HPC), clustering, or MPP (Massively Parallel Processing) resources, almost always you will read about NFS being involved in delivering very high performance I/O. So, why isn’t NFS proposed with confidence in VMware environments?

I have blogged about this before. And I want to use my blog today to reassert what I believe in and hope that more consideration can be given to NFS when it comes to performance, even for virtualized environments.

NFS performance is competitive when compared to Fibre Channel and in a lot of cases, better than iSCSI. It is just that the perception of poor performance in NFS is stuck in people’s mind and it is hard to change that. However, there are multiple credible sources that stated that NFS is comparable to Fibre Channel. Let me share with you one of the source that compared NFS with other transport protocols:

From the 2 graphs of IOPS and Latency, NFS fares well against other more popular transport protocols in VMware environments. Those NFS performance numbers, are probably not RDMA driven as well. Otherwise RDMA could very well boost the NFS numbers into even higher ground.

What is this RDMA (Remote Direct Memory Access)? RDMA is already making its presence felt quietly, and being used with transports like Infiniband and 10 Gigabit Ethernet. In fact, Oracle Solaris version 11 will use RDMA as the default transmission protocol whenever there is a presence of RDMA-enable NICs in the system. The diagram below shows where RDMA fits in in the network stack.

RDMA eliminates the need for the OS to participate in the delivery of data, and directly depositing the data from the initiator’s memory to the target’s memory. This eliminates traditional networking overheads such as buffers copying and setting up network data structures for the delivery. A little comparison of RDMA with traditional networking is shown below:

I was trying to find out how prevalent NFS was in supporting the fastest supercomputers in the world from the Top500 Supercomputing sites. I did not find details of NFS being used, but what I found was the Top500 supercomputers do not employ Fibre Channel SAN at all!  Most have either proprietary interconnects with some on Infiniband and 10 Gigabit Ethernet. I would presume that NFS would figure in most of them, and I am confident that NFS can be a protocol of choice for high performance environments, and even VMware environments.

The future looks bright for NFSv4. We are beginning to see the word of “parallel NFS (pNFS)” being thrown into conversations around here, and the awareness is there. NFS version 4.2 is just around the corner as well, promising greater enhancement to the protocol.

 

Why VAAI?

This is Part 2 of my previous blog about VAAI (vStorage API for Array Integration) with more details about VAAI. VAAI offloads some of the I/O related functions to the VAAI-enable storage array, hence giving the hypervisor more compute and memory resource to do it other functions. And the storage array, upon receiving the VAAI command, will execute whatever that is required of it.

Why is VAAI important? What does it do that makes it so useful and important to the hypervisor?

VAAI is about a set of new SCSI commands. And there are 3 important ones:

  • WRITE-SAME
  • XSET
  • ATS

What exactly do these SCSI commands do?

WRITE-SAME is a SCSI command that instructs the storage array to zeroes the virtual VMDK disks or VMFS LUNs. This usually happens when guest OS require a brand new set of virtual disks and initializing the virtual disks is required. In the past (before VAAI), the hypervisor has to repetitively send 0s to the storage to perform the disks zeroing. As shown in the diagram below, you can see that each zero operation is sent from the hypervisor to the storage.

This back-and-forth of sending 0s and acknowledgments between the hypervisor and the storage is not efficient. With VAAI, the command WRITE-SAME  is sent from the hypervisor to the storage array and the storage array will do the zeroing on the disks and LUNs. The hypervisor will not intervene with the process until it gets and acknowledgment of its completion. See diagram below of how VAAI helps in bulk-zeroing of disks and LUNs in the storage array.

The animated GIFs are the taken from Luke Reed’s blog, a fantastic read.

The second VAAI SCSI command is XSET and it performs hardware accelerated full copy. This command is also known as  XCOPY and it offloads the process of copying the blocks of data that makeup a VMDK file. Such copying operations occur when the hypervisor is doing things like VM cloningStorage vMotion or VM creation from templates (bulk copying to create many similar VMs in one go).

Again with the courtesy of Luke Reed’s animated GIFs, the diagram below shows a full copy without VAAI

and after implementing VAAI, where the full, bulk copy operations is offloaded to the storage array to execute.

The third and last SCSI command of VAAI is ATS or hardware-assisted locking. ATS stands for Atomic, Test and Set and the command allow the hypervisor to lock only the required blocks rather than the entire LUN.

Without VAAI, the entire LUN temporarily could be locked by the numerous VMFS operations of one single hypervisor and this prevents other hypervisors from accessing the shared LUNs. The ATS API offloads lock management from the host to the storage array and keeps the LUN available by locking only required blocks, not the entire VMFS file system. Please see the pleasing diagrams below of

(without VAAI ATS)

(with VAAI ATS)

And if you want to see the VAAI Hardware Accelerated Full Copy (aka XSET) in action, here’s a little video showing how it is done in an EMC environment.

The primary significance and noticeable benefit is definitely performance. The secondary benefit, though not so obvious, is allowing VMware and its hypervisor to scale because it does not get bogged down by some of the I/O functions that it is not meant to do.

There were some new additions in vSphere 5.0 for VAAI. From its FAQ, it listed in ESX5.0, support for NAS Hardware Acceleration is included with support for the following primitives:

  • Full File Clone – Like the Full Copy VAAI primitive provided for block arrays, this Full File Clone primitive enables virtual disks to be cloned by the NAS device.
  • Native Snapshot Support – Allows creation of virtual machine snapshots to be offloaded the array.
  • Extended Statistics – Enables visibility to space usage on NAS datastores and is useful for Thin Provisioning.
  • Reserve Space – Enables creation of thick virtual disk files on NAS.

So, there you have it folks. Why VAAI? Here’s why.