ZFS Replication and Recovery with FreeNAS

We get requests to recover data from a secondary platform all the time. RPO (recovery point objective) of 30 minutes can be challenging to small to medium sized companies, especially if there is an SLA (service level agreement) to meet.

This week, my team and I took some time to create a FreeNAS replication demo for a potential client. I thought I document the whole thing about ZFS replication, the key steps to set it up and show how recovery is done.

ZFS Snapshots

ZFS replication relies on periodic ZFS snapshots. ZFS snapshot is an inherent feature from the ZFS file system, and often used as a point-in-time copy of the existing ZFS file system tree in memory. Once a snapshot has been triggered, either manually or on schedule (periodic), the file system tree and its metadata in the memory are committed to disk to ensure an updated and consistent state of the file system at all times.

To start, a running snapshot policy on a schedule must be in place. This snapshot policy can be on a specific dataset or zvol, or even the entire zpool. Yeah, I am using quite a few ZFS terminology here – zpool, zvol, dataset. You can read more about each of the structures and more here.

Once the ZFS replication task has been setup, every snapshot occurred in the snapshot policy is automatically duplicated and copied to the target ZFS dataset. Usually, the target ZFS dataset is on a secondary FreeNAS storage server, serving as a disaster recovery platform. Sending and receiving data in the snapshots rely on SSH service.

This is the network diagram explaining the FreeNAS ZFS replication setup.

Continue reading

Veaam to boost Cloud Data Management

Cloud Data Management is a tricky word. Often vague, ambigious, how exactly would you define “Cloud Data Management“?

Fresh off the boat from Commvault GO 2019 in Denver, Colorado last week, I was invited to sample Veeam a few days ago at their Solution Day and soak into their rocketing sales in Asia Pacific, and strong market growth too. They reported their Q3 numbers this week, impressing many including yours truly.

I went to the seminar early in the morning, quite in awe of their vibrant partners and resellers activities and ecosystem compared to the tepid Commvault efforts in Malaysia over the past decade. Veeam’s presence in Malaysia is shorter than Commvault’s but they are able to garner a stronger following with partners and customers alike.

Continue reading

Brainy Commvault

[Disclosure: I was invited by Commvault as a Media person and Social Ambassador to their Commvault GO 2019 Conference and also a Tech Field Day eXtra delegate from Oct 13-17, 2019 in the Denver CO, USA. My expenses, travel, accommodation and conference fees were covered by Commvault, the organizer and I was not obligated to blog or promote their technologies presented at this event. The content of this blog is of my own opinions and views]

The waltz across the Commvault-Hedvig mine field will not be easy. Commvault will have a lot of open discussions about their acquisition of Hedvig and how Hedvig “primary storage platform” will fit into a “secondary storage framework” of Commvault. The outcome of this consummation is yet to appear as a structured form. The storyline will eventually form as Commvault’s diligence to define their strategy moving forward.

Day 1

Day 1 was my open day at Commvault GO. I was absorbing the first impressions of Commvault again even though this was my third Commvault GO, after Washington DC and Nashville in 2017 and 2018 respectively. There was certainly a “startup” feeling again in Commvault since the appointment of Sanjay Mirchandani as CEO 9 months ago.

A lot of excitement and buzz were generated around the metallic, the Commvault venture into Software-as-a-Service (SaaS). The SaaS solution is targeted at the mid-market for organizations with 500-2500 staff count. Its simplicity and pricing were the 2 things which gave me a good feeling all over. There is even a 45-day trial for metallic.

Getting Brainy

My Day 2 itinerary was more specific because my agenda for this trip was to seek answers to the realization of Commvault-Hedvig.

Commvault took the distinction of using the vision of a DataBrain (#databrain) to define their strategy. From the picture below, the left and right hemisphere of the DataBrain forms the Storage Management piece on the left and Data Management on the right.

Continue reading

Commvault coming all together

[Disclosure: I was invited by Commvault as a Media person and Social Ambassador to their Commvault GO 2019 Conference and also a Tech Field Day eXtra delegate from Oct 13-17, 2019 in the Denver CO, USA. My expenses, travel, accommodation and conference fees were covered by Commvault, the organizer and I was not obligated to blog or promote their technologies presented at this event. The content of this blog is of my own opinions and views]

This trip to the Commvault GO conference was pretty much a mission to find answers to their Hedvig acquisition just a month ago. It was an unprecedented move for Commvault and I, as an industry observer and pundit, took the news positively. I wrote in my blog about Commvault’s big bet and I liked their boldness in their approach.

But the news did not bode well back here in Malaysia. The local technology news portal, Data Storage Asean picked up the news in a rather unconvinced way. 2 long time Commvault partners I spoke to were obviously unhappy because the acquisition made little sense to them on the back of closing of the Commvault Malaysia office just weeks before this with more unsettling rumours of the Commvault team in Asia Pacific. The broken trust and the fear of what the future held for the Commvault customers in Malaysia and in the region were riding along with me on this trip.

But I have seen the beginning of the Commvault transformation from the Commvault GO conferences I have attended since 2017. This is my 3rd Commvault GO and I ended Day 1 with good vibes.

Here were some of my highlights in the first day. Continue reading

Data Renaissance in Oil and Gas

The Oil and Gas industry, especially in the upstream Exploration and Production (EP) sector, has been enjoying a renewed vigour in the past few years. I have kept in touch with the developments of the EP side because I always have a soft spot for the industry. I have engaged in infrastructure and solutions in the petrotechnical side in my days at Sun Microsystems back in the late 90s. The engagements with EP intensified in my first stint at NetApp, wearing the regional Oil & Gas consulting engineer here in South Asia for almost 6 years. Then, with Interica in 2014, I was dealing with subsurface data and seismic interpretation technology. EP is certainly an exciting sector to cover because there are so much technical work involved and the technologies, especially the non-IT, are breath taking.

I have been an annual registrant to the Digital Energy Journal events since 2013, except last year, and I have always enjoyed their newsletter. This week I attended Digital Energy 2-day conference again, and I was taken in by the exciting times in EP. Here are a few of my views and trends observation in this data renaissance.

Continue reading

Commvault big bet

I woke up at 2.59am in the morning of Sept 5th morning, a bit discombobulated and quickly jumped into the Commvault call. The damn alarm rang and I slept through it, but I got up just in time for the 3am call.

As I was going through the motion of getting onto UberConference, organized by GestaltIT, I was already sensing something big. In the call, Commvault was acquiring Hedvig and it hit me. My drowsy self centered to the big news. And I saw a few guys from Veritas and Cohesity on my social media group making gestures about the acquisition.

I spent the rest of the week thinking about the acquisition. What is good? What is bad? How is Commvault going to move forward? This is at pressing against the stark background from the rumour mill here in South Asia, just a week before this acquisition news, where I heard that the entire Commvault teams in Malaysia and Asia Pacific were released. I couldn’t confirm the news in Asia Pacific, but the source of the news coming from Malaysia was strong and a reliable one.

What is good?

It is a big win for Hedvig. Nestled among several scale-out primary storage vendors and little competitive differentiation, this Commvault acquisition is Hedvig’s pay day.

Continue reading

Perils of avoiding BC and DR

News in recent months have been unfavourable, even to the point of poignancy. Maybe I didn’t have all the details to place my opinion, but it has appeared that these recent events have neglected the practice of  BC (business continuity) and DR (disaster recovery).

The recent bad news

The most recent is one close to home. The KLIA (Kuala Lumpur International Airport) and KLIA2 operations were disrupted quite significantly for 4 days due to “network switch” failure. I followed the news and comments quite intently in those bad days, and I did not see any single comment discussing about BC or DR. If BC and DR were present at the airports, the airport operations would have been restored within minutes or hours, not days. Investigations are still on-going to find out what really happened in the KLIA/KLIA2 incident.

Continue reading

The waning light of OpenStack Swift

I was at the 9th Openstack Malaysia anniversary this morning, celebrating the inception of the OpenInfra brand. The OpenInfra branding, announced almost a year ago, represented a change of the maturing phase of the OpenStack project but many have been questioning its growing irrelevance. The foundational infrastructure components – Compute (Nova), Image (Glance), Object Storage (Swift) – are being shelved further into the back closet as the landscape evolved in recent years.

The writing is on the wall

Through the storage lens, I already griped about the conundrum of OpenStack storage in Malaysia in last year’s 8th anniversary. And at the thick of this conundrum is OpenStack Swift. The granddaddy of OpenStack storage has not gotten much attention from technology vendors and service providers alike. For one, storage vendors have their own object storage offering, and has little incentive to place OpenStack Swift into their technology development. Continue reading

Digital Transformation means Change in People

I wrote about Digital Transformation a few weeks ago. In the heart of it, People are the real key to the transformation of every organization. Following up what I described earlier, Change is the factor that People in every organization have to embrace.

Drowning and going blind

We are swarmed by technology. We are inundated with everything digital and we are attracted to the latest buzz and hype. In the sea of it all, these things have made us, the People reliant of technology. This reliance, this needy dependency, has made us complacent. We settle because the boring and mundane tasks have been taken away from us. Moreover, the constant firehose feeding our lives has created “digital drowning“, a situation I would like describe as gasping for a breather to think clearly. We are bogged by digital quagmire, blinded by what shiny things and we lose sight of the strategic focus.

We shrivel and we go back to what we think is our comfort zone.

Change is constant and uncomfortable

I once read that our known comfort zone is no longer our safety zone. That idea of everyone’s safety zone has been obliterated aeons ago. I love the following quote from Seth Godin, my absolute marketing guru.

No alt text provided for this image

As he rightly pointed out, “There is no ‘ever after’. There’s just the chaos of now“. We don’t arrive at a comfortable place after the change. There is no comfortable place or safety place for that matter … at all. The Digital Transformation or what ever Information Age we described our generation earlier, is constant change. We have to ride the hungry bear and we have to saddle the ferocious dragon at all times. We have to learn to ride the bucking bronco!

So, we learn. We change and change. Continue reading

Secure Private Sync and Share with EasiShare

Shadow IT /sh-A-doE  Eye-Tee/

noun: An IT project outside the organization IT department’s domain and often unapproved. A dark area.

verb: A defiant user-level practice to perform IT activities where the organization’s IT department has little or no control.

Shadow IT or Stealth IT

There was a BYOD (bring your own device) craze about a decade ago. The darling of the BYOD craze, Dropbox was on every vendor’s lips and many look-a-likes sprouted like mushrooms. Microsoft OneDrive (previously known as SkyDrive), Google Drive, and of course, Dropbox and many others are still serving a growing customer base, together with many others. But most of them have taken a different, more mature form, a market where Gartner has defined as Enterprise File Sync and Share several years ago. And today, that market is shifting again, and soon to be known as Content Collaboration Platform.

But Shadow IT remains where many users are facing challenges with their IT department. Rigid, archaic, and difficult have forced end users to take matters into their own hands to share files, away from the controls and structures. And those free GBs from those cloud storage providers looked so tempting …

The picture above is someone unlocking a safe. I have literally seen an IT department keeping their files on disks and then keep them in a safe! When they want to share it, they have to run the safe combinations to bring out the disks, and they did it in front of me. It was funny then but the paranoia is real! Some IT departments are really that pain-in-the-a$$.

A business risk

Shadow IT is a risk. Security is often the touted risk, but the issue goes beyond just security. Often, the compromised issue represents a degradation of the company’s brand, image and customer confidence, and could lead to negative reverberation of the company’s business.

Time to regain control and secure file access

EasiShare, a private military-grade, enterprise file sync and share platform is a solution I am exploring. It is similar to the Dropbox concept many are familiar with, but without the security concerns and heavy applications of Dropbox, OneDrive or Google Drive.

Many organizations in Malaysia have expressed concerns about data privacy and security. And this is a great opportunity for Malaysian companies to consider data privacy and security seriously, especially with Shadow IT looming to comprise the control of the IT departments.

Continue reading