Down the rabbit hole with Kubernetes Storage

Kubernetes is on fire. Last week VMware® released the State of Kubernetes 2020 report which surveyed companies with 1,000 employees and above. Results were not surprising as the adoptions of this nascent technology are booming. But persistent storage remained the nagging concern for the Kubernetes serving the infrastructure resources to applications instances running in the containers of a pod in a cluster.

The standardization of storage resources have settled with CSI (Container Storage Interface). Storage vendors have almost, kind of, sort of agreed that the API objects such as PersistentVolumes, PersistentVolumeClaims, StorageClasses, along with the parameters would be the way to request the storage resources from the Pre-provisioned Volumes via the CSI driver plug-in. There are already more than 50 vendor specific CSI drivers in Github.

Kubernetes and CSI initiative

Kubernetes and the CSI (Container Storage Interface) logos

The CSI plug-in method is the only way for Kubernetes to scale and keep its dynamic, loadable storage resource integration with external 3rd party vendors, all clamouring to grab a piece of this burgeoning demands both in the cloud and in the enterprise.

Continue reading

A Dialogue between 2 Drives

I was talking to an end user who was slowly getting exposed to the cloud amid this Covid-19 pandemic. The whole work from home thingy was not new to him, but the scale of the practice suddenly escalated when more than 80 of his staff have to work from wherever they were stuck at during the past 6 weeks. Initially all of his staff had to alternate their folders and files access because their Sonicwall® Global Client license and SSL VPN Clients were inadequate. Even after their upgrade of the licenses, the performance of getting the folders and files through the Z: drive was poor and the network was chocked up. I told them that regardless, the SMB protocol of the NAS shared folders was chatty and generated a lot of network traffic on the VPN, along with the inadequacies of running this over the wide area Internet network. Staff productivity obviously nosedived.

We are now exploring putting their work in the cloud but maintaining a consistent synchronized set of folders and files at all times. Wasabi® Cloud has emerged the most attractive price/GB/month and no egress or API requests fees.

Combining 2 shared drives into one

NAS Drive talking to Cloud Drive like 2 buddies

Now here is a story of 2 Drives

The end user is not an IT savvy user. They were unfamiliar with Cloud Storage other than the free personal ones like Google Drive, or Dropbox. They have more than 200TB and I have introduced to them Wasabi® Cloud. They were very familiar with their Z:, their NAS Drive. I introduced to them the Cloud Drive.

NAS: Hey, how’s it going?

Cloud: Not bad. My boss and your boss are talking about bringing me and Wasabi® Cloud to join your gang. Hope you are OK with that.

Continue reading

Cloud Sync Prowess of FreeNAS

The COVID-19 situation has driven technology to find new ways to adapt to the new digital workspace. Difficulty in remote access to content files and media assets has disrupted the workflow of the practitioners of many business segments. Many are trying to find ways to get the files and folders into their home computers and laptops to do work when they were used to getting them from the regular NAS shared drives.

These challenges have put hybrid cloud file sharing into the forefront, making it the best possible option to access the NAS folders and files inside and outside the boundaries of the company’s network. However, end users are pressured to invest into new technologies to adjust to this new normal. It does not have to be this way, because FreeNAS™ (and in that aspect TrueNAS®) has plenty of cloud help to offer. Most of the features are Free!

TrueNAS CORE

TrueNAS Core replacing FreeNAS in version 12.0

[ Note: FreeNAS™ will become TrueNAS® Core in the release 12. News was announced 2 months ago ]

FreeNAS™ Cloud Sync

One of the underrated features of FreeNAS™ is Cloud Sync. It was released in version 11.1 and it is invaluable extending the hybrid cloud file sharing to the masses. Cloud Sync makes the shares available to public cloud services such as AWS S3, Dropbox, Google Cloud Storage, Google Drive, Microsoft Blob Storage, Microsoft OneDrive, pCloud, Wasabi™ Cloud and more. This means that the files and folders used within the NAS space in the LAN, can synchronized and used through the public cloud services mentioned.

There are 2 steps to setup Cloud Sync.

  • Add the Cloud Credentials for the cloud provider to use
  • Create the Cloud Sync Task

Continue reading

Falconstor Software Defined Data Preservation for the Next Generation

Falconstor® Software is gaining momentum. Given its arduous climb back to the fore, it is beginning to soar again.

Tape technology and Digital Data Preservation

I mentioned that long term digital data preservation is a segment within the data lifecycle which has merits and prominence. SNIA® has proved that this is a strong growing market segment through its 2007 and 2017 “100 Year Archive” surveys, respectively. 3 critical challenges of this long, long-term digital data preservation is to keep the archives

  • Accessible
  • Undamaged
  • Usable

For the longest time, tape technology has been the king of the hill for digital data preservation. The technology is cheap, mature, and many enterprises has built their long term strategy around it. And the pulse in the tape technology market is still very healthy.

The challenges of tape remain. Every 5 years or so, companies have to consider moving the data on the existing tape technology to the next generation. It is widely known that LTO can read tapes of the previous 2 generations, and write to it a generation before. The tape transcription process of migrating digital data for the sake of data preservation is bad because it affects the structural integrity and quality of the content of the data.

In my times covering the Oil & Gas subsurface data management, I have seen NOCs (national oil companies) with 500,000 tapes of all generations, from 1/2″ to DDS, DAT to SDLT, 3590 to LTO 1-7. And millions are spent to transcribe these tapes every few years and we have folks like Katalyst DM, Troika and more hovering this landscape for their fill.

Continue reading

The Falcon to soar again

One of the historical feats which had me mesmerized for a long time was the 14-year journey China’s imperial treasures took to escape the Japanese invasion in the early 1930s, sandwiched between rebellions and civil wars in China. More than 20,000 pieces of the imperial treasures took a perilous journey to the west and back again. Divided into 3 routes over a decade and four years, not a single piece of treasure was broken or lost. All in the name of preservation.

Today, that 20,000 over pieces live in perpetuity in 2 palaces – Beijing Palace Museum in China and National Palace Museum Taipei in Taiwan

Digital data preservation

Digital data preservation is on another end of the data lifecycle spectrum. More often than not, it is not the part that many pay attention to. In the past 2 decades, digital data has grown so much that it is now paramount to keep the data forever. Mind you, this is not the data hoarding kind but to preserve the knowledge and wisdom which is in the digital content of the data.

[ Note: If you are interested to know more about Data -> Information -> Knowledge -> Wisdom, check out my 2015 article on LinkedIn ]

SNIA (Storage Networking Industry Association) conducted 2 surveys – one in 2007 and another in 2017 – called the 100 Year Archive, and found that the requirement for preserving digital data has grown multiple folds over the 10 years. In the end, the final goal is to ensure that the perpetual digital contents are

  • Accessible
  • Undamaged
  • Usable

All at an affordable cost. Therefore, SNIA has the vision that the digital content must transcend beyond the storage medium, the storage system and the technology that holds it.

The Falcon reemerges

A few weeks ago, I had the privilege to speak with Falconstor® Software‘s David Morris (VP of Global Product Strategy & Marketing) and Mark Delsman (CTO). It was my first engagement with Falconstor® in almost 9 years! I wrote a piece of Falconstor® in my blog in 2011.

Continue reading

Reap at low tide

[ Note: This article was published on Linkedin more than 6 months ago. Here is the original link to the article ]

[ Update (Apr 13 2020): Amid the COVID-19 pandemic and restricted movement globally,  we can turn our pessimism into an opportunistic one ]

Nature has a way of teaching us. What works and what doesn’t are often hidden in plain sight, but we human are mostly too occupied to notice the things that work.

Why are they not spending?

This news appeared in my LinkedIn feed. It read “Malaysian Banks Don’t Spend Enough on Tech“. It irked me immensely because in a soft economy climate (the low tide), our Malaysian financial institutions should be spending more on technology (reaping the opportunity) to get ahead.

Why are the storks and the egrets in my page photo above waiting and wading in the knee-deep waters? Because at low tide, when the waves ebb, food is exposed to them abundantly. They scurry for shrimps, small crabs, cockles, mussels and more. This is nature’s way.

From the report, the technology spending average among the Malaysian banks is pathetic.

No alt text provided for this image

The negative domino effect on SMEs

When the banks are not spending on technology, the other industries, especially the SMEs (small medium enterprises) follow suit. The “penny pinching” and “tightening purse string” effect permeates across industries, slowly and surely putting the negative effect in tech spending into a volatile spin-cycle.

From a macro-economic point of view, spending slows down. Buying less means lesser demands and effectively, lowering supply, and it rolls on. The law of demand and supply just got dumped into an abyss.

A great opportunity for those who see it

When I was an engineer at Sun Microsystems more than 2 decades ago, I read a comment delivered by one of the executives. It said “When times are bad, those who know will get the best parts“. I took his comment to heart because what he said held true, even until today.

This is the best time, when the country is experiencing an economic downturn. When the competitors are holding back and may be reeling from the negative effects of the economy, the banks are in the best position to grab the best deals. This is the time to gain market share, when the competition is holding back for fear that the economy will become softer.

Furthermore, with the low interest rates across the board, there is no better time than the present to step up the tech spending. Banks should know this very well but I am perplexed.

That is why the Malaysian banks must kick start their tech spending campaign now. And the SMEs will follow, overturning the downturn with demands of spending for the best “parts”. The supply “factories” are fired up again, and will lead to a positive growth to the economy.

Bank Negara RMiT is that one opportunity

One thing which has been looming is Bank Negara, Malaysia’s Central Bank, RMiT (Risk Management in Technology) framework. A new version was released in July 2019, and to me as an outsider, is a great opportunity to grab the best parts. And some of these standards will come into effect in January 2020

Bank Negara is strongly encouraging banks to improve the security and the confidence of the country’s financial industry, and the RMiT framework is really a prod to increase tech spending. Unfortunately, in some of my business interactions with a few of the banks, the feet dragging practice is prevalent.

Nature’s lesson

The best time to have your best pick is at low tide. This is nature’s lesson for us. What are we waiting for?

Iconik Content Management Solutions with FreeNAS – Part 2

[ Note: This is still experimental and should not be taken as production materials. I took a couple days over the weekend to “muck” around the new Iconik plug-in in FreeNAS™ to prepare for as a possible future solution. ]

This part is the continuation of Part 1 posted earlier.

iconik has partnered with iXsystems™ almost a year ago. iconik is a cloud-based media content management platform. Its storage repository has many integration with public cloud storage such as Google Cloud, Wasabi® Cloud and more. The on-premises storage integration is made through iconik storage gateway, and it presents itself to FreeNAS™ and TrueNAS® via plugins.

For a limited, you get free access to iconik via this link.

iconik  – The Application setup

[ Note: A lot of the implementation details come from this iXsystems™ documentation by Joe Dutka. This is an updated version for the latest 11.3 U1 release ]

iconik is feature rich and navigating it to setup the storage gateway can be daunting. Fortunately the iXsystems™ documentation was extremely helpful. It is also helpful to consider this as a 2-step approach so that you won’t get overwhelmed of what is happening.

  • Set up the Application section
    • Get Application ID
    • Get Authorization Token
  • Set up the Storage section
    • Get Storage ID

The 3 credentials (Application ID, Authorization Token, Storage ID) are required to set up the iconik Storage Gateway at the FreeNAS™ iconik plug-in setup.

Continue reading

Iconik Content Management Solutions with FreeNAS – Part 1

[ Note: This is still experimental and should not be taken as production materials. I took a couple days over the weekend to “muck” around the new Iconik plug-in in FreeNAS™ to prepare for as a possible future solution. ]

The COVID-19 situation goes on unabated. A couple of my customers asked about working from home and accessing their content files and coincidentally both are animation studios. Meanwhile, there was another opportunity asking about a content management solution that would work with the FreeNAS™ storage system we were proposing. Over the weekend, I searched for a solution that would combine both content management and cloud access that worked with both FreeNAS™ and TrueNAS®, and I was glad to find the iconik and TrueNAS® partnership.

In this blog (and part 2 later), I document the key steps to setup the iconik plug-in with FreeNAS™. I am using FreeNAS™ 11.3U1.

Dataset 777

A ZFS dataset assigned to be the storage repository for the “Storage Target” in iconik. Since iconik has a different IAM (identity access management) than the user/group permissions in FreeNAS, we have make the ZFS dataset to have Read/Write access to all. That is the 777 permission in Unix speak. Note that there is a new ACL manager in version 11.3, and the permissions/access rights screenshot is shown here.

Take note that this part is important. We have to assign @everyone to have Full Control because the credentials at iconik is tied to the permissions we set for @everyone. Missing this part will deny the iconik storage gateway scanner to peruse this folder, and the status will remain “Inactive”.  We will discuss this part more in Part 2.

Continue reading

btrfs butter gone bad?

I wrote about btrfs 8 years ago.

Since then, it has made its way into several small to mid-end storage solutions (more NAS inclined solutions) including Rockstor, Synology, Terramaster, and Asustor. In the Linux world, SUSE® Linux Enterprise Server and OpenSUSE® use btrfs as the default OS file system. I have decided to revisit btrfs filesystem to give some thoughts about its future.

Have you looked under the hood?

The sad part is not many people look under the hood anymore, especially for the market the btrfs storage vendors are targeting. The small medium businesses just want a storage which is cheap. But cheap comes at a risk where the storage reliability and data integrity are often overlooked.

The technical conversation is secondary and thus the lack of queries for strong enterprise features may be leading btrfs to be complacent in its development.

Continue reading

4 Digital Workplace Moves after COVID-19

[ Note: This article was published on LinkedIn on March 24, 2020. Here is the link to the original article ]

We live in unprecedented times. Malaysia has been in Movement Control Order (MCO) Day 7, which is basically a controlled lockdown of movements and activities. In many cases, businesses have grounded to a halt, and the landscape has changed forever. The “office” will not always be a premise anymore, and the “meetings” will not be a physical face-to-face conversation to build relationships and trust.

Trust is vital. A couple of weeks ago, I wrote 關係 (Guan Xi), and having to re-invent Trust in a Digital World.

No alt text provided for this image

The impact on organizations and businesses is deep and powerful and so, as we move forward when the COVID-19 pandemic dies down, organizations’ plans in their Digital Transformation strategy will change as well.

Here are 4 technology areas which I think must take precedence for the Digital Workplace in the Digital Transformation strategy.

Software-Defined Wide Area Network (SD-WAN)

Physically connections have been disrupted. Digital connections are on the rise to supplant “networking” in our physical business world, and the pandemic situation just tipped the scale.

Many small medium businesses (SMBs) rely on home broadband, which may be good enough for some. Medium to large organizations have broadband for business. Larger organizations which have deeper pockets might already have MPLS (multiprotocol label switching) or leased line in place. A large portion might have VPN (virtual private network) set up too.

In time, SD-WAN (software-defined wide area network) services should be considered more profoundly. SD-WAN is a more prudent approach that inculcates digital workplace policies such as quality of service (QOS) for critical data connections, allocating network attributes to different data workloads and network traffic, VPN features and most come with enhanced security addendum as well. .

In addition to performance, security and capacity control, SD-WAN implementation helps shape employees’ digital workplace practices but most importantly, redefine the organization’s processes and conditioning employees’ mindsets in the Digital Transformation journey.

 

Video Meetings & Conferencing

The Video Meetings and Conferencing solutions have become the poster child in the present pandemic situation. Zoom, Webex, Microsoft Teams, Skype (it is going away), GoToMeetings and more are dominating the new norm of work. Work from home (WFH) has a totally new meaning now, especially for employees who have been conditioned to work in an “office”.

I had more than 15 Zoom meetings (the free version) last week when the Malaysian MCO started, and Zoom has become a critical part of my business now, and thus, it is time to consider paid solutions like Zoom or Webex as part of an organization’s Digital Workplace plans. These will create the right digital culture for the new Digital Workplace.

Personally I like Uberconference because of their on-hold song. It is sang by their CEO, Alex Cornell. Check out this SoundCloud recording.

File Sharing

Beneath the hallowed halls of video meetings and conferencing, collaboration happens with data shared in files. We have been with file and folders from our C: drives or NAS Home Directories or File Server’s shared drives that these processes are almost second nature to us.

In the face of this COVID-19 pandemic, files and information sharing has become cumbersome. The shared drive is no longer in our network, because we are not in the organization’s LAN and intranet anymore. We are working at home, away from the gigabit network, protected by the organization’s firewall, and was once slaved … err … I mean supported by our IT admins.

The obvious reaction (since you can’t pass thumb drives anymore at present) is to resort to Dropbox, OneDrive, Google Drive and others, and hoping you won’t max out your free capacity. Or email attachments in emails going back and forth, and hoping the mail server will not reject files larger than 10MB.

The fortunate ones have VPN client on their laptops but the network backhaul traffic to the VPN server at the central VPN server, and overloading it to the max. Pretty soon, network connections are dropped, and the performance of file sharing sucks! Big time!

What if your organization is a bank? Or an Oil & Gas company where data protection and data sovereignty dictate the order of the day? All the very-public enterprise file sync and share (EFSS) like Dropbox or OneDrive or Google Drive totally violate the laws of the land, and your organization may be crippled by the inability to do work. After all, files and folders are like the peanut-butter-jelly or the nasi lemak-teh tarik (coconut rice & pulled tea Malaysian breakfast) combo of work. You can’t live without files and folders.

The thoughts of having a PRIVATE on-premises EFSS solution in your organization’s Digital Transformation strategy should be moved from the KIV (keep in view) tray to a defined project in the Digital Transformation programme.

At Katana Logic, we work with Easishare, and it is worth having a serious plan about building your own private file share and sync solution as part of the Digital Workplace.

Security

In such unprecedented times, where our attention is diverted, cybersecurity threats are at its highest. Financial institutions in Malaysia have already been summoned by Malaysia Bank Negara central bank to build the industry’s expectations and confidence through the RMiT framework. Conversations with some end users and IT suppliers to Malaysian banks and other financial institutions unfortunately, revealed the typical lackadaisical attitude to fortify cyber resiliency practices within these organizations. I would presume the importance of cybersecurity and cyber resiliency practices would take a even further back seat with small medium businesses.

On a pessimistic note, ransomware and DDOS (distributed denial-of-service) have been on the rise, and taking advantage of this pandemic situation. NAS, the network attached storage that serves the organization shared files and folders has become ransomware’s favourite target as I have wrote in my blog.

But it does not have to be expensive affair with cybersecurity. Applying a consistent periodical password change, educating employees about phishing emails, using a simple but free port scanners to look at open TCP/UDP ports can be invaluable for small medium businesses. Subscribing to penetration testing (pentest) services at a regular frequency is immensely helpful as well.

In larger organizations, cyber resiliency is more holistic. Putting in layers for defense in depth, CIA (confidentiality, integrity, availability) triad, AAA (authentication, authorization, audit) pro-active measures are all part of the cybersecurity framework. These holistic practices must effect change in people and the processes of how data and things are shared, used, protected and recovered in the whole scheme of things.

Thus organizations must be vigilant and do their due diligence. We must never bat any eye to fortify cyber security and cyber resiliency in the Digital Workplace.

Parting thoughts

We are at our most vulnerable stage of our lifetime but it is almost the best time to understand what is critical to our business. This pandemic is helping to identify the right priorities for Work.

At any level, regardless, organizations have to use the advantage of this COVID-19 situation to assess how it has impacted business. It must look at what worked and what did not in their digital transformation journey so far, and change the parts that were not effective.

I look at the 4 areas of technology that I felt it could make a difference and I am sure there are many more areas to address. So, use this pessimistic times and turn it into an optimistic one when we are back to normalcy. The Digital Workplace has changed forever, and for the better too.

Continue reading