June 16, 2020

Data Immutability in Cloud Storage

Written by Tyler Moore

The Age of Big Data is here, and converting data into economic value is fueling the emerging data economy. Cloud service providers must invest time and resources in protecting data in this new data-driven world, or the Data Economy will not be able to move forward.

Data is the currency in our digital society, and the loss of it can cause panic or serious consequences. We have all read articles about businesses, movie studios, hospitals and even police departments losing data due to human error or ransomware attacks. A major US city’s police department lost 25% of its body cam videos from accidental file deletion during a software upgrade. Several high profile hospitals and medical groups have paid large sums of ransom money to retrieve patient data. 

Such incidents are frightening, and all too common, but easily preventable with the right cloud storage and the use of  “immutable buckets.”

“Immutability” means that data, once written, cannot be deleted or altered for either a pre-determined length of time or perhaps even forever. Immutability is the protection of content from accidental or malicious destruction or alteration which is different than data durability or data backup. I’ll talk about durability in a moment.

The notion of immutability has existed since the days when paper documents began to be locked in vaults. Today IT departments make backup tapes and store them in a secure warehouse or vault. Certainly the practice of taking media off-line and storing it in the proverbial impenetrable bunker is feasible to protect it from accidental erasure or overwrite, it also makes the data difficult to retrieve to the point of making it almost useless.

Developments in encryption and security technology have made it possible to create immutable storage from the drives in cloud storage systems. When you create a storage bucket, you can flip a switch and make that bucket immutable, meaning any data stored in that bucket cannot be erased or modified for some pre-determined length of time.

Immutability Versus durability

Most people in the IT world worry about equipment failure as the likely cause of data loss. This used to be a real problem, but not so much anymore. Loss from equipment failure is called “durability,” and it is usually expressed in “nines”, for example, 99.999 percent, or “five nines.” At RStor, we provide 11 nines of durability. In other words, if you stored a million one-megabyte files with RStor, statistically you might lose one file every 659,000 years. There is so much redundancy built into modern storage systems that actually losing a file almost never happens. Obsessing over the durability of a storage system is not necessary.

If durability is so good why should you worry about your data? Well, here’s why, even the best run businesses lose data every day mostly due to:  

  • human error
  • malware and ransomware
  • application software bugs that accidentally overwrite or delete data
  • employee or insider sabotage

However, if you store data in immutable buckets, all of those risks disappear. Attempts to delete or modify data stored in an immutable bucket will result in an error message.

Is there a downside of immutability?  Well, if you can’t erase data, it means you’re paying for the storage even if you no longer need the data. Before you put data into an immutable bucket, you need to understand the value of doing so. As the cost of immutable data storage drops, the cost/benefit equation shifts dramatically. Even if you can’t clean up data that is stored in immutable buckets, if the storage cost is super cheap, then who cares? Visit our calculator to determine your costs. 

Some data sets are so valuable that immutability is a no-brainer and some data must be placed in immutable storage to comply with industry regulations such as those in the finance industry. Other examples might be a Hollywood blockbuster film that cost hundreds on millions of dollars to produce, scientific and medical research data, images from expensive (billion dollar +) telescopes (Hubble or the LSST), as we’ll as legal, financial, healthcare, and insurance records. In today’s world a Hollywood film equates to a petabyte of data. In all of these cases you want the option of storing your data immutably so that some overworked person couldn’t mistakenly hit the wrong key and destroy it,.

For many organizations, making electronic data immutable has been something of an afterthought. With the massive growth in data, and the increasing risk of ransomware attacks looming, immutable storage must be a standard feature in cloud storage. With low cost storage, the case for immutability becomes even more compelling.

You may also like…