Over the past year we saw many organisations around the world take their businesses online in a short time span to accommodate to the new remote work format and changing user behavior, as a result of the COVID-19 pandemic. Enterprises in APAC were no exception to these changes as they too had to pivot many of their operations quickly. Many business leaders have come to realise the importance of having robust storage infrastructure – a reliable, resilient and performant partner to protect their most valuable business asset – data.
However, according to the Seagate Rethink Data report, enterprises need help with data management because they are unable to successfully control the growing complexities. Among the challenges business leaders face includes managing the storage and ensuring security of data. These insights point toward a pressing issue that makes us ask if business leaders are equipped to protect their data with the best storage infrastructure for their needs.
Whether organisation leaders are aware of it, storage will continue to be the backbone of businesses both large and small. Here are a few trends on storage that we expect to see in 2021 and beyond:
- Increasing significance of security at every step – whether data is at rest or in flight
A continued growth in the trend of hyperscale software ecosystems is underway, allowing for applications to be developed and deployed on smaller “atomic units” for business and locations that may not have the connectivity infrastructure required. More and more cloud native applications run in points of presence or colocation facilities around the world. With this asset partnership model becoming increasingly more common, it is necessary to protect data at each step of the process. In flight and at rest are critical spheres of protecting user data in a more distributed deployment model.
Data-at-rest encryption is becoming increasingly mandatory in many industries as a way to avoid both external and insider threats. Although it might not be mandatory in your particular industry today, it might well become so tomorrow. Therefore, we recommend moving to using encrypted disks as soon as possible to ensure there are no disruptions in the future if the requirement sneaks up on you.
2. Object storage is augmenting and displacing file storage
With the explosion of useful data, object store is becoming the standard for mass capacity and offers advantages over traditional file stores including prescriptive metadata, scalability, and no hierarchical data structure. Systems benefit from greater intelligence incorporated in data sets and object stores provide this intelligence. Storage types include block, file, and object. Block is critical for so many mission critical applications that are performance sensitive. File has serviced legacy applications and provided robust architecture for years. Object storage is focused on new application development in combination with block storage to provide scale and performance in a symbiotic fashion. Many legacy file applications are also migrating to object storage infrastructure to take advantage of the economies of scale that object storage enables.
Object storage is quickly becoming the de-facto standard for capacity storage quickly augmenting and displacing file storage due to improved economic efficiencies and scalability. Additionally, new programmers graduating today increasingly build workflows assuming object storage interfaces. My advice? Hire those people. If you haven’t yet added object storage to your data center, now is the time to do so.
- A greater adoption of composability
While the idea to separate systems into independent units that can be combined with other independent units is not new, a broader, open source-reliant adoption of composability is underway. Kubernetes—the open-source system for automating deployment, scaling, and management of containerized application—is at the core of this trend. Open source is the future of application development because it enables a much larger community to work on the problems that challenge many industries and it also allows for domain specific solutions that are leveraged from the open architectures. Composing the hardware to optimally meet the software or business needs is a natural migration.
Today’s data centers are moving toward composability because it provides easier deployment and redeployment of resources without requiring a priori configurations and statically configured ratios between compute, memory, and storage. Containers and Kubernetes are the core mechanisms of composability and it behooves all data centers that business leaders need to start embracing these technologies if they haven’t already.
- Tiering in mass storage deployment to locate hot data on flash and all else on disk
NVIDIA designs for GPUs divide the memory into different levels (registers, shared, and global). Each has different properties. Registers, which are low latency, have low memory. Global, which has high latency, offers large memory. NVIDIA provides a software interface that allows one to take advantage of the tiered memory and program optimized solutions to that architecture. Similarly, SSDs and HDDs can be viewed at different tiers. There is simply too much valuable data being created to consider a homogeneous storage strategy efficient.
Why does all this matter? A storage system that consists of all high-performance storage devices is likely more expensive than it needs to be. And a storage system that consists of all-mass capacity devices is likely not as performant as it needs to be. That’s how we arrived at the current tiering trend: it’s the way to strike the most efficient balance between cost and performance needs. With the advent of additional technologies (such as storage class memories), the need for architectures that can extract the most value from all classes of storage is paramount.
In a world with infinite budget, data centers would be composed of merely very expensive storage media such as Intel’s 3DXPoint. Unfortunately, economic realities preclude this and dictate hierarchical tiering—in which hot data resides on high-cost, high-performance media and less frequently accessed data resides on affordable, mass capacity media. Fortunately, data center software has become increasingly adapt at identifying hot and cold data and migrating it accordingly. If your data center is not yet taking advantage of heterogenous media for this purpose, you are either losing performance or paying more than is necessary for your storage.
- Formative AI increases the usefulness of data
Not only is data creation exploding, but the amount of that data that is useful is growing as well. Even archived data is being resurrected because advances related to AI/ML are allowing users to mine additional information from once archived data. Enterprise leaders must be prepared to store more data than ever to train models, for mining of critical information, and for archiving data given that its useful life is likely extending.
Formative AI is a means through which data becomes more insightful. Gartner defines formative AI as “a type of AI capable of dynamically changing to respond to a situation.” IDC sees formative AI as “an umbrella term given to a range of emerging AI and related technologies that can dynamically change in response to situational variances.” Formative AI relates to the tiering trend since it depends on having a flexible architecture that can react to changes intelligently. You might be monitoring an AI model and get a signal that it is drifting. You could then use another model to search for the appropriate training data on your disk tier, and automatically move it to the flash tier to make the training go faster. The disk tier would also likely be an object store, so that ties in the object storage trend as well. The advantages are speed (since the data is moved to your fast tier automatically) and cost (since you can store on inexpensive disk in an easy to access format until you need the data).
Recent innovations in machine learning have finally unlocked the long-promised potential of artificial intelligence. Now these machine learning techniques clamor for ever larger data sets from which to extract ever more accurate insights. Because future insights and advancements in machine learning are challenging to predict, businesses today should start saving as much of their data as they possibly can to ensure that any and all future analyses can be done with the best possible training data.
While these trends will equip users with adequate knowledge of storage to take on the new year, leaders need to assess their current storage strategy and investments, and how well they are managing and protecting important business information. Whether you’re leading an organisation, an IT department or even a small team within a company, it’s imperative to ensure your business has the right storage hardware and software facilities in place to continue delivering positive business outcomes.
