Tech-driven transformation has been underway for many years, but the COVID-19 pandemic has accelerated this change, while helping to clear the air of resistance in those still clinging on to old ways. We are in a world where becoming digital is no longer just about gaining a competitive edge or cost efficiencies, it is table stakes. Companies who digitally transformed by conducting business online have successfully ridden the storm, taking hold of the opportunities COVID-19 has created. They embraced new technologies, such as IOT devices, as well as leveraged the increased amount of data collected to scale, enhance, and even venture into new service offerings. But the ever-increasing volumes of data is a treasure trove of opportunities waiting to be discovered.
According to Gartner, by 2022, more than half of major new business systems will incorporate “continuous intelligence” that uses real-time context data to improve decision-making. With instantaneous data processing a prerequisite to delivering real-time actionable insights, the speed of data presents a new frontier for competitive differentiation. Zoomlion, a Chinese manufacturer of construction machinery and sanitation equipment, is one company that has reaped the benefits of an end-to-end streaming data platform. The platform allows it to ingest, store and process data from its connected machines, internal core business systems, and third-party sources. Data insights from the platform have also allowed the company to offer new services, which led to a 30 percent increase in value-added service revenue.
No time to lose
The default way to organize data for analytics is to do it in batches. The collected messages get saved into a data “lake” and an analytics system would then mine the database minutes or hours later to generate insights. This creates a time delay in obtaining important insights, presenting inefficiencies in use cases such as autonomous vehicles and health monitoring applications. Moreover, data collected and stored in silos, compounded by unstructured data governance, limits access and availability across departments, resulting in a lack of transparency and efficiency. It hinders business decision-making, which needs to be made with accurate insights that come from a clear, complete picture of the organization’s data.
This is where a data streaming (or data-in-motion) platform comes in useful. Specifically, one that enables it to collect and correlate data from different sources – both within (Enterprise Resource Planning and asset management) and external (edge devices) to the business – for real-time decision-making, such as predictive maintenance. Platforms like Cloudera DataFlow will allow the organization to capture data in real-time, understand what these insights mean to the company and how to respond to it, transforming business information from what-has-happened to what-is-happening-now.
And the good news is, costs of real-time data messaging and streaming capabilities have decreased significantly over the past few years, paving the way for mainstream use. These technologies have also enabled a host of new business applications. For example, analytics company Shoppermotion applied big data processing and machine learning analytics to create an IoT solution that helps retailers understand in-store consumer behaviour. Leveraging Cloudera Enterprise Data Hub, the solution enables retailers to measure the loss of traffic in each aisle and predict when there will be a peak of shoppers at the checkout line.
Keeping the data flowing
However, to harness the benefits of real-time business intelligence, organizations will need a well-defined enterprise data strategy.
For digital businesses, their speed and efficiency rely on insights driven by data. But unfortunately, data management is often a stumbling block as many businesses continue to struggle with how to best capture and analyse their data. This is largely due to the fact that legacy data platforms are inadequate in addressing today’s real-time insight needs. 82% of ASEAN organizations are not processing data in motion, citing complexity of data as one of the top three obstacles to implementing real-time analytics. Most legacy data platforms – be it on-premises or data warehouses that have been moved wholesale to the cloud – lack scalability and flexibility, often resulting in expensive IT overheads. As the business moves toward becoming data-driven, the costs will continue to swell. Moreover, the data platform architectures of the past are not designed to understand the complexities of today’s real-time data, which are driven by new connected data sources, and their sophisticated integration across business processes.
Organizations need to reduce time-to-insights and an enterprise data cloud (EDC) encompassing a streaming platform can help with that. An EDC is a modern data architecture that supports the use of data and analytics to unlock the power and value of an organization’s data to drive business outcomes. It has four main characteristics: multi-function, hybrid and multi-cloud, secure and governed, and open. An enterprise data cloud leverages analytics at every stage of the data lifecycle to help organizations extract the true value from data collected from any source and convert them into actionable insights instantly. Furthermore, the EDC’s features facilitate edge-to-cloud data management. For example, leading European telecommunications provider Deutsche Telekom deployed large-scale, high-speed data processing and interactive querying within the Cloudera Data Platform to detect fraudulent activities in real-time and improve network quality. By applying machine learning and artificial intelligence, Deutsche Telekom identifies network problems before customers notice them and detects fraud patterns and real-time threats before the business is affected.
Business at the speed of data
Data and analytics are becoming more embedded in the day-to-day operations at most organizations – which are fast turning into data-centric enterprises. Given the speed at which data technologies are evolving, yesterday’s data architecture can no longer meet today’s need for speed, flexibility, and innovation.
Effective data management with a focus on streaming analytics is key to a business strategy that will enable organizations to thrive in the new normal. This empowers organizations to navigate sharp turns as they gain critical speed to surpass the competition, much like in the world of Formula 1 racing.
