Here are four data-driven technologies, from a macro to micro level, that will drive change over the next 12 months and beyond. They will enable every business to make greater use of their data, improve business efficiency and serve their end customers better and faster than ever before.
- Applied observability – getting down to the detail of what’s happening across a business
Observability has grown up, breaking out from its infant stages as a tech-focused term to something that organisations realise will be vital to keeping track of key data events in an increasingly decoupled business world, spanning from systems architecture to the business operations it supports.
We have now moved onto the next level: “Applied observability” has been recognised by Gartner as a key 2023 strategic tech trend at its most recent Orlando Symposium.
Applied observability enables organisations to exploit their data artefacts for competitive advantage. That means being able to ensure that the right data is delivered at the right time for rapid action, based on confirmed stakeholder actions rather than intentions. “Observable” data include key digitised artefacts such as logs, traces, API calls, dwell time, downloads, and file transfers, which appear when any stakeholder takes any kind of action.
Applied observability feeds these observable artefacts back in a highly orchestrated and integrated approach to accelerate organisational decision-making, and allows the business owner to track how long an action took to fully process at every point in the workflow. By having this visibility, businesses can gain better visibility into how long their systems took to process a workflow and get real-time insights into bottlenecks or areas for improvement to ensure their systems function optimally.
Event-driven architecture and distributed tracing aid applied observability
But unearthing these insights requires an event-driven approach to software architecture. Distributed tracing, for example, is a method of tracking application requests as they flow from front-end devices to back-end services and databases. By definition — and in contrast to traditional tracing — distributed tracing can be visualised to show a searchable, graphical picture of when, where, and how a single event flowed through an enterprise, regardless of the number of hops the workflow took to fully process.
Consider the potential traceability and end-to-end observability benefits across a payment ecosystem underpinned by event-driven architecture (EDA). Embedding distributed tracing into an event mesh emits trace events in OpenTelemetry format so banks can collect, visualise, and analyse them in any compatible tool. This empowers them to not only confirm that a given message was published, but easily understand exactly when and by whom, where it went, down to individual hops, who received it and when, or why not.
When planned strategically and executed successfully, applied observability is arguably the most powerful source of data-driven decision-making. Since OpenTelemetry is a well-defined open standard, it can be implemented across both synchronous and asynchronous workflows. - Hyperautomation and intelligent automation: taking automation to the next level
Gartner and IDC identify hyperautomation and intelligent automation in their yearly technology trends for 2023. Hyperautomation moves automation up a level, adding more intelligence to automation and using a broader set of tools so that previously un-automatable tasks can be automated. Hyperautomation initiatives come in many different shapes and sizes, and are being seen across a wide range of industries, from banking and insurance to manufacturing and healthcare.
Gartner points to US healthcare company CVS Health as a prime example, having taken advantage of hyperautomation to simplify its unwieldy benefits administration processes to improve efficiency, accuracy, and customer service. A new system was developed to streamline tasks from application receipts and payments to issue resolution. These were cross-functional, largely manual, and time-consuming tasks beforehand, which involved analysing data in a wide range of formats, and aligning with complex coding rules. However, using a combination of AI, RPA, machine learning, data analytics, and natural language processes, the company was able to automate much of this work.
As consumers of goods and services continue to demand faster and better customer service, companies need to overlap development cycles and find ways to cost-reduce portions of what they deliver to stay ahead of the curve. Removing the friction that slows down implementation teams is key to success with end customers.
Putting automation on the runway to success
But hyperautomation requires an organisation to be underpinned by an EDA and — more specifically, an event mesh — to maximise success. An event mesh is an interconnected network of event brokers that allows data to be pushed in real time to parts of the organisation where the data is needed. It does this dynamically, meaning that new event types can be added any time, and interest in events can be registered, allowing seamless interchange of data for the applications that are interested in using it.
Take the aviation industry as an example. An event mesh streams information such as flight routes, delays, cancellations, and mileage accruals between applications, connected devices, and people anywhere in the world, instantly. With an event mesh, information about events can be continuously streamed to multiple systems, filtered so each system receives only the data it needs. This ultimately enhances the customer experience as passengers, pilots, and crews are notified in real time when something relevant occurs in each flight.
Or take the working example of Heathrow Airport: After downsizing its IT department during the pandemic, it needed to reduce its dependency on IT to deliver solutions. The answer was to pivot the department to the role of orchestrator, building a low-code/no-code community of practice that enabled employees across the wider business to build-in their own automation for health and safety apps. These apps support a safe return to work, as well as live audits, and are designed with automation to make the process more efficient. As of last August, Heathrow’s hyperautomation efforts had garnered huge savings in potential outsourcing costs, reduced paperwork by 120,000 pages, and decreased manual data entry hours by more than 1,170. - The acceleration of the event-driven API economy
The API economy has exploded with the proliferation of web applications and the rise of digital businesses that need to expose and connect applications and assets using familiar architectural patterns and protocols such HTTP/Representational State Transfer (REST). Research from MIT Initiative on the Digital Economy has shown that over a four-year period, businesses using APIs saw 12.7% more growth in market capitalisation compared to those that did not adopt APIs.
But the API economy is changing, as event-driven architecture and asynchronous event-driven APIs are becoming increasingly important to companies that are striving to make their business processes, customer interactions and supply chains more real-time. Using both synchronous and asynchronous methods can result in an application environment in which system resources are most effectively used.
Merging synchronous and asynchronous APIs — this is not a binary choice
AsyncAPI and RESTful APIs will be the next evolution of the API economy. API management platforms will need to quickly adapt to this new reality. As an example, Forrester recently spotlighted Roche, a global biotech company that is blazing a trail in the API-driven area. As the case study demonstrates, the organisation understood that EDA is a complement to RESTful APIs — another tool in their digital business toolbox, filling in the gaps of a REST-only approach.
As a result, they planned a unified event + REST approach from the outset. Both APIs and events are managed as enabling digital products, with the governance and lifecycle of both unified as one process. A platform team builds a unified API + event platform for app dev teams rather than building each as a separate siloed platform. The results include faster speed to market and new business opportunities.
The company also embraces FAIR data principles: data that is Findable, Accessible, Interoperable, and Reusable. A digital marketplace consisting of both events and APIs makes the data findable and machine-readable via Open API Spec (OAS) and AsyncAPI metadata. Data can then be accessed on demand via APIs and events push data in real time.
From this example it can be clearly seen that modern enterprises realistically require a one-stop shop platform to best suit their unique application integration strategies — one that can deliver unified management to access, reuse, and expose both asynchronous event-driven and synchronous RESTful APIs. - Blurring the physical and digital words; the metaverse will continue at pace
Zooming out a bit, there is no getting away from the increasing prevalence of the metaverse. It is clear to see its potential to reshape everything from consumer habits and public services to health, welfare, and education from an emerging digital universe where providers, creators, and consumers can experience a parallel life to their real-world existence. McKinsey estimates that the Metaverse — which is being driven by technologies such as digital twins, augmented reality, and virtual reality — will be worth $5 trillion by 2030.
Real-time data underpins the metaverse — must go with the flow, just like real world
Data in real time will be the vital common denominator to link the digital and physical divide and optimise the metaverse. McKinsey lists real-time data as a vital element to facilitate adoption of the metaverse in several key industries. These include:- Retail — to enhance shopping/in-store/product experience, capture efficiencies, and explore net-new revenue streams;
- Banking/Financial services — to support decentralised finance structures;
- Transportation — to allow central coordination and project management (e.g., via IoT/digital twins), especially in logistics, with real-time data collection for optimisation;
- Healthcare — enabling fully personalised health consultations, with access to real-time data.
- Retail — to enhance shopping/in-store/product experience, capture efficiencies, and explore net-new revenue streams;
The metaverse by definition cannot be static. It needs to be in real time, in motion. It needs an EDA and a real-time event mesh to support it.
For example, if an avatar visits a retailer in the metaverse, the retailer will need to keep track of countless user actions such as how long someone spent looking at a particular product, which products got the most interest, etc. The metaverse will produce a significant amount of data that companies will be interested in analysing in real time. This data can be used to make decisions, from real-time adjustments, to inventory, to incentivising commerce through real-time offers/discounts/coupons, and even dynamically pricing products.
Data-driven developments shape 2023
The four trends discussed above collectively offer the potential to reshape the way we do business, interact with customers and partners, and make data-driven decisions. The common thread across these trends is the need for better movement of data, in real time, in motion, and all pointing to the need for an underlying EDA.
To ensure success, organisations must adopt an event mesh underpinned by a proven strategy to designing, managing, and governing events as they flow through their enterprise.
