Lately, with a few lessons under our belt, and copious streams of data at our disposal, CIOs –  and those of us at the front line of hardware and software development – have begun thinking about the management of data differently. We ask, “How can we operationalise data? How do you seamlessly derive appropriate and meaningful insights from data? How can you apply design thinking to data operations?”

When you think about what design thinking stands for, one could describe it as an iterative process in which we seek to understand the user, challenge assumptions, and redefine problems in an attempt to identify alternative strategies and solutions that might not be instantly apparent with our initial level of understanding.

Dimensions of design thinking

The process of design thinking is to empathise, design, ideate, prototype, and test. Hitachi Vantara has identified seven dimensions to consider: collaboration, automation and a metadata mindset, data as a shared asset, end-to-end design thinking, enlightened and guided empowerment, silo paranoia, and push down decision-making.

To empathise, we consider that data engineers may need new skill sets to operate new technology. That data stewards could have an inability to access sensitive data. That the compute power required to process huge swathes of data, be it on-premises or on the cloud, may not exist or could be lacking to generate desired outcomes.

Then comes the definition of goals and outcomes, because requirements for DataOps include use cases for analytics, customer 360, Personal Data Protection Act compliance, and predictive maintenance.

Democratise data with DataOps

We believe that DataOps can solve many of the problems companies now face to unlock their data and expand its use across every aspect of the organisation. It will evolve to address data governance, security, and access control in an automated, scalable fashion.

Yet the real power of data lies in our ability to empower end users to put data to use. By empowering them, I mean they need to be able to collaborate to access applicable data and put it together on their own, with their own data, yet be free to ask relevant specialists for help. We have to view data as a shared asset, not control it. For DataOps, data sharing must be the baseline assumption.

To really work, DataOps needs the kind of operational simplicity and rapid iteration that only automation can bring, configured and governed by policies or other metadata.

Many of the principles of design thinking are already at play in DataOps. DataOps expanded the scope of the problem by not seeing it as a Data problem or an Ops problem, but a DataOps problem, which factors the impact on end users.

DataOps can break down the silos that organically emerge in an organisation, democratising access to data. Its influence will be most evident in those who haven’t had access to data, those unaccustomed to data informing their decisions. Like those on the frontlines of a tax department.

Overcoming a data challenge

To deliver self-service and personalised tax payments, Indonesia’s Direktorat Jenderal Pajak (DJP) – or the Directorate General of Taxation – needed to overcome some massive challenges. Indonesians are known only by their first name and do not have a surname. Addresses are not standardised and can be abbreviated, e.g. “Jln” is used instead of “Jalan”. And there is a matter of dialect. According to your region, although it means the same thing, “island” could be written as Pulu instead of Pulau.

To overcome these challenges, thousands of pairs were developed to process, compare, and verify taxpayer information using data from its varied identification sources. To develop data accuracy, higher matching, and linkage find rates, the solution factored the challenges of the user and did not seek to interrupt or change how the data is configured at its source. The achievement is faster delivery of accurate data, and a reduction in overall processing time from weeks to hours, or a few days at most.

To summarise, DataOps is evolving rapidly to address design thinking requirements. This is indeed the future of data management as it is an effective way to navigate the challenges that organisations face while trying to unlock the power of data across all levels. DataOps solutions that help data to move easily and seamlessly without coding, create reports and dashboards, and leverage artificial intelligence, can successfully align business and technical priorities.

To transform the approach to data management, organisations would need a concerted effort to implement a cohesive DataOps culture and design thinking philosophies in order to manage the data of today, and build the future of tomorrow.

We are building answers to the questions we started with.