The pandemic has brought about an acceleration of digitisation as companies and individuals take more of their work and play online. This has changed the way we do business and think about data.
New digital-only businesses have emerged, while existing businesses face increased urgency to take their offerings online. This means that a significant amount, if not all, of a company’s interactions with their customers and prospects will happen via the web or an app.
Digital interactions generate data that can be analysed to power data-driven decisions, but it also creates new challenges that companies will need a modern data stack to solve.
The “old” data stack where tools are connected using a point-to-point approach.
The hub and spoke approach that is gaining traction.
Major data challenges with the traditional data stack
The growing importance of digital products is driving three major challenges that are impacting companies.
- Increased digitisation and the rising importance of understanding digital interactions
Companies have been collecting and analysing data for the longest time. It’s common to look at sales and revenue data, customer demographics, marketing interactions, and more. Even the analysis of digital interactions isn’t new—page visits, bounce rates, session times, and app installs are routinely scrutinised.
Yet understanding digital experiences and driving a digital business is not just about that. Transactional data in CRMs and billing systems, marketing data about email click rates, website visits and campaign responses, can only paint a limited picture around user behaviour.
This exposes the need for more user behaviour data (i.e. how users are using your digital products).
- Data silos remain
Data silos have always existed even without incorporating new data types like user behaviour data. Data is collected in different systems, by different teams. CRM systems would store data in one way, while a billing and accounting system would store data in its own format. There is a constant need to extract data from a source, transform it, and then load it into a central location so various pieces of data can be combined for useful analysis. This process is technical and requires data engineers (and time) to complete.
When this is badly done, consumers of the data and the resulting reports don’t trust the data. Different teams would also be looking at their own data that may not match what other teams are looking at. This leads to the question: Which data is right?
Technology has existed to solve these issues, but it isn’t always easy, fast, or affordable. Increased digitisation leads to growing data volumes and data types so the work needed to break down data silos naturally increases.
- Data democratisation is still top of mind
A reliance on technical skills doesn’t just exist in removing data silos; it’s also a roadblock when it comes to analysis. In many cases, existing analytics tools require a user to be able to write SQL code or have some other technical skills to be able even query the data and answer questions about their business.
In the absence of these skills, users find themselves queueing for data engineers and analysts to help. Even for engineers and analysts with the right technical skills, it takes time to code and build the necessary reports. This is a loss of precious time to market.
What’s more, this process gets repeated every time a new question needs to be answered. This queue can get so bad that even precious data scientist resources may get pulled into these tasks.
This further solidifies the need for data democratisation—the need to make data and insights accessible to everyone, regardless of technical know-how.
Adopting a new data stack to overcome existing hurdles
A modern data stack brings new options to the table and helps companies to address the three challenges more efficiently than before.
Answering questions about user behaviour
With the increased importance of websites and apps, new tools have emerged to help companies better understand user behaviour within digital assets. Product analytics tools are built to track and analyse more than just page views, form fills, and session times.
Such tools are built for answering advanced user behaviour questions like:
- Who are my power users and how do their behaviours differ from other users?
- Why do some users convert, while others don’t?
- What are the top drivers of user engagement and retention?
Best-in-class tools plug into your existing data stack seamlessly, allowing for user behaviour data to be used in other tools for messaging, A/B testing, and more.
Breaking down data silos more effectively
I mentioned earlier that breaking down data silos can be resource-intensive even if there are tools. As with all things tech, innovation is always happening and new tools have emerged in this area, too. Tools in the space of reverse ETL (Extract, Transform, Load) and CDP (customer data platforms) are rapidly gaining traction.
These tools enable improvements in the data process by making it much easier to get data from different sources, clean and transform it, load it into a central data warehouse or data lake, and finally get the data from the data warehouse to various destinations for analysis and action—all with little-to-no coding.
This helps to greatly reduce one of the key causes of data silos.
Removing barriers to data access and analysis
A common trait of tools in the modern data stack is that they support larger data volumes and more diverse data types while making it easier to access and analyse. This is the all-important self-serve aspect of data and analytics.
Self-service analytics tools make it possible to create reports in a few clicks where a similar report in a BI (business intelligence) tool might require 80 lines of SQL code.
This also enables a very important aspect of analytics: the ability to dive deeper into your insights. It is natural that insights lead to additional questions. Without a self-serve analytics tool, users would have to wait for reports to be created for their additional ad hoc questions. This is repeated until they arrive at a final, actionable insight, which might be days or even weeks later. It is a game changer when users can freely ask new questions of their data and get the answers quickly.
There are productivity gains for the teams of data engineers, analysts, and even data scientists who needed to carve out time to help with creating reports. This time can now be used on their core, and more impactful, tasks.
Winning with the new data stack
The new data stack allows for the creation of interoperability between all the tools that are important to a company. It also helps to enable connected and synced data that everyone in the organization can rely on, regardless of the tool they use or what they do.
For digitally focused companies, this means that every team can get insights and take appropriate actions such as:
- Product teams can use the data to inform future product roadmap
- Marketing can use the data to perform sophisticated segmentation and targeting
- Customer success can use the data to determine which accounts/customers are not using the product and are likely to churn so they can take pre-emptive action
As digital interactions become the mainstay of how businesses acquire, activate, and retain customers, there’s no option not to extend the 360-degree view of the customer to include their digital interactions. To be successful, companies need to leverage the tools in the new data stack to help overcome existing data challenges.
