Big data analytics has become one of the most essential tools for enterprises to gather actionable insights they can use to streamline operations and better understand and serve customers. And one of the most essential tools for getting the most insight out of that data is the cloud, which can provide the extra storage and compute resources (as well as the scalability) necessary to help process big data faster.
This was the theme of a recent keynote panel at the Cloud Expo Asia conference in Hong Kong, but in addition to exploring the importance of big data and cloud technologies in driving actionable insights, the panel provided some fascinating insights into the different approaches to those technologies between digital-native companies and traditional enterprises in the fledgling stage of digital transformation.
For example, Torquill Pagdin, Director of Data Engineering Technology at Expedia-owned Hotels.com, explained that until recently, his company had been running its data processes on-premises using Hadoop, MapReduce and Java, but is now moving to the cloud. However, he said, moving to the cloud isn’t a simple matter of dumping apps and data into Amazon Web Services (AWS) – it also involves rearchitecting those apps to take full advantage of the cloud’s capabilities.
“We’re moving everything over to Spark, we’re trying to use Scala more, because that fits perfectly with Spark,” he said. “And we’re also trying to utilize the technologies to re-architect a particular application. So if we can lift up the hood, and have a proper look at each of our applications and work out where we’re going wrong, we can optimize them.”
Pagdin added that Hotels.com is also using as much open-source code as possible for its cloud migration to avoid vendor lock-in.
“Some parts of the Hotels.com business and Expedia use the Google Cloud Platform (GCP), mainly around the Google Analytics side of things. We want to be able to move to a multi cloud strategy so we’re not always beholden to everything within AWS,” Padgin explained. “For example, we’ll use some Lambda functions for some things, but we also want to be able to write our code so that if we wanted to lift that application and put it into Google, that we would actually be able to do that with limited code changes.”
He added that an open-source strategy also comes in handy for negotiating contracts with AWS. “If you mention that you’re using Google as well, that can help.”
Starting from scratch
That’s the experience of a company that is already designed for the digital world. For a company that has only just started its digital transformation journey, it’s a rather different story.
Akina Ho, Head of Digital Transformation and Innovation at real estate company Great Eagle, explained that while she eventually intends to adopt the cloud, her bigger priority has been to not only create a digital data pool, but explain to the board why they should be using big data analytics in the first place.
“We are just starting our digital journey, so the main thing we need to solve is data source, because around 90% of our data is offline, meaning that it’s on Excel, emails and paper,” she said.
“So the first thing I needed to do is start collecting data. But at the same time, I wanted have small wins to show the executives how important and how useful big data is.”
The answer to the latter was Tableau, which allows C-levels in Great Eagle’s hotel business to visualize things like revenue, guest satisfaction scores, global HR turnover and financial data, all on one dashboard – as opposed to the current method of looking at Excel sheets and analyzing the data using nothing but your brain and industry expertise.
This is not only inefficient and limited in terms of gaining unexpected insights, it’s also excruciatingly slow, Ho said. This is because even when problems are spotted, the data is already at least three to six months old – or possibly older, in cases where managers were informed of problems but either failed to fix them or simply ignored the data.
“That means we’re actually running the business in hindsight,” Ho said. “If you get your data live, you have insight. If you don’t, then you’re not running your business with foresight. That’s where I want to get my company to right now. But we’re at the bottom.”
Using Tableau is a quick and dirty way to show executives the value of big data insights, as well as the value of sharing data across departmental silos instead of treating it as secret information, Ho added.
“In the beginning, our financial department CFO refused to share data with us. Now that they see how useful Tableau is, they want us to help them automate 100+ reports that they do on a monthly basis across the group for them.”
Another interesting contrast between Hotels.com and Great Eagle is that where the former employs between 50 to 60 developers, the latter employs none.
In fact, says Ho, her “team” who put together the Tableau dashboard comprised one person and two part-time interns who wrote the basic scripts – and it took them three months to manually input all that offline data.
“Right now, the whole database that we’re putting our data into is on one laptop, and we’re portraying the dashboard for the executive from there,” she said.
Meanwhile, she says her cloud migration strategy is one that ideally involves hiring no developers at all, mainly because she doesn’t have the budget for it. “We’re a real estate developer – we’re not a software company. It doesn’t make sense for us to go build our own system.”
Data into insights
Both company reps on the panel also provided some examples of how they’re successfully converting big data into actionable insights.
For Hotels.com, a key example is multi touch attribution, which enables the firm to know which marketing channels their bookings are coming from – mobile, email, Google search, direct search, or any combination therein – and thus how to allocate their marketing spend.
“When you’re booking a holiday, most people will use multiple devices – maybe they’ll do some searching on mobile, then go home and do some searching on an iPad, but then probably the booking might take place at home on a desktop,” Padgin said. “We use various methods to stitch those journeys together, utilizing the power of the cloud, which gives us the extra compute power that we need.”
The cloud itself also provides valuable insights into how much Hotels.com spends on app optimization, Padgin added. For example, leaving apps to run overnight can be expensive in terms of compute power.
“Now we get individual reports that say, ‘that application costs $6,000 to run overnight’. And when you say it like that, you can say to the engineering team, if you can get that down to three hours, you’re going to save us, $5,000 or $6,000 a day of direct costs,” he said.
For Great Eagle, even though the company is only getting started on big data, the company is already finding ways to leverage the data it has for apps such as tracking ventilation in car parks.
“We’re teaching our engineering team to understand how data can help them reduce energy and reduce tariffs taxes,” Great Eagle’s Akina Ho said. “They can increase ventilation during peak hours and shut it down during non-peak hours because no one is there, which saves energy usage. You can also use the data to implement air quality control, managing carbon dioxide in the building.”
Ho said her next project is to find ways to use big data analytics for predictive maintenance and automating the engineering process “so that our engineers can understand the importance of big data, see the insights and how they can save money, save manpower, and save also risk in terms of defects and things like that, so that they’re willing to make bigger steps.”
Education matters
Indeed, for Great Eagle, education remains the key component of its big data strategy, as well as its overall digital transformation strategy, Ho says.
“I work a lot with the business units, and they are used to having autonomy over their own data, so my takeaway is — find a way to show them that sharing data doesn’t mean that you lose control over your business.”
“It actually helps you empower your users or your executives to develop the business even further,” she said.
Interestingly, Padgin of Hotels.com agreed that education is important even for digital native companies, particularly when it comes to cloud migration.
“My developers were very Hadoop/HDFS based – they needed time to learn how to move things over to the cloud,” he said. “It’s not an enormous change for them, but it is a mindset change – they needed to learn more DevOps-type things than they’re used to doing, launching clusters and things that they would not have been used to doing before with an on-prem solution.”
He added that while it can be a steep learning curve, “once you there, you can move everything over very quickly.”
