Building a real-time enterprise: hidden pitfalls & buried treasures

The real-time enterprise is hailed as the next frontier in digital transformation roadmaps, and running a business at the speed of CPUs should theoretically be possible. Yet there are many hidden inefficiencies in digital workflows, from systems and clouds to platforms and apps.

In a virtual roundtable hosted by Solace and organized by Jicara Media, senior IT professionals identified some of these inefficiencies and discussed ways to solve them. Topics covered included data transfer strategies, the problem of latency, and how the cloud is a key component to the real time enterprise.

COVID-19’s impact on the real time enterprise

The discussion began with participants emphasizing that the real time enterprise cannot become a reality without some crucial preliminary steps in digital transformation. These steps might include moving to the cloud, or unifying disparate platforms under a single Enterprise Resource Planning (ERP) system.

Although most enterprises today have drawn up digital transformation strategies, some of these plans have been stalled by COVID-19.

A Vice President for Information Technology at a prominent Singapore-based engineering firm described how the Group’s migration to SAP HANA had been planned for last year, but has been put on hold.

She said, “The group had planned to move to SAP HANA. In the roadmap, it was planned for last year, but because of COVID-19, it’s a very big investment so we suspended it temporarily. We’ll probably wait for 2- 3 years.”

On the other hand, a prominent real estate company in Asia-Pacific, which started implementing digital transformation pre-COVID-19, has faced fewer obstacles.

Said the Chief Information Officer for Asia, “Our system has been quite resilient during the COVID period. I think it’s partly because, before the start of COVID, we had already planned to change our VPN system, and we managed to port over in time before everyone was impacted by COVID-19. So in terms of VPN, it was not constrained. Way before COVID-19, we also changed to Microsoft Teams for video conferencing, etc. So in general, work from home has been quite seamless. And in conjunction with moving to the cloud, we are taking steps to move to a place where everyone can access it in a faster manner.”

COVID-19 has also impacted businesses by causing severe network congestions, which enterprises are not always in control of.

Cynthia Khoo, Head, IT Quality & Service Management and Head, Central Procurement Office at OCBC, explained, “We found ourselves to be pretty resilient during COVID-19 and managed to port over seamlessly to online platforms. This is a testament to our readiness. However, one area outside our control is the network space controlled by the telcos. In Malaysia, we have many players in that space. During COVID-19, not only enterprises but even consumers did face some notable slowness in their internet connections. As long as the network is not strong, the achievement of milestones can be slowed down somewhat. It’s a bit hard for us to control that bit.”

Slow networks and latency issues

Enterprises are not always able to solve issues with critical infrastructure, or even suboptimal network infrastructure. However, a good understanding of architecture can help develop workaround solutions that might mitigate latency issues. One such way is by using technologies like an event mesh, which provides a new approach to handling network events.

Explained Sumeet Puri, Chief Technology Solutions Officer at Solace, “There’s something I call the ‘waiter pattern’. When you’re ordering food in a restaurant, the waiter takes your order and goes to the kitchen. While the food is being prepared, the waiter has moved to the next table. So the kitchen is working in parallel. Everyone is working in an asynchronous manner. You build up applications this way. For example, think of payment or ecommerce applications. When I send a payment, it just has to do a quick validation, and send a response immediately, saying, ‘I have got your payment, I will process it’.

“Further steps of the transaction can be done after the fact. Even if it is one second or two seconds late, it does not matter. Because you’ve been responded to quickly, locally. Things like AI/ML integration, data lake integration, and all such slow consumers should be sitting down behind guaranteed messaging. An event mesh provides that capability. Basically, you work around latency.”

A VP and Data Architect from one of Singapore’s largest banks added that key to understanding the network is to ensure that observability is built into apps at the design stage.

He said, ”Thought has already been put into an application while building it to help achieve observability”.

Cloud Strategies

Once tools such as cloud platforms and a robust network are put into place, enterprises can deal with building a streaming ecosystem that will facilitate a real-time enterprise. This idea of streaming data, as opposed to uploading it in batches, is a dramatic change to how traditional databases are maintained.

The engineering firm VP explained: “In the past, things were done by batch. If a user wanted to do something, but because it was expensive or it was licensed, they sometimes tried to save cost. This was done by performing the action in batches and sending it through SMTP, and doing things outside of the SAP. That caused a lot of disruptions and disconnect of the data and the data integration, and there were governance issues.”

Although cloud is key to the real-time enterprises, most organizations are unable to become fully-cloud based, often due to data sensitivity and regulatory stipulations. As such, the future of data, for most enterprises, lies in the hybrid cloud or multi-cloud.

She said, “We’re still on-premise. There’s a plan for a hybrid cloud ir a multi cloud strategy. But we do a lot of defense-related projects. So that’s why the core applications, especially the ERP, have to be on-premise.”

Participants from the financial services industry agreed.

The bank’s Data Architect explained, “We have partnerships with Amazon and there are cases where we have partnerships with Google as well, especially when it comes to AI/ML. Currently, a lot of the data that we’re trying to push into cloud is around AI or analytics – not continuous streaming or sensitive data, but highly tokenized and non-sensitive data. So still, not fully going into the cloud.”

Khoo added,The entire bank is mostly on premise, because of customer information. In Malaysia we have to get approval from the Central Bank because we have to present our strategy in the event of cyberattacks, etc, before we go to the cloud.”

Challenges with Integration

Most large enterprises have legacy systems, and this is a massive hindrance in efforts to become more agile, more real-time, and to have an integrated IT system across each enterprise.

The engineering firm’s VP said: “We have hundreds of interfaces to third party applications. We’d love to have a common interface, etc, but practically speaking, some applications were built years ago. Even if you were to ask them to change, they wouldn’t be able to.”

For Tony Lee, VP, Group Technology (Group Systems) at CapitaLand Group, the challenge is compounded by of the multitude of companies and programmes the Group has, across the globe.

He described it thus: “One of the areas we’re trying to get more agile in, to almost real-time, is in the area of decision-making, planning and budgeting. We’re attempting to put all business entities within the CapitaLand Group onto one single planning and budgeting platform, which enabling key decision makers within the company to make agile decisions while being able to clearly identify the impact to those decisions across the various geographies and business. It’s complicated by the need to balance out the issues of cost to enable this agility in the business. The goal is near real-time decision-making. One step before it goes AI.”

The future of data in IoT

Being real-time will become crucial as technology penetrates more deeply into all aspects of life, especially with the rise of IoT.

Illustrating the importance of IoT in the era to come, Lee concluded, “We’ve gotten to a position where a lot of the operating elements within our properties are now IoT-enabled. We’ve just completed, for example, our smart carpark systems across our buildings. And essentially what happens now is that in real-time, data is fed through our smart parking systems that will tell us parking availability, parking lot usage, etc. But there are many, many other applications too. We collect data from other sources, for example power usage, water, maintenance faults, alarm triggers etc. Our data scientists are building our next generations data platform that will provide the most optimum sets of information that can help properties perform much better.”