Unity: Bridging gaming technology with digital twins

Enhancing robotics control with digital overlays, a glimpse into the future of industrial automation. Image courtesy of Unity Technologies.

While today’s video games continue to push the technological envelope, its principles and tools are increasingly being repurposed to tackle complex challenges in sectors such as manufacturing, automotive, and infrastructure.

Unity Technologies, known for its game development platform, is playing a role in this shift. Using its knowledge in gaming, Unity is expanding its horizons to various non-gaming domains, particularly with its digital twin technology and AI-driven solutions.

In an exclusive interview with Frontier Enterprise, Marc Whitten, Senior Vice President and General Manager at Unity Technologies, delves into the crossover of gaming technologies into the enterprise sector. He talks about the transformative impact of these technologies, reflecting on his experiences and insights from his tenure at Xbox to Unity’s current strategic endeavours in the enterprise technology landscape.

In what groundbreaking ways are enterprises incorporating Unity’s real-time 3D technology beyond conventional gaming applications?

There are a lot of companies across industries including retail, automotive, manufacturing, infrastructure, and government using real-time 3D to accelerate their digital transformations.

Marc Whitten, Senior Vice President and General Manager, Unity Technologies. Image courtesy of Unity Technologies.

In the automotive industry, digital twins are transforming the production and driving experience. This ranges from Hyundai’s groundbreaking meta factory (a digital twin of an actual factory) in Singapore, which automates an entire production line of car creation, to Mercedes building the UI for its in-car cockpit with everything from 3D navigation software to mixed-reality experiences using the car’s smart glass.

We’ve also seen innovation in urban planning where the Japanese Ministry of Land, Infrastructure, Transportation, and Tourism (MLIT) worked with Unity on Project PLATEAU to provide 3D city model data of the entire country of Japan. Specifically, efforts are underway to develop, utilise, and convert digital twin data of urban spaces into open data, with new services and innovations. To further this initiative, MLIT will work with the developer community through various approaches like app contests, lightning talks, hackathons, hands-on training, pitch events, and acceleration programs, aiming to bring every citizen into the fold.

How has Unity’s gaming pedigree influenced its approach to solving unique challenges in the enterprise sector?

We believe the world is a better place with more creators in it. Creators — including game developers, artists, architects, automotive designers, and filmmakers — use Unity to bring their ideas to life. While gaming is at our core, we aim to address complex problems for creators across various industries. We provide the necessary tools to create real-time experiences, from games to digital twins.

Reflecting on your tenure in Xbox, what lessons do you bring to Unity’s enterprise technology efforts?

Simplicity matters a lot. In Xbox, we always talked about “console simple” born of the earliest consoles – like Atari and NES: put a cartridge in, turn the power on, and play. We would think about how we can remove complexity out of the system to get people what they needed. For instance, how do you make Xbox’s online functionality “console simple”? How do you add wireless without adding complexity?

At Unity, I think about this idea a lot. Our solutions are very powerful. How do we make it easy for our customers to take these tools and just solve their problems? How can we remove more friction that stands in their way?

How does Unity’s approach to digital twin solutions in non-gaming industries differ from that of other game engines?

Working with digital twin solutions in non-gaming industries is not that much different from what we do in games. The core technology, real-time 3D, has been a foundation in gaming for years.

Across industries, exciting applications are emerging, powered by real-time 3D. These include digital twins, 3D configurators, and AR/VR experiences. However, the use of optimised 3D models, which is essential for supporting these interactive experiences, makes it challenging to unlock due to CAD’s density, complexity, and format.

Augmented reality streamlines machine maintenance, overlaying real-time data and guidance directly onto the equipment. Image courtesy of Unity Technologies.

Demand for this technology is increasing as industrial companies face no shortage of challenges in driving profitable growth: Designs are getting more complex, production timelines are getting tighter, and buyers are getting more selective. Companies are therefore seeing value in how real-time 3D can better navigate these challenges, and recognise the key is having the right talent.

In fact, we think real-time 3D content will be the leading form of content worldwide, and we are seeing companies in a range of industries undergoing major digital transformations using real-time 3D and digital twins to accelerate those transformations. These include the automotive, manufacturing, infrastructure, retail, energy, and government sectors.

As a result, this is fueling a surprising hiring trend: Many former game developers are advancing their careers and joining major corporate enterprises, in part because of their familiarity with software like Unity, which offers the 3D experiences companies need to learn to use.

With companies increasingly adopting real-time 3D technologies to create digital content for their customers, we recently launched a specific offering targeted at helping enterprises in industries outside of gaming. Unity Industry is a set of products and services that retail, infrastructure, manufacturing, and other industries can use to create and deploy interactive 3D experiences.

In light of the reception to the runtime fee model, how has Unity adjusted its overall strategy, particularly in its approach to non-gaming-related ventures like digital twins?

We appreciate that we should have engaged more of our customers in the process. Engaging doesn’t mean achieving consensus, but at the very minimum, we wouldn’t have surprised them. Our customers and community are what make Unity great; we know we need to listen and work hard to earn their trust. That is why we made the changes we have to the runtime fee for our gaming community. Our goal with this policy is to ensure we can continue to support our creators today and tomorrow, and keep deeply investing in our game engine.

For example, at Unite, Unity’s annual developer conference, we announced the upcoming launch of Unity 6 in 2024 (formerly known as the 2023 LTS). This update will bring enhancements in visuals and performance. Additionally, it aims to streamline multiplayer game creation and scalability, while expanding the capabilities in AI, mobile features, and VR device support.

Where do you see the future of gaming and its related applications heading? What developments are you most excited about, and how do you envision them influencing gaming and non-gaming industries?

We believe that AI represents a transformative opportunity across gaming and non-gaming industries. It has the potential to lower barriers for creation, accelerate workflows so creators can do more, and bring games and other real-time 3D experiences to life in all new ways.

For gaming, our goal is to meet developers where they are, help them, and keep them in control. Our tools can help accelerate productivity and supplement existing workflows.  Ultimately, we want creators to iterate and experiment more rapidly so they can find the fun in their games and experiences. Our tools can help them with tasks like optimising code or creating variations in textures.

Examining a digital twin of a vehicle on a tablet, where virtual meets reality in advanced manufacturing. Image courtesy of Unity Technologies.

For digital twins, we see a similar extension of capabilities. A real-time digital twin goes well beyond merely reporting or displaying the status of a complicated system, manufacturing line, or facility. It advances into realms that might have been previously considered science fiction, where the twin proposes solutions to problems and addresses questions that a human operator has not yet asked.

AI is probably the most important trend that will impact real-time digital twins. Unity has really been researching AI for years; we’ve anticipated AI being the second biggest inflection point in the history of the gaming industry in the last 20 years after the emergence of 3D. In our industry business, we use AI for complex, industry-related training scenarios. We’re developing simulation techniques for robotics and industrial machine training, as well as creating advanced synthetic data sets to make training more efficient using full 3D tools and technologies.

We believe that AI-powered real-time digital twins can do more than just status reporting of complex systems, manufacturing lines, or facilities. They have the potential to venture into areas once considered science fiction, empowering human operators to anticipate solutions to problems and preemptively address questions.