It goes without saying that enterprises need an AI strategy to thrive in today’s new business era. The question now is what that strategy should look like to capture AI’s full benefits while minimising risks.
With private AI, an organisation builds or fine-tunes its own AI models, hosts those models inside a protected environment, and feeds proprietary data into the models. A private AI approach is essential for any organisation that wants to move past basic use cases and scale up a holistic enterprise-class AI strategy.
Enterprise IT leaders increasingly recognise that private AI offers the control and protection they need, while public AI services such as ChatGPT could place them at risk if they’re not used carefully and strategically.
Keep your data private
Using public AI is risky for enterprises because of the potential for data leakage. Any training or inference data you feed into a public AI service could be accessed and stored by the service provider. This means that your proprietary data will no longer be under your control: Copies of your data could be leaked or sold to anyone, and you’d be powerless to stop it from happening. In contrast, when you build private AI models, no one outside your business has access to your models or the data you feed into them.
As businesses look to embed data privacy into their AI strategies, they must also distinguish between classical AI models — which have existed in some form for decades now — and generative AI models, which are still relatively new. Generative AI has been getting all the hype over the last two years, but classical AI use cases such as predictive analytics are still important. These varieties of AI have different infrastructure requirements and challenges. Building private AI models can be helpful for both generative AI and classical AI use cases, but the benefits will show up in different ways.
Let’s consider an example: One common enterprise use case for generative AI is employees using chatbots as virtual personal assistants. Employees use these bots to help with everyday tasks such as writing, brainstorming, and research. When they do this, they’re giving the underlying AI models access to all the same data they access, including sensitive proprietary data.
In the very early days of generative AI, there were several high-profile instances of heedless users causing data leakage that put their companies at risk. For example, software engineers at a major tech company inadvertently leaked confidential code by sending it to ChatGPT while searching for a bug fix. In response to incidents like these, other organisations took proactive steps, such as banning employees from accessing ChatGPT at work. This was an early sign that enterprise leaders were beginning to recognise the need for private AI models.
Meet your data governance requirements
With the launch of the European Union Artificial Intelligence Act, we’re starting to see the global AI regulatory landscape form. We can assume that other jurisdictions will follow the EU’s lead in setting new data governance requirements for certain AI use cases. Enterprises need to be ready to comply with these new regulations across their global operations. This may include requirements around data sovereignty, privacy, and lineage. They likely won’t be able to do that if they rely exclusively on public infrastructure to host their AI models and data sets.
A compliance strategy for AI needs to incorporate both training and inference workloads. Even if you do everything right from a training perspective — using private data centres to make sure your data stays within the right borders — public AI could still put you at risk when the time comes to do inference.
Also, public AI uses the internet to move data. From the moment your data hits the public internet, it’s no longer under your control. The service provider for the model could create a copy of your data and store that copy anywhere they want. This means that your business may no longer be complying with data sovereignty requirements, and you wouldn’t know it until well after the fact.
This is in contrast to private AI models, which use only private, dedicated interconnection to move data. And since you’re in complete control over your own private AI models, you can perform the necessary due diligence to ensure that those models aren’t moving or storing data anywhere they shouldn’t be.
Also, generative AI is based on foundational models trained on data crawled from the public internet. Even if you’re only fine-tuning a public model using your proprietary data, that model has already been pre-trained on publicly available data, some of which may be copyrighted material. This means that you have no control over what data the model uses to service your requests.
The public model could leverage data sets that your company isn’t legally allowed to access. Even though you didn’t choose to access those data sets, your company could still be held accountable for the fact they were used on your behalf. This could even make the company vulnerable to future legal action. By using only private AI models for high-risk use cases, you can avoid this risk.
Optimise your costs and performance
To balance the requirements of compute-intensive training workloads and latency-sensitive inference workloads, AI infrastructure must be distributed across different types of data centres in various locations. However, deploying these distributed components while balancing both costs and performance can be challenging, especially when relying exclusively on public AI infrastructure.
Businesses can accrue high inference costs by using public AI models for generative AI use cases. These costs may not seem significant on a per-use basis, but they could very easily grow out of control if all employees from across the organisation are allowed to use public LLMs as much as they want.
To get the AI infrastructure they need while keeping costs low, enterprises often turn to the public cloud. However, this might hurt performance for their AI workloads.
When companies host their AI workloads in the public cloud, they won’t be able to ensure proximity between data sources and compute locations. In turn, this means they won’t be able to effectively support latency-sensitive inference workloads.
Hosting models in private environments can help enterprises simultaneously avoid the high costs and network latency that could come from using public AI infrastructure. This is why it’s the best choice for enterprises looking to ensure predictable costs as they scale their AI strategies.
When enterprises adopt private AI, it doesn’t mean that they can’t incorporate public cloud services at all. Rather, it means they should do so strategically as part of a hybrid infrastructure that allows them to minimise the potential downsides. An ideal hybrid infrastructure would meet the needs of different AI workloads by offering:
- Compute infrastructure near data sources to support latency-sensitive inference workloads. This infrastructure may require advanced cooling and reliable energy resources to support the necessary hardware.
- A flexible data architecture with multi-cloud capabilities to support workloads suited for public cloud environments. Enterprises can maintain control by keeping copies of their data on infrastructure they own while transferring it to the public cloud when needed. This ensures predictable costs and reliable performance.
Laying the groundwork for private AI
The value of a private AI strategy is clear. Now, the question becomes how to build the infrastructure needed to execute that strategy.
The future of enterprise IT will rely on hybrid and multi-cloud models, with AI workloads distributed across private environments, public cloud platforms, and other locations as needed.