2023 is shaping up to be a year of artificial intelligence (AI) milestones, with the rapid pace of change impacting almost every industry. Within the field of generative AI in software development alone, we have started to see a glimpse of the potential for transformation. 

McKinsey defines generative AI as algorithms that can be used to create new content, including audio, code, images, text, simulations, and videos. Once trained, generative AI can then generate new examples similar to those it was trained on. Generative AI is set to revolutionise how engineers communicate with technology and how technologies interact with each other. 

According to IDC Group Vice President Stephen Elliot, the broader goal for AI advancement should “go beyond chatbots and language processors, to holistic intelligence solutions that ensure a single source of truth, break down silos, ensure knowledge workers work smarter, and deliver clear business results.”

The evolution of observability with generative AI

The breadth of new possibilities for using AI in any field can seem immense. With the establishment of a foundational model for AI advancement in observability, a clear roadmap has emerged, aiming to empower engineers to identify and resolve issues faster. What lies ahead is a phased evolution where each application of generative AI improves and enhances the user experience.

Phase 1: Optimise user experience with contextual AI assistance

Today’s generative AI solutions are largely out of band. This means that users must switch from a domain-specific product to an artificial general intelligence assistant like ChatGPT to benefit from generative AI. For instance, users may ask ChatGPT a specific product-related question and then switch to that product to carry out the instruction. While this approach is useful, it is not optimal. 

In the next phase, the user experience will be enhanced as products introduce in-band assistants that are domain-specific and behave similar to ChatGPT. These assistants will be integrated directly within the product experience and fine-tuned to address domain-specific queries and tasks. They will have the added advantage of considering the user’s complete context without requiring the user to provide context with every question. This will enable the assistant to respond to queries and tasks in a manner that aligns with the user’s current context and state.

Additionally, these assistants will have narrower scopes, minimising the likelihood of producing plausible yet factually incorrect responses, a challenge commonly faced by existing general-purpose AI assistants. Ensuring the provision of accurate and factual responses is crucial for building customer trust in generative AI, particularly when it comes to automating complex troubleshooting and incident remediation, which are common use cases for observability solutions.

Phase 2: Improve decision-making and efficiency with predictive/ prescriptive intelligence 

The next phase of the AI assistant aims to generate insights and offer advice without being prompted. As an example, while a user browses an APM-enabled application, they may receive unsolicited advice such as “adjust settings in the Java VM to improve performance.” Upon accepting the recommendation, the assistant can schedule a task to implement it. Conversely, by rejecting the recommendation, the assistant learns to avoid similar suggestions that users are more likely to decline. This way, automated recommendations provide immediate value to engineers, eliminating the need for years of experience to unlock such benefits.

One might wonder how an AI assistant acquires such knowledge. Does it learn through observation, or does it learn from the experiences of others? Both scenarios are plausible. Platforms that gain knowledge through user observation must establish transparent controls, guidelines, and governance to ensure that data and prompts are not used to enhance models without appropriate consent. It is important to note that the effectiveness of generative AI relies heavily on the data it has access to.

Phase 3: Shift left with autonomous discovery and automation

The next, and potentially final, phase will introduce greater autonomy and automation to support the entire practice of shifting left in observability. AI assistants will act on behalf of users with varying degrees of autonomy while still maintaining human-level supervision.

In this phase, engineers will be able to assign common objectives to the assistant, along with constraints and guardrails that must be followed to achieve those objectives. For instance, the assistant may be tasked with identifying opportunities to improve the performance of a specific service without increasing resource allocations. To accomplish this, the assistant can leverage prior knowledge in identifying and resolving N+1 query pattern issues by analysing distributed traces.

To identify these opportunities, devise a plan, and validate the approach, the assistant will have access to various telemetry data, tools, and environments, similar to a human. These resources may include development, testing, automation, simulation, and experimentation tools, as well as different environments such as development, testing, staging, and production. Through self-awareness and self-supervision, the assistant will develop a plan that includes the necessary tasks and sequencing to achieve the objective within the specified constraints and guardrails. The plan and the results used to validate the approach in non-production environments will be shared with a human for approval before the assistant implements the approach in production.

Granting the assistant the freedom to explore, discover, experiment, and iterate on known objectives will bring about an inflection point that allows engineers to focus on more intricate and unique situations that require human-level intelligence.

Although reaching the stage of autonomous discovery and automation may currently seem unrealistic, there are already more complex autonomous systems in existence, such as autonomous vehicles. Embracing the evolution of generative AI throughout the software development lifecycle will enable organisations to achieve new levels of efficiency and performance.