2023 is shaping up to be a milestone year for artificial intelligence (AI), and the rapid pace of change will impact nearly every industry.
Only within the field of generative AI in software development have we begun to glimpse the potential for transformation.
McKinsey defines generative AI as algorithms that can be used to create new content, including audio, code, images, text, simulations, and videos. Once trained, generative AI can generate new examples similar to those it was trained on. Generative AI is set to revolutionize the way engineers communicate with technology and how technologies interact with each other.
According to IDC Group Vice President Stephen Elliot, the broader goal for AI advancement should “go beyond chatbots and language processors, toward holistic intelligence solutions that ensure a single source of truth, break down silos , ensure knowledge workers work smarter and deliver clear business results.” ”.
The recently announced New Relic Grok represents a breakthrough in AI-powered assistants. New Relic Grok is the world’s first generative AI observability assistant powered by large language models (LLM). By extracting the benefits of large language models, New Relic Grok has the potential to dramatically amplify an engineer’s productivity at every stage of the software development lifecycle.
The evolution of observability with generative AI
The breadth of new possibilities for using AI in any field can seem immense. Having established the basic blueprint for advancing AI in observability, we now have a clear roadmap ahead, guided by the goal of empowering engineers to identify and solve problems faster. What lies ahead is a phased evolution in which each generative AI application improves and enhances the user experience.
Phase 1: Optimize user experience with contextual AI assistance
Current generative AI solutions are largely out of band. That is, a user must context-switch from a domain-specific product to a general artificial intelligence assistant like ChatGPT to benefit from generative AI. An example of this is asking ChatGPT a “How to…” question about a specific product and then switching to that product to perform the instruction. While this approach is useful, it is not optimal.
In the next phase, the user experience will improve as products will introduce domain-specific in-band wizards that behave like ChatGPT and are available directly within the product experience. These wizards will be tuned to answer domain-specific questions and perform domain-specific tasks. They will offer the added advantage of taking into account the user’s full context without requiring the user to provide context with each question. This will allow the assistant to respond to questions and tasks in a way that reflects the context and current state of the user. And since these assistants will be much more limited in scope, they will be much less likely to produce plausible but factually incorrect answers, which plagues existing general-purpose AI assistants. Objective, correct answers will need to be provided before customers will rely on generative AI to automate the complex task of troubleshooting and eventually remediating incidents, a common use case for observability solutions.
Phase 2: Improve decision making and efficiency with predictive/prescriptive intelligence
The next phase of the AI assistant will be to produce ideas and advice without being asked. For example, a user browsing an APM-enabled application may receive unsolicited advice to “tweak settings in the Java VM to improve performance.” By accepting the recommendation, the assistant can schedule a task to implement the recommendation, and by rejecting it, the assistant learns to avoid recommendations that users are more likely to reject. For engineers, automated recommendations will provide immediate value without requiring years of experience to unlock that value.
One might wonder how an AI assistant acquires this knowledge. Do you learn through observation or do you learn from someone else’s experience? Both scenarios are plausible, and platforms that gain insights from user observations will require transparent controls, guidelines, and governance to prevent data and cues from being used to improve models without proper consent. Something to keep in mind here is that generative AI is only as good as the data it has at its disposal. By combining large language models with the breadth of a unified telemetry data platform, New Relic Grok is designed to deliver better, more reliable, high-quality AI responses.
Phase 3: Shift Left with Autonomous Discovery and Automation
The next and perhaps final phase will see the introduction of greater autonomy and automation to support the entire practice of shifting left in observability, with AI assistants acting on behalf of users with varying degrees of autonomy and human-level oversight.
In this phase, engineers will have the ability to assign the assistant a common goal, including the constraints and barriers they must follow to achieve the goal. For example, the assistant may be tasked with finding opportunities to improve the performance of a particular service without increasing resource allocation. In this scenario, the wizard can leverage prior knowledge to identify and resolve N+1 query pattern issues by analyzing distributed traces. To identify these opportunities, design a plan, and validate the approach, the assistant will have at their disposal a variety of data, tools, and telemetry environments, not unlike a human.
These may include development, testing, automation, simulation, and experimentation tools, followed by development environments, testing, staging, and finally production. Through a process of self-awareness and self-monitoring, the assistant will design a plan that includes the set of tasks and sequencing necessary to achieve the goal within the allowed limitations and barriers. The plan and results used to validate the approach in non-production environments will be shared with a human for approval before the wizard deploys the approach to production.
Giving an assistant the freedom to explore, discover, experiment, and iterate on goals they know will lead to a tipping point that ultimately frees engineers to focus on more complex and unique situations that demand human-level intelligence.
While reaching the stage of autonomous discovery and automation may seem unrealistic today, more challenging autonomous systems already exist, namely autonomous vehicles. Embracing the evolution of generative AI across the entire software development lifecycle will help organizations achieve new levels of efficiency and performance.
Peter Marelas is chief architect for Asia-Pacific and Japan at New Relic.
