Galileo, a leader in developing generative AI for the enterprise, today announced the launch of its latest groundbreaking Retrieval Augmented Generation (RAG) & Agent Analytics solution. The offering is meant to help businesses speed development of more explainable and trustworthy AI solutions.
As retrieval-based methods have fast become the most popular method for creating context-aware Large Language Model (LLM) applications, this innovative solution is designed to dramatically streamline the process of evaluating, experimenting and observing RAG systems.
“Galileo’s RAG & Agent Analytics is a game-changer for AI practitioners building RAG-based systems who are eager to accelerate development and refine their RAG pipelines,” said Vikram Chatterji, CEO and co-founder of Galileo. “Streamlining the process is essential for AI leaders aiming to reduce costs and minimize hallucinations in AI responses.”
The Problem: Inefficiencies in Working With RAG Systems and Agent Analytics
RAG systems have become increasingly popular with developers of LLMs. RAG supplements an LLM’s general knowledge with domain-specific context, so the LLM can provide domain-specific results. And yet, before Galileo’s new RAG & Agent Analytics solution, the complexity of RAG systems and their many moving parts have required labor-intensive manual evaluation, and their inner workings can be somewhat of a black box for AI builders.
Limited insight into chunking strategies, context data and embedding models can make it difficult to optimize and debug conventional RAG systems. The manual work often led to inefficient retrieval of contextual data, increased production costs, and a lack of transparency in understanding the influence of different components within the RAG system.
The Galileo Way: Unprecedented Visibility into RAG Evaluation
Galileo’s RAG & Agent Analytics transforms this process by embedding advanced insights and metrics directly into the user’s existing workflow, with easy access through an intuitive Galileo user interface (UI). Powered by research-backed metrics developed by the company’s Galileo Labs R&D unit, this solution provides unprecedented visibility into each step of the RAG workflow, allowing for rapid evaluation, error detection, and iteration.
Key Capabilities of RAG & Agent Analytics Include:
- Proprietary Chunk Evaluation Metrics: Galileo’s unique metrics, such as Chunk Attribution and Chunk Utilization, empower users to optimize chunking strategies, leading to more precise and accurate AI responses. For reference, Galileo’s chunk attribution accuracy is 86%, which is 1.36x more accurate than baseline evaluations with GPT-3.5-Turbo, while its chunk utilization accuracy sits at 74% – 1.69x more accurate than baseline evaluations with GPT-3.5-Turbo.
- Context Evaluation and Explainability: With proprietary metrics like Completeness and Context Adherence, Galileo enables applications to explain their responses more clearly, enhancing trustworthiness and reliability. Galileo’s context adherence accuracy is 74% – 1.65x more accurate than baseline evaluations with GPT-3.5-Turbo, and its completeness is 80% accurate, which is 1.61x more accurate than baseline evaluations with GPT-3.5-Turbo.
- Simple Visual Tracing for Debugging: Galileo’s visual traces provide a user-friendly way to track each step from input to output, identifying errors with ease.
