Taming GenAI: Improving Relevance Using Knowledge Graphs

Large language models rely on a lot of factors in creating answers to prompts: access to contextual data from your database(s), well-formed input from properly designed prompts and queries, and near-zero latency across data systems. If the LLM doesn’t get the context and data it needs in near-real-time, the LLM might return an incomplete or oversimplified answer. It might also hallucinate; filling in irrelevant or incorrect data in an attempt to construct a better answer.

Hallucinations and inaccuracies can happen when there are explicit relationships in the data that may not be obvious to an LLM by default. The most common way to reduce these hallucinations is to use retrieval-augmented generation (RAG), providing the LLM with the correct contextual data so it can answer the query. Most AI developers struggle with hallucination reduction. Many factors play into reducing hallucinations. The choice of LLM, data preparation, prompt engineering, construction of a RAG pipeline, and other factors all contribute to the accuracy, or lack of, answers generated by LLMs.

Semantic similarity, powered by vector search, is a good tool for GenAI as it enables massive amounts of data to be considered and retrieved, but semantic similarity can miss relevant data in some situations. For instance, the names of athletes on a sports team are not semantically similar to the team name, but context about the players may be very relevant to answering a question about what happened in last night's game. A legal document might reference a completely different document for additional details that are needed to understand what is being described. These are connections that are important, but are not captured by semantic similarity. In GenAI applications this can lead to incomplete query results, decreased relevancy, and increased hallucinations. Adding knowledge graphs to vector data gives LLMs better content leading to more accurate responses.

Content-centric knowledge graphs take pieces of content, such as a few sentences of text, an image, or a video clip, and represent explicit relationships among the content. These relationships may be structural; for example, this text refers to a specific figure or image, or this document has a hyperlink to another document. KG may also involve metadata. For example, email and phone transcripts of customer service interactions often have a customer ID that tells us two support interactions involve the same customer. KGs define relationships that go beyond semantic similarity to map specific connections between content and provide more complete context to GenAI applications.

Knowledge graphs are not needed in every GenAI application. But KGs can be very useful in some situations where your unstructured data has clear explicit relationships.

• A website or wiki with many hyperlinks between documents.
• Legal or financial documents that frequently reference other documents.
• Multimodal datasets that have customer or similar metadata that tells you all this different content is related.

In these situations, a content-centric KG can provide more relevant context to an LLM than a simple semantic search. KGs also provide a clear explanation of how search results were obtained since they are based on explicit relationships among data.

Employing a content-centric approach when building knowledge graphs helps developers encode more information into their data layer. This up-front work during data ingestion reduces hallucinations without adding latency at run time, and diminishes the need for complex agentic design patterns that have high latency and cost due to multiple LLM calls. Creating smarter context with faster and more accurate query responses meets customer demand for real-world and real-time AI applications. Furthemore, RAG pipelines can be implemented with both semantic search and KG all stored in a single database, reducing system complexity and tool interdependence.

Graph databases have existed for many years. They were first developed to enhance complex analytics and they excel at this. Graph DBs are a complex, brittle solution to the relevancy problem and so aren’t an ideal solution for GenAI.

Graph DBs require provisioning and maintaining an external application. In addition to the infrastructure costs, designing, iterating over and maintaining the relationship model is difficult. Relationship models in graph DBs require specialized skills to develop and iterative testing to ensure the model meets the needs of the application. Using entity Knowledge Graphs from a graph DB is a lossy process, extracting and discarding information. This is not only time-consuming but can result in relevant data being removed during the process. Query latency increases, as do data transfer costs, as results are extracted from the graph DB and fed back to the GenAI application.

No. Entity based knowledge graphs are constructed by sets of triples. Each triple contains two entities, a person, place, thing, or concept, and relationships between them. For example “Walt Disney” (entity) “is the founder of” (relationship) “Disneyland” (entity).

This type of knowledge graph has been the dominant form of knowledge graph used in data and AI use cases for many years. Entity-based KGs are very difficult to build and maintain. Historically this has been a very large barrier to entry and few companies have succeeded with these types of graphs.

LLMs are able to construct KG triples, making these graphs more accessible, but the cost is high due to multiple LLMs calls, and there are challenges with entity deduplication as well as the size of the graph. In order to reduce hallucinations, it’s often necessary to capture one entity relationship triple per “fact” you want your genAI to answer.

Entity-based KGs are very fine-grained and strip away many supporting details. LLMs are very good at using large contexts to answer questions, which is a powerful benefit to content-centric KGs.

Entity-based KGs also often require complex graph traversal methods that are difficult to construct queries for and require specific graph databases to use effectively, adding to overall system complexity.

Many solutions for creating Knowledge Graphs require manually defining graph relationships. We created Vector Graph as part of the DataStax AI Platform. Vector Graph automates the creation of Knowledge Graphs with the addition of just two lines of code. Vector Graph is built into Langflow and Astra DB; try it for free today.