It’s no surprise why I love graphs. As a 10-year veteran of the big data wars, I have seen data tech stacks rise and fall and some rise again. But throughout all that, one thing has remained constant: tabular data was the norm, even if the data doesn’t easily fit into a tabular format.
Put it in a table with 10 foreign keys or a view with six joins and call it a day. Then came the Generative AI and LLM revolution. Data can no longer afford to be stored in a way that does not accurately reflect its natural structure. For example, imagine this written word article stored as a table. It would be disjointed, difficult to read, and missing multiple context clues.
Enter the graph database: a database that builds natural relationships as part of its schema and data structure is easily query-able, and context is always obtainable.
In this article, I will demonstrate how graphs can be used to increase the accuracy of and build better, more accurate LLMs for your business.
Quick Introduction to Graphs
Before we dive into the uses of graphs to build stronger large language models, let’s take a brief introduction to what a graph is. Below is a diagram of a simple graph documenting persons, locations, titles, and companies.
The above graph database primarily consists of the following elements:
1. Vertices (or Nodes): These are the objects that serve as the center of the wheel of the data. They represent tangible objects or nouns (people, places, things). For people more familiar with traditional relational databases, these are the rows in a table, just represented in a non-tabular way.
2. Edges (or Links): Edges are simply described as relationships between vertices. As you can see in the diagram, multiple types of edges represent different types of relationships. Edges can be one way or undirected.
One way indicates the relationship between the vertices is different. For example, I live at 123 1st Ave, but ‘123 1st Ave’ doesn’t live at Elliot Johnson. However, for undirected edges, the edge is mutual between the two vertices. For example, I’m married to my wife, and my wife is married to me.
3. Properties (or Attributes): This is additional information that can be attached to a vertex or an edge. In our example, the person vertex has FirstName, LastName, and ID. An edge might also have properties like “relationship start date,” “relationship type,” etc.
These elements allow a graph database to represent complex relationships between data. It is also intuitive and aligned with how we naturally perceive the world. This will enable you to use graph query language that allows for fast data retrieval and the ability to traverse relationships naturally to gather insight.
How to Improve LLMs with Graphs
The most common way to improve LLMs is with RAGs (Retrieval Augmented Generation). LLMs are trained over a very long time with massive amounts of compute power and large data sets. This is a massive weakness to the current LLM architecture, as those data sets can’t include everything (not to mention the data may change while the models are being trained).
The most famous example of this is if you ask ChatGPT 4.5 the question, “Which planet in our solar system has the most moons?” It will say, Jupiter. This is incorrect. The correct answer is Saturn. While ChatGPT was training, astronomers discovered new moons orbiting Saturn that increased its total to 146, beating out Jupiter. RAGs solve that problem by retrieving new information around the context of the question before responding.
This is where graphs come in: they act as the perfect source of RAG data because they can be updated constantly without needing to retrain a model. Also, as we learned above, the relationships inside a graph database are key to the quality of the data extracted.
Below is a diagram of using graph-based RAGs to retrieve an answer that requires specific knowledge or insight only found in your data. In our example, pro-golfer Lionel Forest just had back surgery and asks a virtual nurse app how many milligrams of ibuprofen he can take.
Without a RAG, the answer would return 3200 mg per day as is standard for adults, but that would be wrong for Lionel and could lead to life-threatening injury. But because the RAG includes notes from the doctor, we learn that Lionel needs to take less because of deteriorated kidney functions.
Now, let’s take a quick look at the graph offerings in the industry today before we jump into the techniques you can use to improve your LLMs.
Wide World of Graph Databases
I often get questions about what graph database is best. But, like most software, there are trade-offs you need to consider based on your use case. I’ve prepared a list of the most popular graph databases focusing on what each does best:
Neo4J: Long live the king! Neo4J popularized the graph database and has led the industry in popularity for many years. With its Cypher query language, Bloom toolkit, and native integration with Apache Spark, Neo4J’s ease of use is unmatched.
Tigergraph: The young prince! Tigergraph, like its name, implies it’s built for speed and power. Entirely in memory and built to scale with business needs, Tigergraph is the darling of enterprise-level graph solutions. Native and Parallel query technology allows this solution to hop from vertex to vertex up to five times with a second response, making it ideal for fraud detection and total customer view dashboards.
Amazon Neptune: Neptune is the first cloud-native graph database. Built and used by Amazon, Neptune is ideal for small businesses already on AWS. Neptune’s use of high availability zones and fully managed support allow the user to feel secure in the reliability of the product and their data.
Azure Cosmos DB: Cosmos is not a native graph database. It is a NoSQL database similar to MongoDB. Many Azure users have started using it as a graph database because of its single-digit millisecond latencies. This allows for a great introduction to graph databases for smaller projects such as low vertex and edge cardinality but high vertex and edge volume applications. It’s multi-region and low latency add increased throughput and reliability.
Now that we have looked at a few of the different offerings in the graph database world, let’s move on to how to use graphs to improve LLMs.
Semantic Relationships
Graphs naturally represent semantic relationships between words or phrases in a text. The semantic relationships allow the models to gather and generate more contextually relevant outputs.
Example
Consider a sentence: “I am going to the bank to withdraw money.”
The word bank could be interpreted as having two meanings: a financial institution or the side of a river. Will your language model know which type of bank you mean? For example, the graph might contain the following relationships:
bank is related to money
bank is related to withdraw
river is related to flow
river is related to water
Based on these relationships, the language model can infer that in this context, bank refers to a financial institution, not the side of a river.
Knowledge Graphs
Knowledge graphs can be used to provide additional context to an input text. For example, if the input text mentions a person, the knowledge graph could provide information about that person. That includes anything stored in the knowledge graph, such as occupation, relationships, etc.
Example
Suppose we have a sentence: “Albert Einstein was born in Germany.”
A language model might generate the next sentence as: “He was a famous scientist.”
Consider we have a knowledge graph that contains more detailed information about Einstein, similar to our example above.
Einstein – is a – Physicist
Einstein – developed – Theory of Relativity
Einstein – won – Nobel Prize in Physics
The language model (with the help of this knowledge graph) can generate a more informative and contextually accurate sentence like: “He was a renowned physicist, best known for developing the Theory of Relativity.”
Graph Neural Networks (GNNs)
GNNs are used to process graph-structured data and generate embeddings (similarity matrices) that capture the structure of the graph. These embeddings are used as additional inputs to the language model, which improves the ability to generate more relevant outputs.
Example
For example, we have a social network graph where vertices represent users and edges represent friendships. Each user has a list of posts made to the site.
We want to generate a new post as a specific user. However, instead of just using the user’s previous posts as input to the language model (because that adds no value), we also want to consider their friends’ posts. This ensures that the post will fit the demographic of the people that will see it.
Here’s how we can use a GNN to achieve this:
First, we use the GNN to generate an embedding for each user based on their relative position in the social network graph. This embedding captures the neighborhood of the graph, such as who is friends with whom.
Next, we combine the user’s personal embedding with the text of their previous posts to form the input to the language model.
Then, we use the language model to generate a new post.
A GNN is special because we can incorporate information about the user’s social network into the language model based on the user’s neighborhood. These posts will not only be similar to the user’s previous posts but also to the posts of their friends.
GNNs can be used with a wide range of graph-structured data, and the embeddings they generate can capture a wide range of structural information, such as medical diagnoses, financial transactions, and fraud detection.
Graph Attention Networks (GATs)
GATs can weigh the importance of different vertices in a graph. This can help the language model focus on the most relevant parts of the input, leading to a more relevant output.
Example
Let’s use our social network graph again, where vertices represent people and edges represent friendships. Each person has a profile description, and we want to create a new profile description for a specific person using an LLM.
However, not all friends contribute equally to a person’s profile. Some of your friends and you share more interests; this makes certain profiles more relevant. This is where GATs come in.
GATs can assign different attention weights to different vertices in the graph. In this case, the GAT could assign larger weights to friends with more common interests.
These weights can then generate a weighted average of the friends’ profile descriptions, which can be used as additional input to the language model. This can help the model generate a profile description that is similar to the person’s current description and similar to the descriptions of their most influential friends.
Graph Convolutional Networks (GCNs)
GCNs can be used to capture local and global information in a graph. Think of it as taking a snapshot of either the local graph (similar to the example photo above) and finding where that lives in the global scheme of the full graph. With this, we can improve the language model’s understanding of the input text and generate better outputs.
Example
Consider a citation network, where vertices represent scientific papers and edges represent citations. Each paper has an abstract, and we want to generate a new abstract for a specific paper using a language model.
GCNs can be used to capture both local and global information in the graph. In this case, the GCN could capture local information about which papers cite or are cited by the specific paper, as well as global information about the overall structure of the citation network.
The GCN can generate an embedding for each paper based on this information. This embedding can then be combined with the text of the paper’s abstract to form the input to the language model.
This can help the model generate an abstract that is not only similar to the paper’s current abstract but also similar to the abstracts of related papers in the citation network.
Summary
Graphs can provide additional structure and context that help language models generate more accurate and relevant outputs.
However, integrating graphs with language models can be challenging and requires careful design and implementation.
phData’s team of expert data consultants can help design and implement graph architecture and bring your LLMs to the next level of quality.
