openGauss AGEGraph + 大模型实现 GraphRAG,助力更强 RAG
传统 RAG(检索增强生成)通常使用向量数据库检索相关文档来提高大模型的问题回答准确性,但是基于向量的 RAG 不能够很好的表达事物之间的联系。而GraphRAG基于知识图谱的检索增强生成,能够提供结构化检索能力,将图作为传统的RAG多路召回的一环,使知识表达更易解释、对复杂关系更易支持。
目前 openGauss AGEGraph 已经提供了图数据库引擎的能力。本文将详细介绍如何借助大模型以及 openGauss 图数据库快速提取文档知识图谱并持久化。 随后,将问题输入大模型,大模型自动提取关键词并转换为图查询语句连接 openGauss 图数据库进行图数据检索。最后,将图数据输入大模型生成问题答案反馈给用户。
环境准备
openGauss 容器化部署
可参考openGauss容器镜像安装部署openGauss。
openGauss容器启动后可以执行docker ps查看容器是否在运行 输出
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
253714c9c869 opengauss:7.0.0-rc1 "entrypoint.sh gauss…" 8 minutes ago Up 8 minutes 0.0.0.0:6543->5432/tcp opengauss-agepython依赖安装
pip install -U langchain_community langchain langgraph langchain-ollama langchain-experimental langchain-openai langchain-opengauss利用 模型和 openGauss AGEGraph 快速构建知识图谱和检索
本文使用百炼大模型,取得api-key,并且配置DASHSCOPE_API_KEY环境变量
初始化大模型
from langchain_community.llms import Tongyi
import os
os.environ["DASHSCOPE_API_KEY"] = "sk-**"
graph_llm =Tongyi(model="qwen-plus", temperature=0, base_url="https://dashscope.aliyuncs.com/compatible-mode/v1")通过大模型提取文本数据中的实体和关系
from langchain_core.documents import Document
from langchain_experimental.graph_transformers import LLMGraphTransformer
#设置需要提取的实体类型和关系
llm_transformer = LLMGraphTransformer(
llm=graph_llm,
allowed_nodes=["Person", "Organization", "Location", "Award", "ResearchField"],
allowed_relationships = ["SPOUSE", "AWARD", "FIELD_OF_RESEARCH", "WORKS_AT", "IN_LOCATION"],
)
text = """
Marie Curie, 7 November 1867 – 4 July 1934, was a Polish and naturalised-French physicist and chemist who conducted pioneering research on radioactivity.
She was the first woman to win a Nobel Prize, the first person to win a Nobel Prize twice, and the only person to win a Nobel Prize in two scientific fields.
Her husband, Pierre Curie, was a co-winner of her first Nobel Prize, making them the first-ever married couple to win the Nobel Prize and launching the Curie family legacy of five Nobel Prizes.
She was, in 1906, the first woman to become a professor at the University of Paris.
Also, Robin Williams.
"""
# 将文本转化为图
documents = [Document(page_content=text)]
graph_documents = llm_transformer.convert_to_graph_documents(documents)
print(f"Nodes from graph doc:{graph_documents[0].nodes}")
print(f"Relationships from graph doc:{graph_documents[0].relationships}")输出
Nodes from graph doc:[Node(id='Nobel Prize', type='Award', properties={}), Node(id='University of Paris', type='Organization', properties={}), Node(id='Marie Curie', type='Person', properties={}), Node(id='radioactivity', type='ResearchField', properties={}), Node(id='Pierre Curie', type='Person', properties={})]
Relationships from graph doc:[Relationship(source=Node(id='Marie Curie', type='Person', properties={}), target=Node(id='radioactivity', type='ResearchField', properties={}), type='FIELD_OF_RESEARCH', properties={}), Relationship(source=Node(id='Marie Curie', type='Person', properties={}), target=Node(id='Nobel Prize', type='Award', properties={}), type='AWARD', properties={}), Relationship(source=Node(id='Marie Curie', type='Person', properties={}), target=Node(id='Nobel Prize', type='Award', properties={}), type='AWARD', properties={}), Relationship(source=Node(id='Marie Curie', type='Person', properties={}), target=Node(id='Pierre Curie', type='Person', properties={}), type='SPOUSE', properties={}), Relationship(source=Node(id='Pierre Curie', type='Person', properties={}), target=Node(id='Nobel Prize', type='Award', properties={}), type='AWARD', properties={}), Relationship(source=Node(id='Marie Curie', type='Person', properties={}), target=Node(id='University of Paris', type='Organization', properties={}), type='WORKS_AT', properties={})]openGauss AGE插件安装
如果对应的数据库已经创建age插件,此步骤可以忽略
import psycopg2
connection = psycopg2.connect(
database = "omm",
user = "gaussdb",
password = "YourPassoword",
host = "Your IP",
port = 8888
)
try:
cursor = connection.cursor()
sql_qeury = "create extension age;"
cursor.execute(sql_query)
connection.commit()
except Exception as e:
print(e)
finally:
cursor.close()
connection.close()实例化 openGauss AGEGraph 客户端,持久化图数据
from langchain_opengauss import openGaussAGEGraph, openGaussSettings
conf = openGaussSettings(
database = "omm",
user = "gaussdb",
password = "YourPassoword",
host = "Your IP",
port = 8888
)
graph=openGaussAGEGraph(graph_name='graph_test1',conf=conf,create=True)
graph.add_graph_documents(graph_documents)
graph.refresh_schema()大模型 Text2Cypher:提取问题关键词并生成图检索语句
通过增加提示词cypher_prompt,可以让大模型生成的图查询语句更加精准
from langchain_core.prompts import PromptTemplate
from langchain.chains import GraphCypherQAChain
cypher_prompt = PromptTemplate(
template="""你是 AGE Cypher 查询生成的专家。
使用以下架构生成一个 Cypher 查询,以回答给定问题。
不要出现name和properties和cypher
架构:
{schema}
问题:{question}
Cypher 查询:""",
input_variables=["schema", "question"],
)
chain = GraphCypherQAChain.from_llm(
graph_llm, graph=graph, verbose=True, allow_dangerous_requests=True, cypher_validation=True, return_intermediate_steps=True,cypher_prompt=cypher_prompt
)
question = "Who get Nobel Prize ?"
result = chain.invoke({"query": question})输出
Entering new GraphCypherQAChain chain... Generated Cypher: MATCH (p:Person)-[:AWARD]->(a:Award) WHERE a.id = "Nobel Prize" RETURN p.id Full Context: [{'p_id': 'Pierre Curie'}, {'p_id': 'Marie Curie'}]
大模型依据检索的图数据生成回答
设置提示词
prompt = PromptTemplate(
template="""You are an assistant for question-answering tasks.
Use the following pieces of retrieved context from a graph database to answer the question. If you don't know the answer, just say that you don't know.
Use two sentences maximum and keep the answer concise:
Question: {question}
Graph Context: {graph_context}
Answer:
""",
input_variables=["question", "graph_context"],
)生成回答
from langchain_core.output_parsers import StrOutputParser
composite_chain = prompt | graph_llm |StrOutputParser()
answer = composite_chain.invoke(
{"question": question, "graph_context": result}
)
print(answer)输出
Marie Curie and Pierre Curie received the Nobel Prize. They were recognized for their groundbreaking work in radioactivaity.总结
结合大模型和 openGauss AGEGraph 图数据库,我们可以快速进行知识图谱的生成、持久化以及检索,完成 GraphRAG 的搭建。此外,当前 openGauss 还支持标量、向量类型存储与检索,只需部署 openGauss 数据库,即可实现支持标量、向量、知识图谱多路召回的更强 RAG 系统。
