The current popular RAG retrieval approach involves using large models' embedding algorithms to embed data into a vector database, then converting user queries into vectors to retrieve similar data from the vector database. This retrieved data is then formatted into a context template and fed into an LLM for querying.

Directly converting user queries into vector queries may not yield good results. For example, if we store product vectors in a vector database and a user asks, "I want a black wool sweater under 20 yuan," the traditional embedding algorithm might convert this query into a vector that only matches "black wool sweater," potentially losing important details.

To address this, optimizing the query statement for retrieval is essential. Langchain's self-querying is an effective method. Here, we use Alibaba Cloud's DashVector vector database and DashScope LLM to demonstrate this optimization, which shows promising results.

Many online resources focus on OpenAI's embedding and LLM, but Alibaba's LLM and vector database are now very advanced. With OpenAI's API access blocked in China, domestic cloud services offer cost-effective and reliable solutions. Consider trying them out. I have written several practical articles on DashVector and DashScope; interested readers can refer to:

LLM - Text Chunking (langchain) and Vectorization (Alibaba Cloud DashVector) Storage, Embedding LLM Practice LLM - Alibaba Cloud DashVector + ModelScope Multimodal Vectorization Real-Time Text Search Image Practical Summary LLM - langchain with Alibaba DashScop (Tongyi Qianwen Large Model) and DashVector (Vector Database) Integration Summary

Prerequisites

• Ensure you have access to the Tongyi Qianwen API key and vector retrieval service API key
• Install dependencies: pip install langchain pip install langchain-community pip install dashvector pip install dashscope

Self-Querying Overview

In short, self-querying allows us to structure user queries into two parts:

• Query: A vector query representing the semantic meaning of the query
• Filter: Metadata about the query content

For example, if the user inputs, "What did Bar say about Foo?" self-querying structures it into two parts:

• Retrieve data related to Foo
• Filter by author Bar

Implementation

Replace DASHSCOPE_API_KEY, DASHVECTOR_API_KEY, and DASHVECTOR_ENDPOINT with your own keys from Alibaba Cloud.

import os

from langchain_core.documents import Document from langchain_community.vectorstores.dashvector import DashVector from langchain_community.embeddings.dashscope import DashScopeEmbeddings from langchain.chains.query_constructor.base import AttributeInfo from langchain.retrievers.self_query.base import SelfQueryRetriever from langchain_community.chat_models.tongyi import ChatTongyi from langchain_core.vectorstores import VectorStore

class SelfQuerying:

def __init__(self):
    # Ensure DASHSCOPE_API_KEY and DASHVECTOR_API_KEY are set
    os.environ["DASHSCOPE_API_KEY"] = ""
    os.environ["DASHVECTOR_API_KEY"] = ""
    os.environ["DASHVECTOR_ENDPOINT"] = ""

    self.llm = ChatTongyi(temperature=0)

def handle_embeddings(self)->'VectorStore':

    docs = [
        Documennt(
            page_content="A bunch of scientists bring back dinosaurs and mayhem breaks loose",
            metadata={"year": 1993, "rating": 7.7, "genre": "science fiction"},
        ),
        Document(
            page_content="Leo DiCaprio gets lost in a dream within a dream within a dream within a ...",
            metadata={"year": 2010, "director": "Christopher Nolan", "rating": 8.2},
        ),
        Document(
            page_content="A psychologist / detective gets lost in a series of dreams within dreams within dreams and Inception reused the idea",
            metadata={"year": 2006, "director": "Satoshi Kon", "rating": 8.6},
        ),
        Document(
            page_content="A bunch of normal-sized women are supremely wholesome and some men pine after them",
            metadata={"year": 2019, "director": "Greta Gerwig", "rating": 8.3},
        ),
        Document(
            page_content="Toys come alive and have a blast doing so",
            metadata={"year": 1995, "genre": "animated"},
        ),
        Document(
            page_content="Three men walk into the Zone, three men walk out of the Zone",
            metadata={"year": 1979, "director": "Andrei Tarkovsky", "genre": "thriller", "rating": 9.9},
        ),
    ]
    # Specify the collection name in the vector database
    vectorstore = DashVector.from_documents(docs, DashScopeEmbeddings(), collection_name="langchain")
    return vectorstore

def build_querying_retriever(self, vectorstore: 'VectorStore', enable_limit: bool=False)->'SelfQueryRetriever':
    
    metadata_field_info = [
        AttributeInfo(
            name="genre",
            description="The genre of the movie. One of ['science fiction', 'comedy', 'drama', 'thriller', 'romance', 'action', 'animated']",
            type="string",
        ),
        AttributeInfo(
            name="year",
            description="The year the movie was released",
            type="integer",
        ),
        AttributeInfo(
            name="director",
            description="The name of the movie director",
            type="string",
        ),
        AttributeInfo(
            name="rating", description="A 1-10 rating for the movie", type="float"
        ),
    ]
    document_content_description = "Brief summary of a movie"
    retriever = SelfQueryRetriever.from_llm(
        self.llm,
        vectorstore,
        document_content_description,
        metadata_field_info,
        enable_limit=enable_limit
    )
    return retriever

def handle_query(self, query: str):
    
    retriever = self.build_querying_retriever(self.handle_embeddings())
    resposne = retriever.invoke(query)
    return response

if name == 'main': q = SelfQuerying()

# Filtering by query attributes only
print(q.handle_query("I want to watch a movie rated higher than 8.5"))

# Filtering by query attributes and semantic content
print(q.handle_query("Has Greta Gerwig directed any movies about women"))

# Complex filtering query
print(q.handle_query("What's a highly rated (above 8.5) science fiction film?"))

# Complex semantic and filtering query
print(q.handle_query("What's a movie after 1990 but before 2005 that's all about toys, and preferably is animtaed"))

The code performs three main steps:

• Perform embedding, storing documents with metadata into DashVector
• Construct the self-querying retriever, requiring metadata field information and a brief document content description
• Execute the query

The output of the code is as follows:

"I want to watch a movie rated higher than 8.5"

[Document(page_content='Three men walk into the Zone, three men walk out of the Zone', metadata={'director': 'Andrei Tarkovsky', 'genre': 'thriller', 'rating': 9.9, 'year': 1979}), Document(page_content='A psychologist / detective gets lost in a series of dreams within dreams within dreams and Inception reused the idea', metadata={'director': 'Satoshi Kon', 'rating': 8.6, 'year': 2006})]

"Has Greta Gerwig directed any movies about women"

[Document(page_content='A bunch of normal-sized women are supremely wholesome and some men pine after them', metadata={'director': 'Greta Gerwig', 'rating': 8.3, 'year': 2019})]

"What's a highly rated (above 8.5) science fiction film?"

[Document(page_content='A bunch of normal-sized women are supremely wholesome and some men pine after them', metadata={'director': 'Greta Gerwig', 'rating': 8.3, 'year': 2019})]

"What's a movie after 1990 but before 2005 that's all about toys, and preferably is animated"

[Document(page_content='Toys come alive and have a blast doing so', metadata={'genre': 'animated', 'year': 1995})]