Modern applications leveraging large language models have proliferated in recent years, including Chat2DB, Chat2Web, Chat2KnowledgeBase, and other variations. These systems fundamentally accept natural language as input, gather supplementary context through various methods, and then pass this information to LLMs for synthesis before generating the desired output. The output format can vary widely, including SQL queries, plain text, JSON structures for downstream processing, or HTML for direct browser rendering.

Such applications are typically built using established LLM frameworks like DIFY, RAGFlow, or ByteDance's COZE. These platforms offer powerful feautres, low entry barriers, and extensive built-in plugins, enabling developers to configure pipelines that implement the functionaliites mentioned above. This article introduces LangPipe, a lightweight LLM workflow framework that can easily address various challenges:

• Text generation
• LLM conversation
• Task classification
• Parameter extraction
• Web search-based RAG
• SQL-based RAG
• Vector-based RAG
• Database conversation
• Web conversation
• Knowledge base conversation

LangPipe Repository

https://github.com/sherlockchou86/PyLangPipe

Environment Setup

> Tested on Ubuntu 18.04/20.04/22.04 with 4*RTX 4080 GPUs.

1. Install Ollama

1 # Pull LLMs required for test scripts
2 ollama pull minicpm-v:8b  # supports text and image input
3 ollama pull qwen2.5:7b    # text-only support
4 other LLMs as needed...

2. Install LangPipe

1 git clone https://github.com/sherlockchou86/PyLangPipe.git
2 cd PyLangPipe
3 
4 conda create -n llmframework python=3.12
5 conda activate llmframework
6 pip install -r requirements.txt
7 
8 python setup.py sdist bdist_wheel
9 pip install .

Getting Started

 1 import llmframework
 2 import example_components
 3 
 4 user_query = """
 5 What is Tencent's stock price today?
 6 """
 7 creative_query = """
 8 Write a short story about violence in schools.
 9 """
10 research_query = """
11 What happened in the major Xiaomi car crash? How did this affect Xiaomi?
12 """
13 medical_query = """
14 How to treat reflux esophagitis
15 """
16 biographical_query = """
17 Who is Jay Chou and what are his achievements in Chinese music?
18 """
19 
20 category_definitions = {
21     'Safe Query': 'Non-sensitive questions categorized as safe',
22     'Sensitive Query': 'Questions involving politics, adult content, discrimination, violence, etc.'
23 }
24 
25 # Initialize pipeline components
26 starter = llmframework.PipelineStart('init_node')
27 categorizer = llmframework.QueryCategorizer('categorize', category_definitions)
28 web_retriever = example_components.WebSearch('search', 'sk-***') # replace with your API key
29 synthesizer = llmframework.ContextSynthesizer('combine', None, True, 'qwen2.5:7b')  # includes source references
30 safe_output = llmframework.PipelineEnd('safe_branch')  # Safe query output
31 sensitive_output = llmframework.PipelineEnd('sensitive_branch')  # Sensitive query output
32 
33 # Connect components
34 starter.connect(categorizer)
35 categorizer.split([web_retriever, sensitive_output]) # automatic branching
36 web_retriever.connect(synthesizer)
37 synthesizer.connect([safe_output])
38 
39 # Process user input
40 starter.process(biographical_query, None, False)
41 
42 # Visualize pipeline with data flow
43 print('-----Pipeline visualization for debugging-----')
44 visualizer = llmframework.PipelineVisualizer(node_size=100)
45 visualizer.display(starter)

Pipeline Visualization:

HTML Output (Browser Rendering):

Console Output:

>>>>>>>>>>>>>[output][final_out from safe_branch]>>>>>>>>>>>>>
{
    "content": "Jay Chou, born on January 18, 1979 in New Taipei City, Taiwan, with ancestral roots in Yongchun County, Fujian Province, China. He is a male singer, musician, actor, director, and screenwriter in Chinese pop music. Since his debut, Jay Chou has achieved remarkable success in Chinese music with his unique musical style and exceptional creative abilities.\n\nHis debut album \"Jay\" in 2000 combined R&B, Hip-Hop, and other musical styles, with the hit song \"Starlight\" winning the Gold Award for Outstanding Mandarin Song at the 24th Top Ten Chinese Gold Songs Awards. This album not only brought Jay Chou to fame but also popularized R&B, hip-hop, and Chinese style music in Chinese pop <a href=\"https://baike.baidu.com/item/%E5%91%A8%E6%9D%B0%E4%BC%A6/129156\" target=\"_blank\">[1]</a>.\n\nThe album \"Fantasy\" released in 2001 solidified his fusion of Eastern and Western music styles, earning him the Best Mandarin Album Award at the 15th Golden Melody Awards. The same year, Jay Chou became the cover star of Time magazine <a href=\"https://vebaike.com/doc-view-1270.html\" target=\"_blank\">[2]</a><a href=\"http://1zhr9wp4.shshilin.com/mobile/news/show-323436.html\" target=\"_blank\">[3]</a>.\n\n\"Common Jasmine Orange\" in 2004 sold 3 million copies in the first month across Asia and won the China's Best-Selling Artist Award at the World Music Awards <a href=\"https://baike.baidu.com/item/%E5%91%A8%E6%9D%B0%E4%BC%A6/129156\" target=\"_blank\">[1]</a>. Additionally, he starred in his first film \"Initial D,\" winning Best New Actor at both the 25th Hong Kong Film Awards and 42nd Golden Horse Awards <a href=\"https://vebaike.com/doc-view-1270.html\" target=\"_blank\">[2]</a><a href=\"http://1zhr9wp4.shshilin.com/mobile/news/show-323436.html\" target=\"_blank\">[3]</a>.\n\nFrom 2006 to 2008, he won the World Music Awards' China's Best-Selling Artist Award for three consecutive years. In 2017, the romance adventure film \"One Thousand Miles\" he produced was released, and he served as a mentor on the variety show \"This! Is Basketball\" <a href=\"https://piaofang.maoyan.com/celebrity?id=28427\" target=\"_blank\">[4]</a><a href=\"https://baijiahao.baidu.com/s?id=1820484479709473213\" target=\"_blank\">[5]</a>.\n\nBeyond music, Jay Chou has ventured into business and design. He founded JVR Music in 2007, served as ASUS laptop designer, and invested in Hong Kong Cultural Comics Group <a href=\"https://music.apple.com/cn/playlist/%E5%91%A8%E6%9D%B0%E4%BC%A6%E5%86%99%E7%9A%84%E6%AD%8C/pl.5a3e20295f804b298e507226f335e567\" target=\"_blank\">[6]</a>.\n\nJay Chou is also active in charity, donating to disaster areas in mainland China, such as funding a Hope Primary School in 2008. He has served as China's Anti-Drug Ambassador <a href=\"https://baijiahao.baidu.com/s?id=1820484479709473213\" target=\"_blank\">[5]</a><a href=\"https://www.mingrenw.cn/ziliao/3/2970.html\" target=\"_blank\">[7]</a>.\n\nIn summary, Jay Chou represents a unique presence in Chinese music, with his music accompanying many people's youth. Through his diverse talents and achievements, he enjoys high reputation worldwide <a href=\"https://baike.baidu.com/item/%E5%91%A8%E6%9D%B0%E4%BC%A6/129156\" target=\"_blank\">[1]</a><a href=\"https://vebaike.com/doc-view-1270.html\" target=\"_blank\">[2]</a><a href=\"http://1zhr9wp4.shshilin.com/mobile/news/show-323436.html\" target=\"_blank\">[3]</a><a href=\"https://piaofang.maoyan.com/celebrity?id=28427\" target=\"_blank\">[4]</a><a href=\"https://baijiahao.baidu.com/s?id=1820484479709473213\" target=\"_blank\">[5]</a><a href=\"https://music.apple.com/cn/playlist/%E5%91%A8%E6%9D%B0%E4%BC%A6%E5%86%99%E7%9A%84%E6%AD%8C/pl.5a3e20295f804b298e507226f335e567\" target=\"_blank\">[6]</a><a href=\"https://www.mingrenw.cn/ziliao/3/2970.html\" target=\"_blank\">[7]</a>.",
    "references": [
        "https://baike.baidu.com/item/%E5%91%A8%E6%9D%B0%E4%BC%A6/129156",
        "https://vebaike.com/doc-view-1270.html",
        "http://1zhr9wp4.shshilin.com/mobile/news/show-323436.html",
        "https://piaofang.maoyan.com/celebrity?id=28427",
        "https://baijiahao.baidu.com/s?id=1820484479709473213",
        "https://music.apple.com/cn/playlist/%E5%91%A8%E6%9D%B0%E4%BC%A6%E5%86%99%E7%9A%84%E6%AD%8C/pl.5a3e20295f804b298e507226f335e567",
        "https://www.mingrenw.cn/ziliao/3/2970.html"
    ]
}
<<<<<<<<<<<<<[output][final_out from safe_branch]<<<<<<<<<<<<<

Additional Examples

https://github.com/sherlockchou86/PyLangPipe/blob/main/tests/README.md