Freaks City

This article provides a detailed technical guide on implementing a Retrieval-Augmented Generation (RAG) intelligent assistant using the InternLM framework and the Huixiangdou toolset. The process involves creating a domain-specific knowledge base, deploying the assistant on cloud platforms, and integrating it with popular communication tools li ...

Large Language Models (LLMs) demand significant computational resources, primarily defined by the product of parameter count and numerical precision. To minimize memory overheadd, developers use quantization—a technique that maps high-precision weights to lower-precision formats. Taxonomy of Quantization Post-Training Quantization (PTQ): Conve ...

The llama.cpp project provides a high-performance C++ implemnetation for running Large Language Models (LLMs) like Llama 2 with minimal overhead. It is designed for efficient inference on various hardware setups, ranging from standard consumer laptops to cloud environments, without requiring heavy dependencies. Building llama.cpp from Source To ...

The ecosystem of open source tools for local LLM inference spans from command-line interfaces to full-featured desktop applications. This guide categorizes these solutions into three main categories: Integrated desktop applications Command-line and API server solutions Frontend interfaces for back end connectivity Each category presents disti ...

Managing heterogeneous accelerators for Retrieval-Augmented Generation pipelines requires a unified serving layer. GPUStack provides an open-source infrastructure to aggregate NVIDIA, Apple Silicon, Ascend, and Moore Threads hardware into a single cluster, enabling on-premise deployment of the three foundational components for RAG: conversation ...

GPUStack is an open-source platform designed to simplify the deployment and management of large language models (LLMs) across heterogeneous GPU clusters. While public cloud LLM APIs are widely accessible, organizations seeking private, on-premises LLM hosting face significant complexity in infrastructure setup, model orchestration, and resource ...

Introduction to Enterprise Knowledge Retrieval Challenges Modern enterprises often accumulate vast amounts of internal documentation, ranging from product specifications and compliance guidelines to operational procedures and technical manuals. New employees frequently encounter a deluge of information, often struggling to locate specific detai ...

Getting Started with LLM Chat Applications Developing a functional chat application using large language models (LLMs) can be achieved quickly with the right tools. Streamlit simplifies the process by allowing rapid prototyping and deployment, especially for developers less familiar with frontend development. Prerequisites Before starting, ensu ...

LangPipe is a lightweight framework designed for building applications with large language models. It provides solutions for various tasks including: Text generation Conversations with LLMs Task classification Parametre extraction Web search-based RAG SQL-based RAG Vector-based RAG Database interactions Web content conversations Knowledge base ...

Evaluation: The Gatekeeper to Production Before deploying any language model-powered application, rigorous evaluation is non-negotiable. Unlike traditional rule-based systems, LLMs produce probabilistic outputs — meaning errors are inevitable. The goal isn't perfection, but consistent alignment with user expectations under real-world conditions ...