In the fiercely competitive field of global multimodal document parsing, Baidu's PaddleOCR-VL model has made a significant impact. This lightweight multimodal model, with only 0.9B parameters, not only topped the OmniDocBench V1.5 leaderboard with a comprehensive score of 92.6 but also achieved state-of-the-art (SOTA) results in four core capabilities: text recognition, formula recognition, table understanding, and reading order, setting a new performance benchmark in document parsing.

At the core of PaddleOCR-VL is PaddleOCR-VL-0.9B, which innovatively combines a NaViT-style dynamic resolution visual encoder with the lightweight ERNIE-4.5-0.3B language model, balancing structural understanding with resource efficiency. It can accurately parse multi-column newspapers, nested tables, and mathematical formulas, and intelligently restore document reading order, demonstrating near-human comprehension in complex real-world scenarios. Supporting 109 languages, it is one of the most flexible and powerful document parsing models currently available.

How can you efficiently and stably deploy this SOTA model in a local or cluster environment? This article will demonstrate how to use the GPUStack platform to perform inference deployment of the PaddleOCR-VL model in a local environment and showcase its performance on real document parsing tasks.

Deploying GPUStack

First, follow the official GPUStack documentation to complete the installation (**https://docs.gpustack.ai/latest/insatllation/nvidia-cuda/online-installation/**). A containerized deployment is recommended. On an NVIDIA GPU server, install the corresponding version of the NVIDIA driver, Docker, and NVIDIA Container Toolkit as required by the documentation, then start the GPUStack service using Docker.

The following tests were performed on an NVIDIA RTX4090 GPU:

Check that the NVIDIA driver and NVIDIA Container Toolkit are correctly installed and configured:

nvidia-smi >/dev/null 2>&1 && echo "NVIDIA driver OK" || (echo "NVIDIA driver issue"; exit 1) && docker info 2>/dev/null | grep -q "Default Runtime: nvidia" && echo "NVIDIA Container Toolkit OK" || (echo "NVIDIA Container Toolkit not configured"; exit 1)

Deploy GPUStack:

docker run -d --name gpustack \
    --restart=unless-stopped \
    --gpus all \
    --network=host \
    --ipc=host \
    -v gpustack-data:/var/lib/gpustack \
    gpustack-registry.cn-hangzhou.cr.aliyuncs.com/gpustack/gpustack:v0.7.1-paddle-ocr \
    --disable-rpc-servers

Check the container logs to confirm GPUStack is running normally:

docker logs -f gpustack

If the container logs indicate the service started normally, use the following command to retrieve the initial login password for the GPUStack console:

docker exec -it gpustack cat /var/lib/gpustack/initial_admin_password

GPUStack performs compatibility checks when deploying models. The PaddleOCR-VL model architecture is not yet supported in the official GPUStack release, so a custom vLLM version is needed to bypass the architecture check. Execute the following command to create a symbolic link for the custom vLLM version in the /var/lib/gpustack/bin/ directory:

docker exec -it gpustack ln -sf /usr/local/bin/vllm /var/lib/gpustack/bin/vllm_paddle_ocr

Access the GPUStack console via the server IP and port 80 in a browser (http://YOUR_HOST_IP). Log in with the default username admin and the initial password obtained in the previous step. After logging in, check the node's GPU resources in the Resources menu:

Downloading the PaddleOCR-VL Model

In the GPUStack UI, navigate to Resources > Model Files, and click Add Model File. You can search and download the PaddleOCR-VL model from Hugging Face or ModelScope. For users in China, downloading from ModelScope is recommended:

Wait for the model download to complete:

Deploying the PaddleOCR-VL Model

Once the download is complete, click the Deploy button next to the model file to deploy the model. Since the PaddleOCR-VL-0.9B model is located in the PaddleOCR-VL-0.9B subdirectory within the downloaded model directory, you need to adjust the path to point to this subdirectory for deployment.

In the model path input field, append the full subpath:

/var/lib/gpustack/cache/model_scope/PaddlePaddle/PaddleOCR-VL/PaddleOCR-VL-0.9B

Expand the Advanced Configuration section. Set the model category to LLM. In the backend version field, enter the custom backend version as paddle_ocr (pointing to the symbolic link /var/lib/gpustack/bin/vllm_paddle_ocr created earlier). GPUStack will automatically invoke the model for execution.

In the backend parameters, set the following startup parameters:

• --trust-remote-code (compatibility check requires trusting custom code execution)
• --max-model-len=32768 (set context size)
• --chat-template=/opt/templates/chat_template.jinja (set the Chat Template required by this model)

By default, vLLM uses 90% of the GPU memory. If you need to deploy multiple models, use the --gpu-memory-utilization parameter to manually control the GPU memory allocation ratio for the model (range 0-1):

Save the deployment and wait for the model to start. Confirm the model is running normally (status shows Running):

You can then test the model in the playground. For more usage details, refer to the official PaddleOCR-VL documentation:

https://www.paddleocr.ai/latest/en/version3.x/pipeline_usage/PaddleOCR-VL.html

OCR Performance Testing

Note: To ensure optimal test results, follow the official PaddleOCR-VL documentation and set Temperature to 0.1 and Top P to 0.95.

Comprehensive Document Parsing

Text

Tables

Formulas

Charts

Amid the rapid evolution of large models and inference technologies, new models are constantly emerging, and inference frameworks and technical approaches are becoming increasingly diverse. Different models often require specific backends, configurations, or dependencies. Quickly integrating and efficiently running these models without sacrificing flexibility and performance has become a key challenge in building AI infrastructure.

Against this backdrop, GPUStack is accelerating its iteration. The next version will introduce a pluggable backend mechanism and a general API proxy, allowing users to freely customize any inference engine and runtime configuration, flexibly building an MaaS platform like building blocks. Whether for cutting-edge large models or traditional machine learning models, they can be quickly loaded and seamlessly invoked on the same platform.

Empowering every developer and every enterprise to keep up with the rapid changes in large model technology and bring innovation to real-world scenarios more easily. Let us witness the next transformation of GPUStack together!

Join the Community

To learn more about GPUStack, visit our repository: https://github.com/gpustack/gpustack. If you have any suggestions for GPUStack, feel free to submit a GitHub issue. Before experiencing GPUStack or submitting an issue, please star our GitHub repository ⭐️ to follow us. Everyone is welcome to participate in this open-source project!

If you encounter any issues during use, feel free to join our Discord community: [ https://discord.gg/VXYJzuaqwD ]. Alternatively, you can scan the QR code to add the GPUStack Assistant and join the GPUStack WeChat discussion group for technical support from the GPUStack team or to exchange ideas with community enthusiasts.

If this article is helpful, please like, share, and follow.