Running Qwen2.5 Inference on Ascend 910B NPUs
Configuring the Ascend Environment
On a server equipped with Ascend NPUs, first verify the NPU driver installation:
npu-smi info
Download the appropriate CANN Toolkit (development suite) and Kernel package (CANN operators) for your architecture from the official repository. The following example uses specific version URLs for the Ascend 910B:
...
Posted on Sun, 25 Oct 2026 16:13:33 +0000 by TF@TheMoon
Deploying DeepSeek-V4 on Ascend 910B Clusters Using GPUStack: Configuration and Benchmark Analysis
The DeepSeek-V4 architecture leverages a Mixture-of-Experts (MoE) design, offering variants like the 284B Flash and 1.6T Pro models. During inference, only a fraction of parameters are activated, balancing computational overhead with output quality. The integration of expanded context windows and refined attention mechanisms improves performanc ...
Posted on Tue, 29 Sep 2026 16:35:28 +0000 by Zamees
Multi-Node Distributed Deployment of Qwen3.5-397B-A17B on Ascend 910B
While vLLM commonly relies on Ray for distributed multi-node inference, it is possible to achieve cross-node coordination without an external scheduler by combining data parallelism (DP) and tensor parallelism (TP). This article walks through a concrete deployment on two Atlas 800I A2 servers (each with 8× Ascend 910B 64 GB) using the quantized ...
Posted on Thu, 11 Jun 2026 18:34:50 +0000 by djp120