GPUStack v2 has been completely redesigned with a focus on high-performance inference and production-grade stability. The architecture now features flexible component decoupling and deep optimization for multiple inference engines and heterogeneous computing resources, fully unleasshing the performance potential of inference engines in throughput, latency, and concurrency.
This architectural design prioritizes native Linux environments to leverage their mature ecosystem and AI infrastructure advantages.
For Windows users, GPUStack v2 is recommended to run through WSL2 (Windows Subsystem for Linux), creating an AI runtime environment on Windows that closely matches Linux production setups while fully benefiting from v2's performance optimizations and architectural improvements.
Key Configuration Points:
- Driver Installation: Install NVIDIA drivers on the Windows host system, not within WSL
- CUDA Environment: No CUDA Toolkit required (neither Windows nor WSL)
- Container Environment: Install native Docker Engine within WSL, avoid Docker Desktop
- Network Access: Configure network mode as "Mirrored" in WSL Settings for LAN access
- Environment Detection: Use default user for environment checks, not root user
- IP Configuration: Use WSL's actual IP address for Worker configuration
Environment Configuration
System Requirements
- Operating System: Windows 10/11
- Hardware: NVIDIA GPU with WSL2 support
- Virtualization: Enable virtualization in BIOS/UEFI settings
Architecture Considerations
Driver Management
Install NVIDIA graphics drivers on the Windows host system only. WSL automatically accesses host drivers through passthrough mechanisms. Installing NVIDIA drivers or CUDA Toolkit within WSL Linux distributions may disrupt the existing environment.
Container Runtime
Use native Docker Engine rather than Docker Desktop integration. Docker Desktop employs a different network model that can cause communication failures between GPUStack components.
If Docker Desktop is already installed, disable integration with the target subsystem through Docker Desktop settings.
Setup Procedure
Step 1: Install NVIDIA Drivers (Windows)
Download and install the latest Windows-compatible drivers from NVIDIA's official website based on your GPU model.
Step 2: Install/Update WSL (Windows)
Open PowerShell or Command Prompt as administrator and execute:
# Install WSL with default Ubuntu distribution
wsl --install
# Update WSL to latest kernel
wsl --update
System restart may be required after initial WSL installation.
Step 3: Install Docker Engine (WSL)
Access WSL Linux terminal and install Docker Engine following official documentation.
Step 4: Install NVIDIA Container Toolkit (WSL)
After installation, configure with:
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker
Network Configuration (Optional)
By default, WSL uses NAT mode, preventing direct access to services running in WSL from other devices on the local network.
For LAN accessibility, configure WSL network mode as "Mirrored" using WSL Settings.
Deploying GPUStack Server
Once environment preparation is complete, WSL functions as a standard Linux environment.
Follow GPUStack official documentation for installation procedures.
Worker Configuration Guidelines
User Context for Script Execution
When running Worker validation scripts, use the WSL default user account, not root.
This is necessary because nvidia-smi command paths may not be included in root user's environment variables within WSL, while the default user can directly invoke these commands.
IP Address Specification
Configure Worker IP addresses using WSL's actual IP address (e.g., 172.x.x.x), which can be checked using ip addr command.
Using 127.0.0.1 may cause Worker addition failures.
Troubleshooting and Considerations
- Restart Requirements: System restart typically needed after Windows driver and WSL installation; Docker service restart required after NVIDIA Container Toolkit installation
- WSL Limitations: While NVIDIA's WSL support is comprehensive, rare compatibility differences may occur in specific scenarios
- Component Roles: This guide primarily addresses GPU compute resource provisioning (Worker). GPUStack Server has minimal GPU dependency and offers flexible deployment options