Optimizing Large Language Models through Weight Quantization
Large Language Models (LLMs) demand significant computational resources, primarily defined by the product of parameter count and numerical precision. To minimize memory overheadd, developers use quantization—a technique that maps high-precision weights to lower-precision formats.
Taxonomy of Quantization
Post-Training Quantization (PTQ): Conve ...
Posted on Fri, 31 Jul 2026 16:49:00 +0000 by harinath
TVM Practical Applications in AI System Deployment
Introduction to TVM Workflow
This article explores how to deploy neural networks on new hardware using AI compilers, covering the entire process from algorithm design to actual execution. We'll use TVM (Tensor Virtual Machine) as our example, examining its workflow:
Model Import: TVM can import models from TensorFlow, PyTorch, ONNX, and other ...
Posted on Thu, 09 Jul 2026 17:52:17 +0000 by newbtophp
Extracting and Quantizing PyTorch Model Parameters and Activations
Extracting Model Parameters
import os
import torch
os.makedirs('weights', exist_ok=True)
model.load_state_dict(torch.load('model_weights.pth'))
item_counter = 0
for param_name, param_tensor in model.state_dict().items():
print(f"{param_name}: {param_tensor.shape}")
with open(f'weights/{item_counter}-{param_name}.txt', 'w') a ...
Posted on Thu, 14 May 2026 22:53:31 +0000 by Stu