Notes on quantization, inference, and GPUs

HOME
CATEGORIES
TAGS
ARCHIVES
ABOUT
HARDWARE & KERNELS
INFERENCE SYSTEMS
MODEL OPTIMIZATION
DIARY

Home Categories Inference Systems

Category

Inference Systems 1

From CUDA Graph Crash to Tensor Core: Tracing a vLLM INT8 Bug Apr 27, 2026

Recently Updated

From CUDA Graph Crash to Tensor Core: Tracing a vLLM INT8 Bug
Processing-in-Memory for Decode-Stage GEMV: Mixed-Signal, Digital, and the Cost of Noise
SVDQuant: Absorbing Weight Outliers into a Low-Rank Branch for 4-Bit Diffusion Models
TwinQuant: Learnable Subspace Decomposition for W4A4 LLM Quantization
Int8Tensor: A PyTorch-Native INT8 Quantization Subclass for torchao

Trending Tags

quantization llm-inference low-rank pytorch diffusion-models hardware-accelerators

© 2026 Namgyu Youn. Some rights reserved.

Using the Chirpy theme for Jekyll.

Trending Tags

quantization llm-inference low-rank pytorch diffusion-models hardware-accelerators

New content available