GPU 共享怎么选：KAI vs HAMi vs MIG
Kubernetes 上的 GPU 共享：KAI、HAMi 与 MIG 对比。
Topic
#nvidia
8 篇笔记
AI Chip Architectures
A look at AI Chip Architectures. NVIDIA, AMD, TPUs, Trainium, Groq, Cerebras.
SemiAnalysis 发布 AgentX InferenceXv3 研报，Mooncake 成为关键基础设施
SemiAnalysis 发布 AgentX InferenceXv3，用大规模真实 Agentic workload 重新审视推理系统性能。随着长上下文、多轮交互和 KV Cache 复用成为常态，Agentic Inference 正从“拼算力”走向系统级优化，而 Mooncake 也正在其中扮演越来越重要的基础设施角色。
Profiling LLM Inference with SGLang and Torch Profiler
This blog is my attempt to learn how to profile an llm served using sglang and its in built torch profiler integration that comes out of box...
Efficient Decode Context Parallelism with vLLM for Long Context Workloads
Decode Context Parallelism (DCP) in vLLM shards KV cache across GPUs by sequence dimension, enabling 3× higher throughput on long-context agentic workloads comp
LLM Inference: Past, Now, and Future — Where Does the Value Move?
LLM Inference — Past, Now, and Future: Where Does the Value Move?For the past two years, LLM inference has been one of the most contested la...
LLM Inference: Past, Now, and Future — Where Does the Value Move?
LLM Inference — Past, Now, and Future: Where Does the Value Move?For the past two years, LLM inference has been one of the most contested la...
nvml-mock
nvml-mock 📝 摘要 nvml-mock 是 NVIDIA 官方的 Kubernetes Helm Chart，通过 CGo 编译的 mock 共享库，在无物理 GPU 的集群中模拟完整的多 GPU 环境，使 、device plugin、DRA driver 等 GPU 相关 K8s 组件无需真实硬件即可运行和测试。它部署为 DaemonSet + ConfigMap + RBAC，在…