AI Chip Architectures
A look at AI Chip Architectures. NVIDIA, AMD, TPUs, Trainium, Groq, Cerebras.
Topic
#prefill-decode
2 篇笔记
Profiling LLM Inference with SGLang and Torch Profiler
This blog is my attempt to learn how to profile an llm served using sglang and its in built torch profiler integration that comes out of box...