Profiling LLM Inference with SGLang and Torch Profiler
This blog is my attempt to learn how to profile an llm served using sglang and its in built torch profiler integration that comes out of box...
Topic
2 篇笔记
This blog is my attempt to learn how to profile an llm served using sglang and its in built torch profiler integration that comes out of box...
Twenty-two thousand five hundred and eighty. That’s how many GPT-2 (2019) models fit inside KimiK3 (2026). We scaled up by a factor of 22,58...