英伟达优化 Kimi K3 模型推理速度 提升 1.68 倍NVIDIA optimizes Kimi K3 model for 1.68 times faster inference


AI free token/AI免费积分

IT之家 10 月 18 日消息,调研机构 SemiAnalysis 今日发文称,英伟达借助一项上游 vLLM 优化,让 2.8 万亿参数量的 Kimi K3 模型达成 423 tok/s/user(每使用者每秒每 Token)的推理速度,比较相同硬件(GB300 NVL72)表现提高 1.68 倍。

据介绍,该项优化由英伟达 CentML 工程师 Benjamin 完成。它借助大规模 Megakernel 融合,使用 Cute DSL 编写英伟达新推出的解码专用内核。该 Megakernel 包含 1.2 万行代码,代表着 AI 辅助生成高级计算内核的将来方向。英伟达优化 Kimi K3 模型推理速度,提高 1.68 倍。

英伟达优化 Kimi K3 模型推理速度 提升 1.68 倍

IT之家注意到,英伟达除此之外,还在为 vLLM 开发 DeepSeek V4.1 等其他模型的优化。

本文来源:https://www.ithome.com/1/014/595.htm
若有版权问题,请来信告知:13951431913#139.com 或在右方提交评论。

IT House October 18 news, research institute SemiAnalysis today said that Nvidia with an upstream vLLM optimization, so that 2.8 trillion parameters Kimi K3 model to achieve 423 tok/s/user (per user per second per Token) inference speed, compared to the same hardware (GB300 NVL72) performance of 1.68 times.

The optimization was reportedly done by Nvidia CentML engineer Benjamin. It uses Cute DSL to write Nvidia's new dedicated decoding kernel with massive Megakernel fusion. The Megakernel contains 12,000 lines of code, representing the future direction of AI-assisted generation of advanced computing cores. NVIDIA optimizes the inference speed of the Kimi K3 model by 1.68 times.

NVIDIA optimizes Kimi K3 model for 1.68 times faster inference

IT Home noticed that Nvidia was also developing optimizations for other models such as DeepSeek V4.1 for vLLM.

This article is from: https://www.ithome.com/1/014/595.htm For copyright issues, please write to: 13951431913#139.com or submit a review on the right.

暂无评论,快来发表第一条评论吧!No comments yet. Be the first!

📮 需求咨询📮 Inquiry