Skip to content
← All news

News3 min read

6 KDA PRs merged into SGLang

Since August, SGLang has merged six pull requests carrying kernels our agents wrote, the best adding 8.7% end-to-end throughput. The MLSys contest kernels are public with a benchmark anyone can rerun. One merge elsewhere, Flash-KMeans, was reverted the next day.

A kernel that wins a benchmark is a claim. A kernel a maintainer merges into the engine people serve models with is something else, and that is the bar this post is about. Since August, six pull requests carrying KDA-written kernels have been merged into SGLang, all from Xiaoyu Zhang. We also count one merge elsewhere that did not stay.

Merged upstream · 2026

Six in, one out

Swipe the plot sideways to see all of it →

Into SGLang ​

Pull requestWhat it doesMeasured
#36865NVFP4 GEMM for Qwen3.x on SM1201.319× kernel geomean over 16 production shapes; Qwen3.5-4B output throughput +8.73% at concurrency 1
#38082FP8 skinny GEMM on SM120end to end +0.2% to +4.06% across five models
#41305SANA-Video residual-gate add on H2002.54× on the two real SANA-Video layouts; −1.7% end-to-end time
#41459LTX-2.3 QK norm + split RoPE on H2002.58× to 2.61× kernel geomean; −1.0% to −1.2% end-to-end time
#36845QSA decode on SM121 (GB10)a correctness fix: the old path silently corrupted long-context decode; the new kernel is 2.07× the old correct one
#37385registers the merged diffusion agent kernels under KernelBackend.KDAprovenance, so the engine records which kernels an agent wrote

Two things in that table are worth reading slowly.

A kernel speedup is not a model speedup. The two diffusion kernels are about 2.5× faster on their own and save one to two percent of a whole video generation, because they are a small part of it. Both pull requests report both numbers. Switch the chart's baseline to see each as a fraction of the other.

The same change, measured twice
#41305SANA-Video · H200
2.540×
1.017×
#41459LTX-2.3 · H200
2.590×
1.011×
#36865Qwen3.5-4B · SM120
1.319×
1.087×
Kernel: geometric mean over the pull request's workloads (#41459 is the median of its three runs). Model: SANA-Video and LTX-2.3 saved-request time; Qwen3.5-4B output throughput at concurrency 1.

#36865's gain is a decode gain, so it shows most when one request has the GPU to itself:

#36865 · NVFP4 GEMM · RTX PRO 6000 (SM120)

Output throughput gain by concurrency

Swipe the plot sideways to see all of it →

#38082 is the same idea for FP8 at small batch sizes, measured across five models. The gains are small and real, and they vary by model and batch size more than by anything else:

#38082 · FP8 skinny GEMM · end-to-end gain by batch size M
Qwen3-14B-FP8
3.13%
3.29%
3.75%
4.06%
Qwen3.8-27Bmixed ModelOpt
1.40%
0.26%
0.20%
3.40%
Qwen3-8B-FP8
1.00%
0.74%
1.00%
1.58%
Nemotron-3-Super120B-A12B
1.08%
1.19%
1.89%
0.56%
Llama-3.1-8B-FP8
0.75%
1.08%
0.97%
0.92%
Against the FlashInfer/cuBLAS fallback, with the existing GEMV path disabled for the measurement, on 2 × RTX PRO 6000.

A correctness fix can also be a speedup. On GB10 (SM121), the old QSA decode path returned token 0 for every long-context request while reporting success. #36845 replaces it with one kernel KDA wrote, which groups the twelve query heads that share a KV head into one block. Against the original correct Triton kernel, across batch sizes:

#36845 · QSA decode · GB10 · both tensor-parallel layouts

Decode speedup over the correct Triton kernel

Swipe the plot sideways to see all of it →

64 of 64 sweep cases pass, with a maximum relative L2 error of 0.0024.

A pull request can get smaller under review. #41305 began with 40 kernel families. It merged with one, after every family that showed a measured regression, or had no established end-to-end benefit, was taken out. The rule: keep a family only if end-to-end time improved by at least 1.5% in both of two measurement groups. The other 39 are listed in its selection record.

#41305 · 40 kernel families in, 1 out
merged40 families■ removed · 39■ kept · 1

Swipe the diagram sideways to see all of it →

1/0 KDA, running with Codex and Kimi K3, proposed 40 kernel families.

Three of the six (#36845, #41305 and #41459) credit the kernel to KDA running with Codex and Kimi K3.

The contest kernels, public ​

The MLSys 2026 FlashInfer contest kernels are now public (Dongyun Zou), as release KDA 0.5, with a benchmark that pins the human winners' own repositories and refuses to run if they are modified. On B200, as the geometric mean over the official workloads:

KDA 0.5 vs. the best human entry · B200
GDN prefill
1.688×
DSA attention
1.408×
MoE FP8
1.173×
  • KDA 0.5

The repository states its own caveats, and they apply here:

  • The timing is CUPTI kernel spans. The contest was scored by the organisers' wall-clock harness, so these are not contest scores.
  • The kernels are tuned to the official shapes, so do not expect the same speedups elsewhere.
  • The pinned human repositories are a few commits past the contest deadline.
  • The repository numbers its releases from 0.1, the original contest entry. Our August post first called the same two generations KDA 1.0 and 1.5, and reported B300 numbers under a different protocol; it now leads with these.

One merge that did not stay ​

On 27 August, svg-project/flash-kmeans#23 merged a KDA-generated CuTe backend for Flash-KMeans, by Dongyun Zou and Ligeng Zhu, on a real LongCat-Video workload:

Flash-KMeans · CuTe backend vs. Triton · LongCat-Video [64, 37440, 128], K = 293
B200cold · 10 iterations · 30.33 → 4.15 ms
7.31×
B200warm · 2 iterations · 6.43 → 0.92 ms
7.01×
RTX PRO 6000cold · 26.93 → 7.59 ms
3.55×
RTX PRO 6000warm · 6.41 → 1.60 ms
4.00×
  • CuTe backend (reverted)
Both backends must agree on inertia (relative difference 2e-5) before any time is reported. The backend was reverted the next day.

The next day the maintainer reverted it to keep the repository simple and asked for it to go to FlashLib instead. That has not happened yet. So, for now, it is not upstream, and we do not count it.

And the long version ​

How our agents wrote Kimi Delta Attention kernels up to 2.96× faster than FlashKDA on B300, and how they tried to cheat along the way, is in KDA²: KDA optimizes KDA.

KDA · kernel-design-agents

Read next

BlogKDA²: Kernel Design Agents (KDA) Optimize Kimi Delta Attention (KDA)Our agents wrote Kimi Delta Attention kernels that run up to 2.96× faster than FlashKDA on B300 with a tenth of its state error. Here is how, and how the agents tried to cheat along the way.Read it News6.5× faster MSA prefill indexerOn B300, KDA found the hardware the MSA prefill and decode indexers were leaving idle: a 6.5× geometric mean on prefill, up to 14× on chunk tails, up to 3.3× on decode, and bitwise-identical output throughout. Humanfia-reported.Read it BlogA Quality Tier for the Fast KernelsMost profitable diffusion fusions change the output at the level of bf16 rounding order, enough to fail a bitwise CI and not enough for a person to see. So we gave the request a quality tier instead of arguing about the diff.Read it