Skip to content
← All news

News1 min read

2 KDA PRs merged in AMD's FlyDSL

On AMD MI350X, with thin documentation and few published kernels to copy, KDA landed two pull requests in FlyDSL: a batch-aware routing fix worth up to 8% on GQA attention, and a correct fp8 attention forward path.

Everything published about KDA so far had been on NVIDIA hardware, with excellent documentation, a mature profiler and a large public corpus of prior kernels. That is a reasonable place to start and a bad place to stop, because it leaves the obvious objection open: perhaps the agent is retrieving, not reasoning.

So we pointed it at hardware with much less to retrieve: AMD's MI350X (gfx950), through FlyDSL, AMD's Python kernel language, driven by rocm-KDA-pilot. Two of its pull requests are merged.

ROCm/FlyDSL · June 2026

Opened to merged

Swipe the plot sideways to see all of it →

#685: send each shape to the faster kernel ​

FlyDSL has two attention forward kernels on gfx950: a software-pipelined one that wins on long sequences, and a generic one that wins on very short ones. The old rule picked between them by sequence length alone. #685 makes the rule batch-aware, because the pipeline's fixed cost is paid back over batch × sequence length. The shapes it re-routes get faster:

Re-routed shapes · forward time · MI350X
GQA bf16causal · B 8 · S 192
35.1 µs
32.3 µs
GQA fp16causal · B 8 · S 192
35.1 µs
32.5 µs
GQA bf16non-causal · B 8 · S 192
34.6 µs
32.8 µs
GQA fp16non-causal · B 8 · S 192
35.1 µs
33.0 µs
Switch the baseline to read each as a fraction of the old time. MHA at the same shape is within ±2%, and every other shape keeps its old routing; 160 of 160 correctness runs pass.

#711: a correct fp8 attention forward ​

#711 adds an fp8 (e4m3fn) forward to the pipelined kernel, with three operand-precision modes. All three pass the fp8 accuracy gate, and the existing bf16 and fp16 paths are byte-identical.

It does not reach the hand-written assembly. The pull request says so: a further 19 rounds of profile-driven tuning did not beat the default path, which is held back by register-limited occupancy, not by compute.

Attention forward · B 1 · S 2048 · H 64 · D 128 · non-causal
aiter asm fp8the target · 1
1,300 TF
FlyDSL bf16reference
1,113 TF
FlyDSL fp8default · high-precision P
969 TF
FlyDSL fp8native · opt-in
721 TF
FlyDSL fp8packed P · opt-in
643 TF
  • TFLOPS
The assembly kernel is drawn at the low end of its 1,300–1,351 TFLOPS range. The default fp8 path reaches about 72–75% of it.
The same loop, less to read
KDA agentMI350Xkernel + reference checktimings & profile

Swipe the diagram sideways to see all of it →

1/0 The agent writes a FlyDSL kernel with little prior code to copy.

This is the property that decides whether any of this lasts. New accelerators arrive with no corpus at all. A loop that works only where the answers already exist on the internet is a search engine with extra steps. A loop that works on thin documentation, and says plainly where it stopped, is a method.

KDA · rocm-KDA-pilot · NVlabs/kda

Read next

News6 KDA PRs merged into SGLangSince August, SGLang has merged six pull requests carrying kernels our agents wrote, the best adding 8.7% end-to-end throughput. The MLSys contest kernels are public with a benchmark anyone can rerun. One merge elsewhere, Flash-KMeans, was reverted the next day.Read it BlogKDA²: Kernel Design Agents (KDA) Optimize Kimi Delta Attention (KDA)Our agents wrote Kimi Delta Attention kernels that run up to 2.96× faster than FlashKDA on B300 with a tenth of its state error. Here is how, and how the agents tried to cheat along the way.Read it News6.5× faster MSA prefill indexerOn B300, KDA found the hardware the MSA prefill and decode indexers were leaving idle: a 6.5× geometric mean on prefill, up to 14× on chunk tails, up to 3.3× on decode, and bitwise-identical output throughout. Humanfia-reported.Read it