Skip to content
← All posts

Blog2 min read

Lossless Is a Choice

A competing stack bought its speed with approximate kernels and skipped steps. Reading it closely turned into three PRs that take about a fifth off our LTX-2.3 time on B200, two of them byte-identical end to end.

We spent a while reading another framework's implementation closely, because its published numbers were better than ours and we wanted to know why.

The answer was that a good deal of the speed was bought. The stack uses agents to build many Triton fused kernels based on approximate computation, and layers a number of sparsification tricks on top: caching results, skipping certain steps. Each of these is a legitimate technique. Together, and ungated, they noticeably hurt accuracy: with a slightly more sensitive prompt, the gap between their output and ours becomes clear.

This is worth naming as a general hazard rather than as a complaint about one project. When an agent loop is scored on latency and the correctness check is loose, approximation is the path of least resistance, and the loop will find it. It is reward hacking with a plausible engineering story attached, which is the hardest kind to catch.

When the check is loose, approximation is the path of least resistance.

What we did with it ​

Rather than match the trade, we sorted the opportunities in that analysis into the ones that change the numbers and the ones that do not, and kept only the second kind.

Sorting the speedups
same output as the reference?KDA-Pilot → PR■ approximate · cached · skipped■ lossless

Swipe the diagram sideways to see all of it →

1/0 Every speedup idea the analysis turned up.

Three pull requests came out of it, each measured on its own against main, on LTX-2.3 on B200:

  • #29708 fuses the Q/K RMSNorm and split RoPE into one CUDA kernel, from a KDA-Pilot task whose name ends in __bitwise;
  • #29390 computes the LTX-2.3 Ada values once per block, a fusion it adapts from NVlabs/Sana's sol-engine, as its description credits; and
  • #29361 adds a CUDA fast path, from KDA-Pilot, for the residual-gate update that runs all through the transformer.
LTX-2.3 · B200 · end-to-end seconds per video
#29708QK norm + split RoPE · HQ
17.02 s
15.46 s
#29390fused Ada values · HQ
46.86 s
42.18 s
#29361residual-gate add · HQ
46.64 s
45.20 s
Each pair is that pull request's own measurement. #29708 was timed on a shorter HQ run than the other two, so compare within a pair, not across. Switch the baseline to read each as a fraction of main.

They save 9.2%, 10.0% and 3.1% of the time. Measured one at a time, they compose to about 21% less time, or 1.26× the speed, which is where the "more than 20%" we first reported comes from.That is arithmetic on three separate measurements, not one run with all three applied.

How lossless is lossless ​

Each pull request checks the decoded video against main, frame by frame.

Byte-identical video

2 of 3

#29708 and #29390: SSIM 1.000 and PSNR infinite. #29708 is also torch.equal on every Q and K tensor.

Close, not identical: #29361

41.3 dB

PSNR 41.3 dB and SSIM 0.985 against main.

So the honest version of our first claim is narrower: two of the three are lossless all the way to the pixels, and the third is close but not bit-for-bit.We first wrote "no accuracy loss at all". #29361's own video check says otherwise, so this section replaces that claim. Where on the scale it sits:

Decoded video vs. main · PSNR
  1. 125 dB the later bar for the opt-in fast tier
  2. 241.3 dB #29361 against main
  3. 3→ ∞ #29708 and #29390: identical
At this PSNR, the output isnot bit-identical, but above the bar

The residual-gate kernel is also the one with the most to show at kernel level. Against the Triton path it replaces, it wins on every shape it was tuned for; against eager PyTorch, it wins on the large shapes and is slightly slower on the two smallest.

#29361 · residual-gate add · B200 · microseconds
LTX-2 broadcast32640 × 4096
3.12×
1.00×
0.90×
LTX-2 full gate8160 × 4096
1.39×
1.00×
0.88×
Ideogram 4 broadcast4096 × 4608
1.85×
1.00×
0.57×
FLUX.2 broadcast4608 × 3072
1.38×
1.00×
0.45×
FLUX.2 broadcast4096 × 3072
1.20×
1.00×
0.45×
FLUX.2 broadcast512 × 3072
0.39×
1.00×
0.45×
LTX-2 full gate126 × 2048
0.38×
1.00×
0.39×
Time per call; lower is faster. Against a baseline, each bar is a fraction of that baseline's time.

Where a deviation genuinely is worth having, it should be something the caller asks for. That is what the quality tier turned into a month later.

KDA · KDA-Pilot

Read next

News6 KDA PRs merged into SGLangSince August, SGLang has merged six pull requests carrying kernels our agents wrote, the best adding 8.7% end-to-end throughput. The MLSys contest kernels are public with a benchmark anyone can rerun. One merge elsewhere, Flash-KMeans, was reverted the next day.Read it BlogKDA²: Kernel Design Agents (KDA) Optimize Kimi Delta Attention (KDA)Our agents wrote Kimi Delta Attention kernels that run up to 2.96× faster than FlashKDA on B300 with a tenth of its state error. Here is how, and how the agents tried to cheat along the way.Read it News6.5× faster MSA prefill indexerOn B300, KDA found the hardware the MSA prefill and decode indexers were leaving idle: a 6.5× geometric mean on prefill, up to 14× on chunk tails, up to 3.3× on decode, and bitwise-identical output throughout. Humanfia-reported.Read it