Kimi K3.1 Pro vs Claude Opus 5.5: Both Built All 8 Apps, One Bill Was 4.3x the Other

Both models built all 8 apps on the first try. One cost a quarter as much; the other won the blind judge 6 times. The difference is about 84,000 more billed tokens, most of them reasoning you never see.

The two models

Both ran through the same endpoint and account inside the same hour, on the same briefs and a 40,000-token cap.

A single setting differs. Kimi ran with reasoning_effort: "low", because at its default it thinks for 13,000 to 25,000 tokens before writing a line of HTML, and these briefs want the HTML. Opus ran with no effort field set. The probe suite later in this post runs Kimi at its default, so both sides of that trade are measured.

Three published numbers

The two are within a point on terminal work, 20 points apart on science workloads and 3 points apart on the composite. Those are the published numbers. The rest of this post is what the two did with 8 identical briefs.

The harness

Every brief carries the same rules: 1 HTML file, no libraries, a 900 x 600 stage, a seeded random number generator, an FPS counter, and a scripted 10-second autoplay so the file performs on its own. That last rule is what makes the run measurable.

1. bench.py sends each brief to both models through GMI's chat completions endpoint, streamed with usage, and records tokens, time to first token, tokens per second, reasoning tokens where the model exposes them, and cost at list price.

2. render.mjs opens every file in its own headless Chrome tab with GPU rendering on, takes stills at 1.0, 3.5 and 6.5 s, records a 10-second screencast, and measures frame rate, JavaScript exceptions and pixel motion with ffmpeg.

3. score.py applies 11 wording checks, 6 shared and 5 from each brief.

4. judge.py shows Gemini 4 Argon the 3 stills of each app, with which model is App A chosen at random, and asks for a pick with a reason.

Opus cost 2.7x to 7.1x more per brief; the pool table alone cost $0.73

The pool table is the one to look at. Opus spent 36,271 output tokens and $0.73 on a 20 KB file, within a cent of what Kimi's entire set cost.

The list prices are 2 to 1. The bill came out 4.3 to 1.

Cost per clean app is the number I care about most, because a cheap failure still costs money: $0.092 for Kimi, $0.395 for Opus.

Where the other 2.15x comes from

The list price explains half the gap. The other half is in the output token counts: twice the price times 2.15x the completion tokens in total (156,741 against 72,985) is 4.3.

Opus billed 1.3x to 3.6x Kimi's completion tokens on every brief

Opus billed more completion tokens than Kimi on all 8 briefs, twice as many at the median: 16,638 against 8,148. Opus's reasoning is hidden, so those tokens are absent from the response and present on the bill. Kimi's reasoning is visible and counted too, 2,100 to 4,300 tokens per brief at low effort, and it is included in Kimi's totals above.

Opus shipped the smaller file on all 8 briefs while billing more tokens

Plot billed tokens against the size of the HTML that came back and the two models separate cleanly. Kimi's files were the larger ones on every brief; Opus's were smaller and cost more. The further a point sits to the right at the same file size, the more reasoning you paid for and did not receive.

Where the time goes

Kimi's first token lands in 3 s; Opus thinks silently for 45 s to 4 min first

Time to first token shows where that gap sits. Kimi starts streaming its reasoning within 2.8 to 3.5 seconds on every brief. Opus shows nothing for 45 seconds to 4 minutes while it thinks, then writes at 210 to 440 tokens per second and catches up. Total time per brief ends up about the same, medians of 168 s and 156 s, but the shape of the wait is different, and that matters if a person is watching the screen.

Total time per brief is about the same: medians 168 s Kimi, 156 s Opus

What the judge saw

The judge is a third model, Gemini 4 Argon. It picked Opus on 6 briefs and Kimi on 2, and its reasons line up with what the automated rule cannot see. The SPH fluid from Kimi held 60 frames per second and passed every check, and the judge's note reads "the fluid collapses into a flat 1D line along the bottom boundary." The carousel from Kimi never left card 1: the 3.5 s and 6.5 s stills are identical. On the kanban board the dragged card stayed in the bottom-right corner. Opus's pool break left the rack clustered, which the judge noted and forgave for the cue stick, the aiming guide and the English readout.

Kimi's 2 wins were polish wins. Its cloth draws structural and shear constraints, sags between the pins and moves in the wind, and the judge preferred it to Opus's stiffer grid, whose UI text overlapped the mesh. Its command palette took the other star.

The scoreboard from the video: cost per brief with the judge's pick

The 444-call probe suite

A sample of 8 briefs is small, so I ran 12 categories of short, machine-scored tasks behind them: code, bug fixes, JSON, tool calls, instruction following, counting math, facts, 3 languages, vision, a needle at up to 400K tokens, SVG drawings and short-answer speed. Kimi ran at its default effort here, so this is the other side of the setting.

On correctness these two are peers. Kimi missed 1 of 30 counting problems, 1 trial of a divisibility sum, and nothing else. The split is in time and tokens.

Short prompts: Opus answers in 1 to 20 s, Kimi in 4 s to 2 min at 35 to 45 tok/s

Opus lists 5 fruits in 1.2 s. Kimi takes 4.3 s, because at default effort it thinks about 60 tokens before even a 5-line list and 400 before a function, then writes at about 40 tokens per second against Opus's 150 to 290 on the same 4 tasks. A coding task is 6.5 s against 17.6 s. An SVG drawing is 20 s against 2 minutes, and the drawing is where Kimi won: the judge took Kimi's bicycle and Kimi's pelican on a bicycle, and Opus's clock for its minute ticks.

Tokenization was the result I did not expect. The same 1.6 MB document is 493,000 tokens on Opus's count and 271,000 on Kimi's, so the 400K-needle question cost $1.97 on Opus and $0.49 on Kimi: 4 times the money on twice the price. Kimi's price edge shows up on long inputs, and on long outputs with its effort turned down. On short prompts at default effort, Kimi's thinking costs about what Opus charges for the whole answer, and on JSON and the multilingual questions Kimi's bill was the higher one.

Physics Twin

The hardest task I gave either model was Physics Twin: take up to 48 frames of a short clip of something moving, track the object frame by frame, decide what kind of motion it is, write the fitting code and run it. The model comes back with a number after a loop of tool calls. I ran 10 clips on the same page with both models at effort medium; the model id was the only other change.

Opus finished all 10, between 51 s and 6 minutes each, and landed inside the expected range on 9. The range means within 10% of 9.81 for the 3 free-fall clips, below 3 m/s² for the 2 slow-motion clips, 3 to 8 for the incline, and no g claimed for the 4 clips where gravity cannot be recovered. Kimi finished 6, between 4.4 and 14.5 minutes; on the other 4 it stopped issuing commands twice, hit the 12-turn limit once and ran past the 900 s limit once. Where both finished, Kimi's fits were tighter: on the string pendulum Kimi read g as 9.97 ± 0.05 and Opus 10.00 ± 0.1; on the bouncing ball Kimi's error bar was a third of Opus's. On the 2 clips in the video, Kimi stopped issuing commands on the stopwatch pendulum and ran past the 900 s limit on the rail incline; Opus returned a fit in 51 and 60 seconds. The stopwatch fit, g = 10.89 ± 0.75, is its only miss of the 10.

Where to be careful

• Count tokens per model. The same text is 1.8x the tokens on Opus. Long-context bills scale with the tokenizer as much as with the price list.

• Set Kimi's effort. At default it thinks 13,000 to 25,000 tokens before a large file. At low it finished every brief for a quarter of Opus's bill. Left at default, Kimi finished 4 of the 8 briefs inside the 900 s limit, and on those Opus cost 1.0x to 1.4x Kimi; the other 4 came back without a complete file, 3 of them cut off mid-file by the limit.

• Budget for hidden reasoning. Opus billed 1.3x to 3.6x the completion tokens Kimi did for the same brief, and the wait before the first token is where that time goes. Price Opus jobs from measured completion tokens; the size of the answer undercounts them.

• One-shot is the hardest test. Both models passed it 8 times. In a pipeline with a checker, a Kimi retry costs 9 cents and an Opus retry 40; if the output ships unchecked, pay for Opus.

• Watch the loop length. On the 10 Physics Twin clips Kimi stalled on 4. Agent loops that run many turns favour the model that finishes.

A note on the numbers. Every number in this post comes from the GMI Cloud MaaS inference engine on a Tier 5 account, so your cost and wait times will vary with your account tier, queue load on the day, and the provider's own capacity. If you need faster or more predictable turnaround, a dedicated endpoint runs your model on reserved capacity and removes the queue wait.

Three takeaways

1. The 2-to-1 price gap is the one number you can trust before you run anything. The bill came out 4.3 to 1 because Opus bills every token of its hidden thinking while Kimi ran with its thinking turned down. At Kimi's default effort the briefs that finished came out at 1.0x to 1.4x. That 1 setting is what turns a 2-to-1 price gap into a 4.3-to-1 bill.

2. Pick by the brief. On the 4 physics briefs the judge took Opus 3 times, and its SPH note shows what the automated check missed. On the 4 UI briefs it took Opus 3 times again, and Kimi took its 2 wins at a quarter of the price. On SVG drawings Kimi won 2 of 3.

3. Multiply by how often you run it. 1,000 apps a day turns a 30-cent gap per app into about $300 a day. If a checker catches a collapsed fluid, Kimi plus a retry is still cheaper than 1 Opus run. If nothing checks the output, the more polished file is the cheaper one.

Where I Land

Both models cleared 8 of 8 on the first try. Kimi K3.1 Pro did it for 74 cents and beat Opus outright on the cloth and the palette. Opus 5.5 did it for $3.16 and built the more polished file 6 times out of 8, and in a long tool loop it was the one that finished. I would run Kimi, at low effort, anywhere a checker sits downstream and the volume is high, and Opus anywhere the first answer ships. Both are one string apart on GMI Cloud, so the choice can change per request.

Ready to build? Get Started