Mercury 3 and Diffusion LLMs: Where Speed Pays Off

Almost every language model you have used writes one token at a time. It predicts the next piece of text, appends it, and predicts again. That loop is why long answers feel slow, and why speed gains usually come from faster chips or smaller models.

A diffusion language model works on a different principle. It starts with a rough draft of a whole block of text and refines many positions at the same time, the way image diffusion models refine noise into a picture. Inception Labs builds its Mercury family this way, and on October 19, 2026 it released Mercury 3.

The headline number from the launch is 1,068 tokens per second. Inception measured that itself on NVIDIA hardware it did not fully specify, and as of October 20 no independent group had published a matching result. Treat it as a claim. Even if the real figure in your region is half of that, it is several times faster than the output speed most frontier APIs deliver, and that gap is large enough to change what you can build.

The other facts worth having in one place: a 1M token context window, up from 260K on the previous Mercury, and a 131,072 token output ceiling. It takes text in and gives text out, with no images or audio. It has four reasoning effort settings, called instant, low, medium and high, and it defaults to low. New accounts get 100 million free tokens.

My view: speed is a product feature, not a spec sheet number. When a response arrives in a third of a second instead of four seconds, you stop designing around a loading spinner and start designing around a conversation. That is worth understanding even if you never use this particular model.

List pricing for Mercury 3 is $0.20 per million input tokens and $0.70 per million output tokens. Inception is running a 50 percent launch discount that takes those to $0.10 and $0.35, and the launch material did not state an end date. Cached input is priced 90 percent below the input rate.

Put that next to the rest of the market. Anthropic's smallest model, which I wrote about in our piece on Claude Haiku 5.5 at $0.10 per million, charges $0.10 input and $0.50 output for prompts up to 100K tokens. At list price Mercury 3 is a bit above that on both sides, and at the launch discount it ties on input and undercuts it on output. Frontier models in the $4 and $20 range cost 20 to 30 times as much per token.

A discount with no end date is the thing to be careful about. I do not build a unit-economics model on a launch promotion. I model the list price, and if the discount sticks around, that is margin I did not plan on. Gemini 4 Argon showed this month what happens to teams who modeled the intro rate, and our Argon budget post has the worksheet.

Here is what the arithmetic looks like for a feature that handles 1 million requests a month, each with 1,500 tokens in and 300 tokens out. That is 1.5 billion input tokens and 300 million output tokens.

• Mercury 3 at list: about $300 for input plus $210 for output, so roughly $510 a month.
• A frontier model at $4 and $20: about $6,000 plus $6,000, so roughly $12,000 a month.

Those are illustrative figures, and they ignore caching, retries and reasoning tokens, which all push the real bill up. The ratio is the point. If a feature can run on the fast tier without hurting quality, the difference is not a rounding error. It is the difference between a feature you can give away on a free plan and one you have to gate.

Speed pays off where a human is waiting, or where a pipeline calls the model many times in a row. Here are the places I would look first.

Autocomplete and inline suggestions. Anything that completes text as the user types has a latency budget under 300 milliseconds. A slow model makes suggestions arrive after the user has moved on. Search-bar query rewriting, email reply drafts and code completion all live here.

Voice agents. In a phone conversation, the model's thinking time is silence on the line. People tolerate about a second of dead air, and the speech recognition and speech synthesis stages eat most of that. A faster language model gives you headroom to spend on a better voice or a retrieval step. If you are building one, our guide on how to build an AI voice agent walks through the latency budget stage by stage. Mercury 3 is text only, so it replaces the language step, not the speech stages.

Classification and routing. Deciding whether a support message is a refund request, a bug report or spam needs a short answer and a quick turnaround. Doing it with a frontier model is like hiring a surgeon to apply a bandage.

Bulk transformation. Tagging 200,000 product descriptions, normalizing addresses or cleaning transcripts. Throughput and price decide these jobs, and a model that emits a thousand tokens a second finishes the batch in an afternoon.

Agent inner loops. A coding or research agent that takes thirty steps pays the latency thirty times. Using a fast model for the cheap steps, such as reading a file or picking the next tool, and a stronger model for the planning steps, can cut the wall-clock time of a run by half or more.

I have no interest in selling you a model, and a fast tier has real limits. Know them before you route anything to it.

The company's own benchmark claim is that Mercury 3 completed 25 of 50 live Mind2Web web navigation tasks, which is 50 percent, and Inception says that beat a recent OpenAI model. Inception ran that test itself, and there was no Artificial Analysis score on launch day. Half the tasks done is a useful result for a cheap model and also a reminder that it fails half of a hard benchmark. Do not hand it a task where one failure in two is acceptable only if a human checks the output.

Specific places I would keep a stronger model in charge:

• Multi-step reasoning where an early mistake poisons everything after it, like financial calculations or contract analysis.
• Anything with legal, medical or money consequences for the end user.
• Long, precise instructions with many constraints. Smaller and faster models tend to drop the fifth rule in a list of ten.
• Tasks that need images, audio or video, because this model does not take them.

There is also a quieter risk, which is that diffusion models are new to production. The tooling around them is thinner than for ordinary models. Structured output guarantees, tool-calling formats, fine-tuning options and observability integrations all vary between providers, and you should verify each one you need rather than assuming it matches what you use today. If your app depends on strict JSON from the model, test that with 500 real inputs and count the malformed responses.

The reasoning setting matters too. At the default of low, you get speed. At high, you get a slower, more expensive answer, and the gap to a frontier model may close or may not. Test each setting on your own tasks and log the cost per accepted answer, not per call.

You do not want to pick one model for your whole product. You want a small router that sends each request to the cheapest model that handles it well. Teams that do this routinely report bill reductions in the range of 30 to 70 percent, depending on how much of their traffic is easy. That range is my experience, not a published statistic.

A simple version has three tiers.

1. Fast tier. A diffusion or small model handles classification, extraction, short rewrites and the inner steps of agents.
2. Default tier. A mid-priced model handles most user-facing generation.
3. Heavy tier. A frontier model handles planning, hard reasoning and anything flagged as high stakes.

The router can be a few lines of code that looks at the feature name, not a clever classifier. You know which endpoint is the autocomplete and which is the contract analyzer, so let the endpoint choose the tier. Add a fallback so that if the fast tier returns malformed output or a low-confidence score, the same request is retried one tier up. We cover patterns for this in our guide to AI model routing and LLM cost optimization.

Two engineering details save pain later. First, put every model call behind your own wrapper, with the provider and model name in config. Second, log the tier, token counts, latency and a pass or fail signal for each call. Without those logs you cannot tell whether the cheap tier is quietly degrading quality.

For a new vendor like Inception, add a failover rule as well. A smaller company can have a bad week, and a feature on the critical path needs an automatic fallback to another provider. A fast model that is down is slower than a slow one.

You can answer the question of whether a fast tier fits your product in a week. The 100 million free tokens for new accounts mean the test costs you engineering time and nothing else.

1. Monday. Pick one feature. Choose the one where latency or volume hurts most today. Not three features. One.
2. Tuesday. Build the dataset. Pull 200 real inputs, with your current model's output and a note on whether it was accepted. Remove personal data.
3. Wednesday. Replay on each reasoning setting. Run all 200 through instant, low, medium and high. Record time to first token, total time, tokens and the answer.
4. Thursday. Grade. Have someone who knows the product score the outputs without seeing which model wrote them. Count malformed or truncated responses separately.
5. Friday. Decide. Compare cost per accepted answer and p95 latency against your current model. Write down what you would need to see to move the feature.

If the fast tier accepts 90 percent as often as your current model at a tenth of the cost, route that feature to it with a fallback. If it lands at 70 percent, find out which input types fail. Often the failures cluster in one category you can send up a tier while the rest stays cheap.

Write the result down even if the answer is no. In six months, when a faster tier from another vendor ships, you will rerun the same test in a day because the dataset and wrapper already exist. That reuse is the real return on the first week.

If you have not shipped yet, the practical lesson is narrow. Do not hard-code one model and one latency assumption into your product design. The cost and speed of language models are moving fast enough that a feature that was too slow or too expensive in the spring may be viable by winter, and the reverse can also happen when a vendor retires a model or doubles a price.

Concretely, scope your MVP so that the AI features can run on a cheaper tier if the unit economics demand it, and so that you can raise the tier for specific paths if quality demands it. Keep prompts in files you can edit, not buried in code. Keep a short list of the features where latency is part of the experience, since those are the ones a fast tier changes most.

If you are a post-launch team with a meaningful model bill, the sequence is different. Find your three highest-volume calls, check how many of them are easy, and test the fast tier on those first. Most teams discover that a large share of their spend goes to calls a small model could handle.

And if you only take one thing from this post, take this: the model with the best benchmark score is rarely the one that makes your product feel best. The one that answers in time, at a price you can afford at 10 times your current volume, often is. If you want help picking a tier setup for your product or scoping the routing layer, book a free strategy call and we will work through your numbers.