Same Model, Double the Price: Cracking Open the LLM Inference Pricing Black Box

Search for Kimi K2.7 Code on OpenRouter and 14 providers come up. Among third-party standard tiers, DeepInfra lists $0.74/$3.50 (input/output per 1M tokens), Together lists $0.95/$4.00 — a 28% gap on input alone. It's the same model, the same weights, and benchmark scores are nearly identical. Why such a price difference?

This isn't unique to Kimi. DeepSeek V4 Pro, GLM 5.2, and Kimi K2.6 all look the same: one model, a dozen to three dozen providers selling it, and list prices that differ by more than 2x.

We surveyed public pricing and performance data from 75 providers on OpenRouter to dissect where the gap comes from.

First, Two Prices: List Price vs. Effective Pricing

Every provider on OpenRouter has two prices for each model.

List price is the input/output unit price the provider publishes — the table you see when you open a model's detail page. For GLM 5.2, DeepInfra's list price is $0.93/$3.00; Wafer Fast is $3.00/$10.25.

Effective pricing is OpenRouter's 30-day volume-weighted average, already net of prompt-caching discounts — the real amount users pay.

The gap between the two can be enormous. DeepSeek V4 Pro's average list price is $0.435/$0.87; its average effective price is $0.225/$1.87. Input is 48% cheaper, while output is 115% more expensive. Behind this reversal is the prompt-caching hit rate: DeepSeek's official endpoint reaches an 86.2% cache hit rate, pushing effective input cost down to $0.063. Output doesn't enjoy the cache discount, and a large share of traffic flows to third-party providers with higher output quotes, pulling up the weighted output average.

So to understand price differences, you need to look at both list price and effective pricing. List price reflects a provider's "pricing strategy"; effective pricing reflects the user's actual cost experience.

Four Sources of Price Variance

Spread the data from 75 providers and price differences come down to four variables.

1. GPU Cost: Hardware Procurement Sets the Floor Price

The biggest chunk of inference cost is GPU. The same model running on different GPUs can have per-token costs that differ by several multiples. Providers' published GPU instance prices make the gap directly visible:

For the same H100, Fireworks's unit price is 3.2x DeepInfra's. This directly determines that Fireworks's list price for DeepSeek V4 Pro ($1.74/$3.48) is 1.3x DeepInfra's ($1.30/$2.60). Most of the GPU cost difference is passed through to token prices.

NovitaAI takes a different route: consumer-grade GPUs (RTX 4090/5090) in place of datacenter cards. Its RTX 4090 on-demand is just $0.33/hr, spot instances $0.17/hr — about 1/10 the H100 price. This is the cost basis that lets NovitaAI become OpenRouter's #1 provider by traffic (1.3T tokens/day): cheap inference built on consumer GPUs. The trade-off is stability and throughput ceiling: the RTX 4090 has only 24GB of VRAM, so running hundred-billion-parameter models requires more cards for tensor parallelism, raising latency and failure rates.

The type of underlying infrastructure also shapes the cost structure:

• Self-built clusters (DeepSeek, Z.ai, Moonshot, Alibaba Cloud): heavy upfront capex, but controllable marginal cost, allowing the lowest prices on the network. DeepSeek V4 Pro's official output is $0.87, 40% cheaper than the cheapest third party (StreamLake at $1.496).
• Third-party inference clouds (Together, Fireworks, DeepInfra): none of the three build models — they only sell inference — but how they hold the underlying hardware differs substantially. DeepInfra takes the most asset-heavy route, buying B300/H100s and operating them in its own US datacenters; its site states plainly "Our Hardware. Our Data Centers," which is how it gets H100 down to $2.20/hr. Together runs a hybrid model, leasing some capacity from third-party datacenters like Crusoe — $3.99/hr reflects a mid-range wholesale price. Fireworks is closer to a cross-cloud orchestration layer, scheduling compute through AWS/GCP and partners, with the highest unit price (H100 $7.00/hr).
• Aggregated consumer compute (NovitaAI, io.net): pools scattered consumer GPUs for the lowest unit price but lower consistency. NovitaAI also lets users supply their own GPUs for rewards, crowdsourcing the supply side too.
• Optimization-stack resellers (Decart, ModelRun): neither builds models nor sells GPUs — they compress unit cost with self-developed inference-optimization engines and resell via API. Decart is backed by NVIDIA and its optimization stack spans NVIDIA/Trainium/TPU.

2. Inference Engine: Software Optimization Widens the Throughput Gap

Same GPU, same model — throughput can differ by 5-10x. This is where providers genuinely compete on technology: the gap that comes from the inference engine.

There are four or five mainstream inference engines: vLLM (open source, most widely used), SGLang (open source, optimized for long sequences and multi-turn dialogue), TensorRT-LLM (NVIDIA's official offering, closed source but high performance), TGI (HuggingFace, easy to use but middling performance), plus various self-developed engines. The core optimization techniques in these engines determine how many tokens can be produced per unit of time:

• Continuous batching: dynamically stitches tokens from different requests into batches, lifting GPU utilization from 30% to 80%+. Nearly every modern engine supports it, but implementation quality varies.
• PagedAttention / KV cache management: treats VRAM like virtual memory to reduce fragmentation. This is vLLM's core contribution, directly doubling the number of concurrent requests a server can handle.
• Quantization: compresses model weights from FP16 to INT8 or lower, halving VRAM usage and raising throughput at some cost to precision. Providers usually don't disclose their quantization strategy, but benchmark scores hint at it: aggressive quantizers drop 1-3 points on GPQA.
• Speculative decoding: a small model guesses first, the large model only verifies — throughput can rise 2-3x. Most effective in latency-sensitive scenarios.

ModelRun is a textbook case of software widening the gap. On Kimi K2.7 Code it achieves 211 tok/s throughput at 0.28s latency; the runner-up Together is 137 tok/s. Running the same Kimi K2.7 Code, ModelRun's per-token GPU cost is roughly 1/3 of Together's. Its list price ($0.92/$4.00) isn't the lowest, but the performance-to-price ratio is exceptional. ModelRun's website is inaccessible and its stack is entirely undisclosed; inferring from the performance data, it most likely runs a self-developed inference engine with an aggressive batching strategy.

Fireworks publicly claims its inference engine delivers 250% higher throughput and 50% lower latency than open-source engines, and prices accordingly. It offers a "Fast" high-speed tier (Fireworks Fast): GLM 5.2's Fast version is $2.10/$6.60, 1.5x the standard tier. Wafer has a similar Fast tier at $3.00/$10.25. This is a transparent "pay more for performance" strategy.

Together's path is a self-developed engine plus a full-stack product line. Its Dedicated Inference claims 31% higher TPS than open-source engines on coding-agent workloads, and it offers a complete product line from Serverless to PTU (provisioned throughput) to Dedicated GPU to training clusters — covering everything from experimentation to large-scale production. Product-line completeness is why it consistently sits in the mid-to-high pricing tier across multiple models.

3. Prompt Caching: The Core Variable in Actual Cost

Same provider, same model — the actual payment can be 60-80% below list price, and prompt caching is entirely responsible.

The mechanism of prompt caching: if multiple requests share the same prompt prefix (say, all carrying the same system prompt or identical context), the provider stores the KV cache for that prefix, and subsequent requests pay only a very low "cache read" price — no recomputation needed. DeepSeek's official cache read price is $0.003625/M tokens, 120x cheaper than a cache miss ($0.435).

Cache hit rates vary wildly across providers, directly determining effective pricing:

With an 86% cache hit rate, DeepSeek's official endpoint compresses actual input cost to 14% of the list price. Baidu Qianfan's cache hit is just 9.7%. Its list price ($0.76) looks half of NovitaAI's ($1.57), but its effective price ($0.693 vs. $0.351) is in fact double NovitaAI's.

Cache hit rate depends on two things: whether the provider has implemented caching (a technical capability), and whether the user's request pattern is cache-friendly (a workload trait). Coding-agent scenarios are naturally suited to caching — each turn carries the full code context, so prefixes are highly repetitive. This is why cache hit rates across providers on Kimi K2.7 Code are commonly 80%+, while general chat workloads see lower hit rates.

A direct case in point: Moonshot AI Highspeed lists at $1.90/$8.00, the most expensive provider for K2.7 Code. But its cache hit rate reaches 95.9%, pushing effective input down to $0.443 — 77% below list price, and even lower than the effective pricing of many "cheap" providers. It runs a "high list price + high cache hit" playbook, earning from users with high-frequency repetitive requests.

4. Pricing Strategy: Low-Price High-Volume vs. High-Price High-Margin

Providers with similar cost structures can still price differently because their strategies differ. Pricing choices fall roughly into three categories.

Low-price, high-volume: prices are pressed near the cost line, relying on scale to amortize fixed costs. DeepSeek's official endpoint is the archetype. Backed by High-Flyer's capital, it doesn't treat API revenue as its primary profit goal; its V4 Pro official output at $0.87 is the lowest on the network, directly capturing 43.5% of the token share. DeepInfra takes a similar line — 85 models, the most in the industry, and H100 prices at the low end of the network, winning cross-model traffic through thin margins and high volume. Both earn on turnover, not unit price.

High-price, service-oriented: they sell more than tokens — stability, compliance, latency guarantees, enterprise support. Together (SOC 2/HIPAA compliance), Fireworks (ZDR/Audit Logs/SSO), and Weights & Biases (MLOps integration on CoreWeave infrastructure) all price 30-50% above DeepInfra, but their customers are enterprises willing to pay for compliance credentials and SLAs. These providers' pricing anchor isn't GPU cost — it's "the total cost of building equivalent capability yourself."

Fast-tier premium: the same model is split into a standard version and a fast version, with the fast version priced 50-100% higher. Fireworks Fast, Wafer Fast, and Moonshot Highspeed all do this. The justification is lower latency and higher throughput, but from the data the premium often exceeds the performance gain. GLM 5.2's Fireworks Fast ($2.10/$6.60) costs 50% more than the standard version, but throughput only goes from 38 tps to 60 tps (+58%), and latency actually drops from 1.72s to 1.02s. Unless the business is extremely latency-sensitive, the fast tier's value-for-money is questionable.

Official Endpoint vs. Third Party: A Traffic Contest

With the sources of price variance understood, look at a structural pattern: except for DeepSeek, almost no model's official endpoint leads in traffic.

DeepSeek is the only model whose official endpoint firmly holds #1 traffic. The reasons were covered above: lowest list price on the network, highest cache hit rate, and an effective price no third party can beat. Under the "aggressive price-cutter" strategy, the official endpoint swallows 43.5% of share outright; add StreamLake (25.8%) and NovitaAI (19.3%), and the top three hold 89%.

GLM 5.2 is the opposite extreme. Z.ai's official endpoint takes only 6.8%, with traffic nearly split between NovitaAI (36.6%) and Alibaba Cloud Int. (35.1%). Z.ai's list price ($1.40/$4.40) sits only mid-pack among GLM 5.2 providers, with high latency (5.64s), middling throughput (23 tps), and despite an 89.2% cache hit rate, its overall value proposition can't match NovitaAI's (30% off at $0.98/$3.08, cache hit 81.7%). With no decisive advantage on either performance or price, the official endpoint's traffic naturally flows to third parties.

The contrast between the two Kimi generations is also telling. K2.6's Moonshot official took 16.2% (#2 traffic); K2.7 Code took only 2.9% (#6 traffic). K2.7 Code is a new model released on 2026-06-12, and official capacity is still ramping up — traffic has been seized by Ambient (50%) and Together (10.2%). In the early days of a new model's launch, when official-endpoint capacity can't keep up, third parties step in. This pattern repeats across both Kimi generations.

Five Provider Business Models

Group the 75 providers by business model and several clear patterns emerge.

Model developers (official endpoints): DeepSeek, Z.ai, Moonshot, Anthropic, OpenAI. They sell only their own models and hold pricing power. The upside is the ability to set the lowest price on the network (DeepSeek); the downside is slow capacity ramp-up, leaving new-model traffic vulnerable in the early days.

Cross-ecosystem aggregators: NovitaAI, DeepInfra, SiliconFlow, Parasail. They host dozens to hundreds of models, covering DeepSeek + Qwen + GLM + Kimi + Llama + Gemini, capturing traffic on "everything's here + it's cheap." NovitaAI is the traffic king (1.3T tokens/day, #1 on OpenRouter); DeepInfra has the most models (85). These providers' moat is scale: more models and more traffic mean lower per-GPU cost.

High-performance inference specialists: ModelRun, Decart, Fireworks. Few models (2-7), but they push throughput to the extreme through engine optimization. ModelRun hits 211 t/s on Kimi K2.7 Code with a dominant lead; Decart took 32.4% of Kimi K2.6 traffic. Their moat is technology: a self-developed engine's throughput advantage translates directly into a price advantage.

Hyperscaler clouds: Alibaba Cloud, Amazon Bedrock, Google Vertex, Azure. The most complete product lines (IaaS + PaaS + AI), the deepest ties to enterprise customers and compliance regimes, but not the lowest pricing tier.

MLOps extensions: Weights & Biases. The main business is experiment tracking; Inference is a new offering running on CoreWeave GPUs. The selling point isn't price (same tier as Together/Fireworks) but built-in tracking and evaluation for inference — suited to teams that need observability.

How to Choose a Provider

Based on the survey data, a few decision rules for selection:

Chasing the lowest cost: look at cache hit rate, not list price. For DeepSeek V4 Pro, pick DeepSeek official (effective $0.063/$0.87, cache hit 86%); for Kimi K2.6, pick Decart ($0.272/$3.41, cache hit 75%); for Kimi K2.7 Code, pick DeepInfra ($0.223/$3.50, cache hit 87.7%); for GLM 5.2, pick NovitaAI ($0.331/$3.20, cache hit 81.7%).

Chasing extreme speed: for the Kimi family, pick ModelRun (K2.7 Code at 211 t/s, K2.6 also top three). For general models, pick Together or Weights & Biases (stable throughput, low latency). Note that high-speed providers typically list 20-30% higher — worth it only when latency directly affects the business (a real-time coding assistant, a chatbot).

Chasing stability and reliability: look at uptime. Avoid Fireworks (31.71% uptime on DeepSeek V4 Pro — clearly anomalous), Cloudflare (13.63s latency on GLM 5.2), DigitalOcean (76% uptime on K2.6). DeepInfra sits near 100% uptime across multiple models — one of the few "stable + cheap" combinations.

Avoiding traps: the provider with the lowest list price isn't necessarily the cheapest in practice. Baidu Qianfan's DeepSeek V4 Pro lists at $0.76 and looks cheap, but with only a 9.7% cache hit, its effective price is $0.693 — double NovitaAI's (list $1.57, effective $0.351). Before choosing, look at effective pricing, not list price.

A Few Judgments

From the data above, several takeaways emerge:

First, price differences are not random — they are the superposition of cost structure and strategic choice. GPU procurement sets the floor; inference-engine optimization sets the ceiling; prompt caching determines how far effective pricing deviates from list price; and pricing strategy fixes where a provider stands between "low-price high-volume" and "high-price service." The same model priced differently across providers is, in essence, a different combination of these four variables.

Second, prompt caching is the biggest cost-reduction lever. Effective pricing sitting 50%+ below list price is entirely thanks to it. Providers with high cache hit rates (85%+) collect payments far below list; providers with low cache hit rates save little even with cheap list prices. When choosing a provider, cache hit rate deserves more attention than list price — yet most users never look at this number when selecting.

Third, the official endpoint's edge is weakening. Apart from DeepSeek, which holds 43.5% share through aggressive pricing, no other model's official endpoint is the traffic leader. Third-party providers — with lower GPU costs, better inference engines, and higher cache hit rates — can deliver better value than the official endpoint in most scenarios. This trend is most pronounced on GLM 5.2: Z.ai's official takes only 6.8%, while the top two third parties hold 71%.

Fourth, the provider ecosystem is churning fast. Kimi K2.6's #1 traffic provider was Decart (32.4%); K2.7 Code's is Ambient (50%) — two completely different companies. In a new model's early days, whoever secures capacity first and optimizes the engine first seizes the early traffic. The answer to "best provider" isn't fixed; it shifts with model version, time since launch, and each provider's optimization progress.

Conclusion

Spread out the data from 75 providers and four popular models and one thing is clear: prices in this market are falling for structural reasons, not promotional ones. Prompt caching compresses actual input cost to half of list price; inference-engine optimization squeezes more tokens out of the same GPU. Both rest on genuine gains in technical efficiency, not burning cash to subsidize prices — so they won't rebound tomorrow.

At the same time, competition is intense. Except for DeepSeek, no official endpoint is the traffic leader, and provider rankings can reshuffle entirely within months (Kimi K2.6's Decart replaced by K2.7 Code's Ambient). No one locks down the market with first-mover advantage.

For users, this means choosing a provider need not be a quest for a single "forever correct" answer. Check back on effective pricing and cache hit rate every month or so — just look at the data and pick whichever offers the best cost.