The break-even on self-hosted open weights is 72 percent utilisation

On 8 August, Alibaba put 2.4 trillion parameters on Hugging Face and let anyone pull them. Three weeks later the counter on Qwen3.8-2.4T-A95B reads about 22,000 downloads. Qwen3.8-27B, released three days earlier, is past 3.4 million.

The gap is not licensing and it is not quality. It is memory. The checkpoint is 2,446,182,725,504 parameters, which at FP8 is roughly 2.4 terabytes of weights before you allocate a single byte of KV cache. An H200 carries 141GB. You need twenty-four of them, three full nodes, with an interconnect fast enough that routing each token to ten of 512 experts across a node boundary does not stall the batch.

The weights are free. Nothing downstream of the download is.

The rent is the smallest line

RunPod's on-demand board today reads $3.29 an hour for an H100 SXM on the secure tier and $2.69 on community, $4.59 and $3.59 for an H200, $6.79 and $5.98 for a B200. Those are per GPU, per hour, from a provider already well under hyperscaler rates. An average month is 730 hours.

Take gpt-oss-120b as the deployment, because it is the friendliest case anyone can construct. OpenAI shipped it in August 2025 at 116.8 billion total parameters with 5.1 billion active, MoE weights in MXFP4, sized deliberately so the whole model fits on one 80GB H100. Run two, because a single GPU is a demo and not a service. Two times $3.29 times 730 is $4,803 a month.

For throughput, use a measurement rather than a vendor blog. SemiAnalysis's InferenceX benchmark puts gpt-oss-120b on H100 at 4,148 output tokens per second per chip when each user is receiving 143 tokens per second. Push interactivity up to 197 tokens per second per user and the chip falls to 2,636. Let it drop to 90 and the chip does 6,362. Take the middle.

Two chips at 4,148 is 8,296 tokens a second. Held flat for 730 hours that is 21.8 billion output tokens for $4,803. Twenty-two cents per million.

Together AI will serve the same weights at $0.15 per million input and $0.60 per million output, with no pager rotation attached. So self-hosting wins by 2.7x. On rent. At perfect utilisation. Before anybody touches it.

They costed the entire day at peak occupancy, which is the correct way to do it, because you cannot hand a node back for six hours and expect it waiting at nine the next morning.

The engineer costs more than the GPU

Set an assumption and label it as one: a single infrastructure engineer, fully loaded at $220,000 a year, spending a quarter of their time on this. Substitute your own figure. Do not substitute zero.

That quarter costs $4,583 a month, within a few hundred dollars of the entire GPU bill. The work is version churn, mostly. Hugging Face moved its recommendation to vLLM and SGLang after TGI went into maintenance mode in December 2025, and both engines ship breaking changes at a pace that assumes you are following upstream weekly. Quantisation regressions after a kernel update. Capacity planning nobody enjoys. Cold starts at 3am when a node dies and a 60GB checkpoint has to reload before the queue drains. A new model release that upstream has not registered yet, which means you are now running somebody's fork in production.

Total: $9,386 a month. At 100 percent utilisation that is $0.43 per million output tokens, so the break-even against Together's $0.60 arrives at 72 percent sustained utilisation.

Put it in volume terms instead. That $9,386 buys 15.6 billion output tokens from the serverless endpoint. To match it you must hold roughly 6,000 output tokens a second, every second, for a month. Including 4am on Sunday.

Idle hours are what actually kill it

Nobody's traffic is flat. A cluster serving a company from 09:00 to 18:00, Monday to Friday, is live for 45 of the week's 168 hours. That is 27 percent before you account for the shape of a working day, and inside the window the load is not flat either. A peak-to-mean ratio of three to one is ordinary in an internal tool, and you size the cluster for peak or you drop requests at 11am.

Fold that in and the arithmetic inverts. At 27 percent utilisation the $0.43 becomes $1.59 per million output tokens, nearly triple what the serverless endpoint charges. At the nine percent a peak-sized cluster on a business-hours workload actually achieves, it is $4.78, eight times.

The GPU does not sleep cheaply. It bills at $3.29 an hour whether it is decoding tokens or sitting idle waiting for Monday.

This is the number that never appears in the announcement. When a company says it cut inference costs by sixty percent by moving to open weights and gives no utilisation figure, it is quoting the perfect-utilisation number and hoping nobody divides.

DeepSeek published its own arithmetic

In February 2025, DeepSeek did something almost nobody else has done and published the operating figures for its V3 and R1 inference fleet across a 24-hour window. Peak occupancy: 278 nodes of eight H800 each. Average occupancy: 226.75 nodes. Input: 608 billion tokens, 342 billion of them cache hits. Output: 168 billion tokens. Cost at an assumed $2 per H800 hour: $87,072. Theoretical revenue if every token were billed at R1 rates: $562,027. Cost-profit ratio: 545 percent.

Pull two things out of that. The first is that 168 billion output tokens across 2,224 GPUs in a day works out to 874 output tokens per second per GPU. That is what a team writing its own kernels extracts from a 671B mixture-of-experts with 37 billion active parameters. If your planning estimate was higher, revise it downward.

The second sits in the gap between 278 and 226.75. DeepSeek serves demand across every time zone on earth and applies automatic discounts at night specifically to drag load into the trough, and it still averaged 18 percent below its own peak occupancy. They costed the entire day at peak, which is the correct way to do it, because you cannot hand a node back for six hours and expect it waiting at nine the next morning.

They also said, in the same post, that actual revenue was substantially lower than the theoretical figure. Web and app access are free. Off-peak pricing applies. The 545 percent is a ceiling, published by the organisation with the strongest possible incentive to publish a flattering number.

Twenty-four H200s for a checkpoint that is text-only

Back to the 2.4 trillion parameters. Twenty-four H200s on RunPod's secure tier at $4.59 an hour each is $80,417 a month. The community tier brings it to $62,897 on machines you do not control and can be preempted from, which is not where a production endpoint lives.

Together serves those same weights at $2.50 per million input and $6.25 per million output. To match $80,417 on rent alone you need to generate 12.9 billion output tokens a month, and that is with engineering set to zero, which for a three-node expert-parallel MoE it very much is not.

Against a frontier API the comparison flips, which is exactly how these press releases get written. Claude Opus 5 is $5 per million input and $25 per million output. Sonnet 5 is $2 and $10, and the increase to $3 and $15 that was scheduled for 1 September has been cancelled. Haiku 4.5 is $1 and $5. The Batch API halves all of them. Eighty thousand dollars of Opus 5 output is 3.2 billion tokens, so if you genuinely generate more than that, the self-host wins on paper.

The catch is buried in the phrase "good enough". The open 2.4T release is text-only, with no vision encoder and without the native million-token context of the hosted version. The model you can download is not the model on the benchmark chart you were shown.

Residency and latency override the spreadsheet

Everything above assumes price is the only question. For a real set of organisations it is not the question at all.

A Nigerian bank operating under central bank data rules, a hospital group holding patient records, a ministry processing court filings: none of them are comparing $0.43 against $0.60. They are asking whether the data is permitted to leave the building. Where the answer is no, a threefold cost penalty is not a bad deal, it is the cost of being allowed to operate. Arguing utilisation economics at somebody in that position answers a question they never asked.

Latency is the second honest override. From Lagos, the nearest frontier inference region is Frankfurt or Northern Virginia, and you spend something like a tenth of a second of round trip before the model has read the prompt. In a chat completion that is invisible. In an agent making twelve sequential tool calls it is seconds of pure transit per turn, and it is part of why Nigeria is building: 17 operational data centres, somewhere between 65 and 86 megawatts installed, $10 to $15 million per megawatt to construct, Kasi Cloud's $250 million campus in Lekki, Airtel's Nxtra at $120 million targeting the first quarter of 2026. No table of token prices produces that capital plan. Something else does.

Choosing sovereignty is defensible and often correct. Claiming it also saved money is where the deck starts lying.

A price card in Huaqiangbei

Second floor of one of the market buildings off Huaqiangbei, past the phone-repair counters, a stallholder turned a laminated card around: 24,000 RMB, roughly $3,350, for an RTX 4090 with the memory rebuilt to 48GB and a blower shroud bolted over the top. Not a forum rumour. A price, on a card, with three of the cards in anti-static bags on the shelf behind him.

I asked who buys them. 私有化部署, he said. Private deployment. Small firms putting a quantised 70B on-premises because it is the only shape of the problem they can afford. Not one of those buyers ran a utilisation model and concluded self-hosting was cheaper. They walked into a wall: export controls mean an H100 is not on their menu at any price, and a modified consumer card holding 48 gigabytes is.

Huawei's Ascend line is the institutional version of the same trade, a domestic accelerator bought for availability rather than for performance per dollar. Same logic, three orders of magnitude apart in budget.

Which is the honest frame for the whole argument. Self-hosting is what you do when the API is closed to you or the data cannot travel. It is rarely what you do because the arithmetic favoured it.

Flat load, hard residency, or resale

Three groups genuinely come out ahead, and it is worth being precise about which.

The first has a flat, high, predictable batch load. Overnight document extraction. Embedding regeneration against a corpus that keeps growing. Bulk classification on a queue that never empties. These workloads can be shaped to fill the trough, which means the utilisation figure that destroys everyone else's model is the one variable they can actually control.

The second has a residency or air-gap requirement and has already priced the penalty honestly. There is no API in their comparison, so there is nothing to compare against.

The third sells inference. If you are reselling capacity, other people's traffic fills your idle hours, which is precisely why DeepSeek's numbers work and why the identical hardware, bought by one enterprise for one internal workload, sits at nine percent and loses money every month. The margin was never in the model. It is in the aggregation.

Everyone else is buying a 24/7 asset to serve a 9-to-5 problem and booking the difference as a saving. Open weights redistributed the right to inspect a model, to fine-tune it and to audit what it refuses to say. They did not redistribute the right to serve it. The licence file on the Qwen release is a few kilobytes. The checkpoint is 2.4 terabytes. Only one of those was ever the barrier.

Sources

RunPod GPU cloud pricing (H100, H200, B200 hourly rates) : https://www.runpod.io/pricing

Together AI serverless inference pricing (gpt-oss-120b, Qwen3.8-2.4T-A95B) : https://www.together.ai/pricing

Anthropic Claude API pricing documentation : https://platform.claude.com/docs/en/about-claude/pricing

DeepSeek-V3/R1 Inference System Overview, open-infra-index : https://github.com/deepseek-ai/open-infra-index/blob/main/202502OpenSourceWeek/day_6_one_more_thing_deepseekV3R1_inference_system_overview.md

Qwen/Qwen3.8-2.4T-A95B model card, Hugging Face : https://huggingface.co/Qwen/Qwen3.8-2.4T-A95B

openai/gpt-oss-120b model card, Hugging Face : https://huggingface.co/openai/gpt-oss-120b

SemiAnalysis InferenceX, gpt-oss-120b B200 vs H100 benchmark : https://inferencex.semianalysis.com/compare/gptoss-120b-b200-vs-h100

TechCabal, Nigeria's first AI data centre in 2026 : https://techcabal.com/2026/01/08/will-nigeria-get-its-first-ai-data-centre-in-2026/