The Frontier Model Tax, Revisited: Is Fine-Tuning a 7B Still Worth It?

A fine-tuned 7B running on your own GPU feels free. There's no meter, no invoice and no per-call price, so the bill looks like zero.

That zero leaves things out. The card loses value, it can fail, it draws power, and the models worth running move faster than the hardware does. Most of us have never had a GPU die on us, so that cost never makes it into the math. A used RTX 4090 sold for about $2,800 on eBay in October. Spread over five years or three, that's about $47 to $78 a month before the first token, against about $137 a month for the same job on Claude Haiku 5.5.

In April I wrote The Frontier Model Tax. The argument was that the tax was optional once you had a stable task, 100 to 500 good examples and real volume. Six months later, the small tiers from OpenAI and Anthropic cost $0.10 per million input tokens. So I re-ran the math and priced a fine-tune the way you'd price any asset: the compute, the one-time work to build it and the upkeep. A rented H100 used to beat Claude Haiku above about 560,000 documents a month. Against Haiku 5.5 it needs about 4.3 million. A 4090 you already own still beats Haiku 5.5 on the power bill at any volume. Add the depreciation, the failure risk and the hours a fine-tune takes, and at this post's volume it pays about $7 to $15 an hour for your work.

What a local model actually costs to own

Four costs sit behind the zero on a local model's bill. Here they are for one RTX 4090 running an 8B model on the job this post uses throughout: 300,000 documents a month, 2,000 input and 300 output tokens each.

Depreciation. A GPU wears out and goes stale, so its price belongs in the monthly cost. Used 4090s averaged about $2,800 on eBay in October, up from about $2,080 a year earlier (BestValueGPU). Over a five-year to three-year life, my assumption, that's about $47 to $78 a month. Plenty of cards run well past three years, so I'm using a range. The rising resale price hides this cost right now. If you own a 4090, running it instead of selling it is a $2,800 decision, at a resale price set by a shortage you don't control.

Failure. Puget Systems, which builds workstations, reported a 0.68% GPU failure rate across 2024 (Puget Systems). Those are mostly newer cards in professionally built machines. A 4090 bought at launch is four years old now. I couldn't find a published failure rate for consumer cards that age, so I'm using 1% to 5% a year as an assumption. On a $2,800 card that's about $2 to $12 a month in expected loss. In practice it arrives all at once, as $2,800 on the day the card dies.

Electricity. In the llama.cpp CUDA benchmarks, a 4090 processes prompt tokens for a 4-bit Llama 2 7B at about 12,000 a second (llama.cpp). Database Mart measured 7B models on one 4090 in vLLM at about 3,500 output tokens a second over a 300-request run (Database Mart). The two tests used different engines and models, so I doubled the time as a cushion. NVIDIA rates the card at 450 watts (NVIDIA). The extra 150 watts for the rest of the system is my estimate. The EIA puts July 2026 residential power at 18.31 cents per kWh nationally and 33.61 in California (EIA).

• Tokens: 600 million input and 90 million output a month.
• GPU time: 600 million at 12,000 a second plus 90 million at 3,500 a second is about 21 hours. I doubled it to about 42 hours as a cushion.
• Power: about 600 watts for the card and the rest of the system, so about 25 kWh a month at the doubled time. That's about $4.60 at the US average and about $8.50 in California.
• Card: $2,800 over a five-year to three-year life is about $47 to $78 a month.
• Failure allowance: 1% to 5% of $2,800 a year, about $2 to $12 a month.

By my arithmetic the power runs $2 to $9 a month, depending on the cushion and the state. The card is busy about 3% to 6% of the month, so the job barely touches it.

Obsolescence. The 4090's 24GB fits an 8B model at 16-bit weights with room to spare, since 8 billion weights at 2 bytes each is 16GB. Database Mart's same test saw throughput collapse on a 9B model, which they put down to VRAM. Small models are getting more capable fast. If the small model worth fine-tuning next year is 14B, this card runs it only at lower precision or with a smaller batch.

Add it up and a 4090 costs about $51 to $98 a month for this job: $47 to $78 of depreciation, $2 to $12 of failure allowance, $2 to $9 of power. Haiku 5.5 runs the same job for about $137. The card still wins, by about $39 to $86 a month, a lot less than the zero on its bill suggests. If the card sits in your machine for gaming anyway, the job adds only power and a little wear, and the cost drops back toward $2 to $9. Either way, that's before the hours the fine-tune itself takes.

What changed: a $0.10 small-model price floor

OpenAI and Anthropic now sell their small models at $0.10 per million input tokens and $0.50 per million output. The price war happened in steps. OpenAI cut GPT-5.6 Luna 80% in late July. GPT-6 Luna launched September 22 at $0.10 and $0.50 (VKTR). On October 7, Claude Haiku 5.5 matched it to the cent.

Haiku 4.5 listed at $1 and $5. That makes the Haiku 5.5 list price 90% lower. Anthropic's own headline says something smaller:

"around 75% less to run" Anthropic, Haiku 5.5 launch, Oct 7 2026

Both numbers are honest. The gap between them is fine print, and the fine print matters for anyone doing break-even math.

Why Haiku 5.5's 90% price cut bills closer to 75%

Two things eat the discount: a bigger tokenizer and a long-prompt tier. Start with the tokenizer. Per Anthropic's pricing docs, Claude 4.7 and later models produce roughly 30% more tokens for the same text. You pay the new price on more tokens.

One Hacker News user ran the same classification job through both cheap models. Haiku billed 11.9M input tokens and Luna billed 7.9M. Same list price, and the commenter reported $0.42 for Haiku against Luna's $0.31.

The second is the 100K cliff. Prompts over 100,000 tokens pay a tier five times higher, $0.50 in and $2.50 out. Cached tokens count toward the threshold. Anthropic says short prompts were about 90% of requests to the previous Haiku, so most traffic never sees the cliff.

Agent traffic does. Browser Use found Haiku 5.5 matched Luna on its benchmark but was "2.8x more expensive" because long tasks crossed the tier.

So where you land depends on your prompts. On the short-prompt job below, the per-document cut works out to 87%. Anthropic's 75% is a blended figure, with the long prompts and the bigger tokenizer both priced in.

Self-hosting break-even math: one H100 vs Haiku 5.5

One rented H100 now has to process about 4.3 million documents a month before it beats Haiku 5.5 on price. Against Haiku 4.5 it was about 560,000.

The April post used a 10,000-documents-a-day job as its example. I kept that volume and made the token counts explicit: 2,000 input and 300 output tokens per document, 300,000 documents a month.

The API is a variable cost per document. A dedicated GPU is a fixed monthly cost. Break-even is the volume where they cross.

• Volume: 300,000 documents a month (10,000 a day), each 2,000 input and 300 output tokens.
• Haiku 4.5: $1 per million input and $5 per million output tokens, about $0.0035 per document.
• Haiku 5.5: $0.10 and $0.50, plus about 30% more tokens from the newer tokenizer, about $0.00046 per document.
• One H100: $2.69 an hour on RunPod, about $1,964 a month running full time.
• Break-even: the GPU's monthly cost divided by the API cost per document. That comes to about 561,000 documents a month against Haiku 4.5 and about 4.3 million against Haiku 5.5.

The April example said this job cost about $50,000 a year on GPT-4, and a fine-tuned small model could bring it under $5,000. Haiku 5.5 runs it for about $1,640 a year with no training run at all.

The break-even moved about 7.7x in one product cycle. Then add people. Braindetox's illustrative example budgets a fifth of an engineer, about $1,600 a month, for ops. With that in, the line lands near 7.8 million documents a month. That's 260,000 a day.

I tried moving the inputs. At DeepInfra's $2.20 an hour the GPU-only break-even drops to about 3.5 million. At Lambda's $4.29 it rises to about 6.9 million. If your text inflates 1.5x, like the Hacker News job did against Luna's tokenizer, it drops to about 3.7 million. None of those get you anywhere near April's 560,000.

I checked whether one GPU could even carry that. At 4.3 million documents, the box has to average about 490 output tokens a second, plus the prefill for 2,000 input tokens per document. Database Mart's vLLM benchmark measured a 7B model on one H100 at roughly 5,800 to 6,300 output tokens a second over a 300-request run. The hardware has room. The question is whether you have the traffic to keep it busy, because an idle H100 bills the same as a busy one.

Luna is the cheapest API row, at about $105 a month. If your task fits OpenAI's Decisions API, in public beta since October 6, which returns typed yes/no, choice or score outputs at $0.10 per million input tokens with output unbilled, it gets cheaper again. That product is a direct replacement for the small classifier people used to fine-tune.

What I got wrong about fine-tuning in April

The April post priced a fine-tune against GPT-4. A fine-tuned 7B was competing with the lab's own small model the whole time, and I never priced that matchup.

The row I expected to win this time was the hosted fine-tune. A LoRA is a small add-on layer trained on your examples while the base model stays frozen. Fireworks serves fine-tuned 4B to 16B models at the same $0.20 per million tokens as base models. That's the setup I recommended in April: train a LoRA, serve it at base-model prices, skip the GPU.

It costs $0.00046 per document. So does Haiku 5.5. The price advantage of the hosted fine-tune rounded to zero.

The strongest case I found for the old view came from July. Braindetox walked through a case where a $500 fine-tuned 9B reached 87.3% of the achievable score against 76.9% for the best frontier setup, with an illustrative break-even around 640,000 items a month. It almost convinced me. Then I checked the baseline. The July comparisons still in circulation priced the API side at $3 and $15 per million tokens or more. The small tier now sits 6 to 30 times below those baselines.

Capability moved too. Anthropic reports Haiku 5.5 at 72.4% on OSWorld 2.1, up from Haiku 4.5's 15.7%. The 2025 pitch was that a tuned 7B matches a weak, expensive small model on your task. In 2026 the small model is neither weak nor expensive.

OpenAI reached the same conclusion from the other side. It is winding down its own fine-tuning platform, and active customers can start new jobs only through January 6, 2027. Its stated reason, via Tessl:

"Prompt-based approaches are now cheaper and faster" OpenAI, 2026

Total cost of a fine-tune: API vs owned 4090 vs rented H100

The card is one of three buckets. The full cost of a fine-tune is:

• Compute. The API bill, a rented H100, or the owned 4090 priced above.
• The one-time build. Labeled examples, an eval set and the training runs.
• Upkeep. Serving the model, checking it against the eval set, and retraining when a better base model ships.

Training compute is the cheap part. The April setup, 500 examples over three epochs, costs under $2 at Fireworks and pocket change in electricity on the 4090. The labeling is where the hours go. I couldn't find a sourced figure for labeling time on a task like this, so these are my estimates: about 2 minutes per example for 500 training and 200 eval examples, plus about 10 hours of setup and training runs. Call it 33 hours once. Upkeep I'd put at 3 hours a month.

The hours cost the same on every row, so the useful number is what the savings pay for them.

• Haiku 5.5 for this job: $137 a month.
• Your time in year one: 33 hours to build plus 3 a month to maintain, 69 hours in all.
• Savings on power alone: $137 minus about $9 is about $1,536 a year, or about $22 per hour of your time.
• Savings counting depreciation and failure risk: $137 minus about $51 to $98 is about $468 to $1,032 a year, or about $7 to $15 an hour.

On a card you already run for other things, the fine-tune pays about $22 an hour in year one, and about $43 from year two once the build hours are behind you. Count the card's depreciation and failure risk and it pays about $7 to $15 an hour. Against Luna at $105 a month, the power-only rate drops to about $17. At 300,000 documents a month, most people's evenings are worth more than that.

Volume changes the answer, because the hours and the card stay fixed while the API bill grows. At 3 million documents a month, Haiku 5.5 costs about $1,365. The 4090 is busy about 58% of the month on the cushioned estimate and draws about $85 of California power. The same 69 hours then earn about $200 each, by my arithmetic.

So the price war settled two of the three compute rows. A rented H100 needs about 4.3 million documents a month to win, and the hosted fine-tune costs the same per document as Haiku 5.5. The owned card is the one row where a fine-tune still saves money at this volume, and the savings have to cover your hours along with the card.

Why the $0.10 API price floor may not hold

The $0.10 price is a competitive price, set by two pre-IPO labs. Nothing in it promises it stays.

On October 8, the FT reported that OpenAI told investors its annualized revenue was "approaching $50 billion". The figure in circulation since late September was $68 to $70 billion. The FT's explanation is accounting. Anthropic books sales through AWS and Google Cloud at gross, and OpenAI books its share. Oracle fell nearly 6% and CoreWeave nearly 8% that day (CNBC).

That story was still moving as of October 10. A later report says the FT was told the $70 billion was net. I'd hold both numbers loosely until OpenAI says something on the record.

The subsidy shows up elsewhere too. SemiAnalysis estimates Anthropic's subscriptions are about 10% of revenue and over 40% of inference compute (SemiAnalysis). And DeepSeek raised its prices in August. Peak-hour output on its flash model now costs $1.20 per million, more than Haiku 5.5 or Luna.

Why the labs are cutting is contested. Gavin Baker thinks open weights are taking share. Ara Kharazian at Ramp says it's lab-versus-lab competition, and puts open source at under 5% of business spend in a post on X. I haven't seen that figure published anywhere else. ETR data has open weights at 34% of enterprise tokens. Both can hold at once. One routing gateway's open-weight traffic averages about 7x cheaper per token. Apply that to ETR's third and you get roughly 7% of the dollars, by my arithmetic.

When fine-tuning still wins

Fine-tuning still wins on latency, data boundaries, deprecation risk and narrow tasks that a tiny model can learn. Price still counts at high volume, or on a card already in your machine for other work. The others got stronger.

Latency. Haiku 5.5 thinks before it answers. Artificial Analysis measures its time to first token, the wait before any output appears, at about 10 seconds on low effort and 13 to 14 on medium, as of October 10. That figure includes thinking, and I couldn't find a no-thinking measurement. A small model on your own GPU answers in a fraction of a second. If a person is waiting on the response, that gap decides the product.

Data boundary. Some data can't leave your network. The API version of that costs extra: Anthropic charges 1.1x for US-only inference. This one cuts both ways. In ETR's survey, 62% of enterprises name unfinished security and compliance reviews as the main obstacle to running open weights in production (NYU Shanghai RITS).

Deprecation. A fine-tune on a vendor platform dies when the vendor says so. OpenAI just proved that. A LoRA on open weights keeps running on whatever hardware you point it at.

Narrow tasks on tiny models. Unsloth took a 0.8B Qwen model from 20.7% to 74.3% accuracy across three decision benchmarks, on 4GB of VRAM. A model that small runs on hardware you already own.

A classifier I already run. I use a custom fine-tuned classifier in Foliome, an open-source project for personal finance and wealth management. It sorts bank transactions into spending categories on a laptop CPU, so the descriptions never leave the machine. That's the shape where a small bespoke model still earns its place: a narrow task, private data, and a model small enough to run on hardware you already own.

How I'd decide on a fine-tune now

Price both bills honestly, then decide on the reasons price can't buy.

1. Default new narrow tasks to the cheapest capable API and measure the real bill, tokenizer included.
2. Keep your eval set and your labeled examples current, even if you never train on them. That's most of the one-time cost of a fine-tune, paid in advance.
3. Price a local fine-tune all-in: the card over a life you'd bet on, a failure allowance, power and your hours. Compare that total to the API bill.
4. Fine-tune when a year of savings pays your hours at a rate you'd accept, or when one of the surviving reasons applies: sub-second latency, data that can't leave, a vendor you can't afford to lose.
5. Rent a GPU only past roughly 4 million documents a month.

My own stack already splits this way. The agent loops run on Claude through the API. The local 4090 gets the jobs where latency or privacy decides. If I were starting a fine-tune today, I'd still reach for Unsloth and a Qwen base, now Qwen3.5, and start at the smallest size that clears my eval.

The April post said the tax was optional. Six months later, most of us should just pay it, and the local alternative turned out to have a bill of its own, in depreciation, failure risk and hours. What still justifies a local model is what no price cut touches: latency, data that can't leave, and not living at a vendor's mercy. Keep building the labeled data and the eval set either way. If the $0.10 floor moves, they're what let you leave.