Buy, build, or fine-tune

Buy an off-the-shelf API, build something bespoke, or fine-tune a base model on your own data? Teams argue this as a technology question, which is why the argument never ends. It is not a technology question. It is a question about where your competitive differentiation lives in a market where intelligence itself is commoditizing - and once you frame it that way, the answer stops being "it depends" and becomes a short, reasoned sequence.

This post gives you that sequence: the escalation ladder the 2026 practice has converged on, the break-even numbers at each rung, the specific cases where fine-tuning genuinely wins, and the total-cost trap that makes self-hosting look 3 to 5 times cheaper than it is. The one-sentence preview: buy the commodity, fine-tune the differentiation, build almost nothing - and treat every rung as reversible.

Table of contents
1. Reframe: differentiation, not technology
2. The escalation ladder: prompt, RAG, fine-tune, distill
3. What fine-tuning is actually for
4. The economics at each rung
5. The self-hosting TCO trap
6. When "build" is real
7. The decision sheet
8. Honest tradeoffs and reversibility
9. Takeaways

1. Reframe: differentiation, not technology

The pattern across the credible 2026 frameworks is unanimous on the reframe. The decision is about where to place your competitive differentiation in a rapidly commoditizing intelligence market: successful organizations buy to learn, then build only where it matters, treating model weights and infrastructure as disposable commodities. Meanwhile training from scratch is justified only for frontier labs, national programs, and total-provenance regulatory cases - for everyone else, the compute and data burden buys no advantage over a tuned open-weight model.

Two corollaries do a lot of work. First, the binary is dead: the standard enterprise answer in 2026 is hybrid - a managed API for general tasks plus a tuned open-weight model for the domain-specific core. Second, whichever path you choose, the decision is really a data-operations decision: buy, build, and fine-tune each relocate the burden of corpus curation, annotation, and evaluation - none eliminates it. If your data operations are weak, no procurement choice fixes that.

2. The escalation ladder: prompt, RAG, fine-tune, distill

The sequencing rule that should be printed above every AI team's desk, per the converged decision frameworks:

1. PROMPT     free, hours to iterate. Stay until prompts are
              longer than the task and still failing.
2. RAG        add when answers must cite current documents.
              Knowledge that CHANGES lives here. (See our
              /blogs/rag-is-an-architecture for the build.)
3. FINE-TUNE  add when you need the same BEHAVIOR every time:
              style, format, refusals, classification.
              LoRA/QLoRA adapter on a strong base - not a
              new model.
4. DISTILL    frontier model generates labeled outputs on your
              task -> tune a small open model on them -> swap it
              in for 60-80% inference savings at scale.

Escalate on EVIDENCE (eval failures at the current rung),
never on ambition.

The dividing line between rungs 2 and 3 is the single most useful heuristic in this space: RAG is for knowledge that changes; fine-tuning is for behavior that shouldn't. Fine-tuning is for form, not facts - it cannot cite the document that justified an answer, so anything needing provenance or auditability stays retrieval-shaped. Teams that fine-tune to inject knowledge buy themselves a model that is confidently out of date and can't show its work.

3. What fine-tuning is actually for

The legitimate cases, in descending order of how often we see them pay off:

• Distillation for cost and latency. The strongest 2026 case: use the frontier model as a teacher, tune a small open model (Llama, Qwen, Mistral class) on its outputs for your task, and cut inference spend 60 to 80 percent at equal-or-better task quality. This is the industrial version of the routing lever from our inference-cost post.
• Consistency at scale. The tell is specific: the model understands what you want but fails to do it reliably across thousands of interactions. Structured outputs, house style, refusal patterns, domain classification - behavior you can demonstrate but not reliably prompt.
• Domain specialization where general models are close-but-wrong. Your terminology, your document formats, your label taxonomy - with proprietary data as the moat that makes the tuned model hard to copy.

And the standing caution from the same literature: most teams considering fine-tuning should not - they should fix prompts, build a real retrieval pipeline, and write evals, in that order. Fine-tuning without an eval harness is spending money to make your system harder to debug.

4. The economics at each rung

The numbers that anchor the decision, from the published 2026 analyses:

• Fine-tuning cost: a 7B-13B open model runs roughly $2,000 to $30,000 in compute and data preparation - but budget 3 to 5x the training cost for the first year of lifecycle ownership: adapter versioning, rollback plans, retraining cadence, base-model drift. The training run is the down payment, not the price.
• Fine-tuning break-even: somewhere between 100K and 1M calls per month for most tasks - below that, the lifecycle tax exceeds the per-call savings and prompting a bigger model wins.
• Self-hosting break-even: the threshold scales with the hardware the replacement model needs. Small open models on modest GPUs break even around 500K to 2M tokens per day; replacing frontier-class workloads with large open models (bigger hardware, higher fixed cost) breaks even around 100 to 256 million tokens per month. A simpler screen: if your API bill is under $5k a month, self-hosting is a hobby, not a strategy.

5. The self-hosting TCO trap

The most expensive spreadsheet error in this domain: comparing API token rates against raw GPU rental prices. The published warning is quantified - teams starting from GPU prices and token rates underestimate self-hosting TCO by 3 to 5x. The missing lines: ops staffing (the 0.25-0.5 FTE that becomes 2 FTEs at scale), utilization (a GPU you use 20 percent of the time costs 5x its sticker per useful token), electricity and networking, model upgrades and re-quantization, rate-limit and failover engineering, and the downtime that an API vendor absorbs for you today.

A related budget honesty note that applies to every rung: a 2025 analysis found teams exceeding LLM budgets by 340 percent - with the root cause almost never token rates but missing per-tenant usage tracking and query-level cost attribution. Whatever you buy, build, or tune: instrument attribution first. You cannot manage a bill you cannot decompose.

6. When "build" is real

Cases where the math and the constraints genuinely force ownership - and note that even these mean "self-host and tune open weights," almost never "train from scratch":

• Compliance walls: data that legally cannot leave your infrastructure - and with the EU AI Act reaching full enforcement in August 2026, regulated industries increasingly want the logging and audit control that self-hosting gives.
• Physics: sub-100ms latency budgets that network round-trips to an API cannot meet - on-device or edge serving becomes the only architecture that works.
• Volume: sustained token volumes past the break-even thresholds above, with predictable load - at 10M-plus tokens a day, the 60-80 percent per-token savings pays for the ops team it requires.
• The system IS the product. If your company's moat is the model itself, renting your moat from a vendor is a strategy question that answers itself.

7. The decision sheet

The one-page instrument we fill in with clients before this decision is made - every row is checkable, none is a vibe:

1. Where does differentiation live?   workflow / data / model / UX
   (if not "model" or "data" -> BUY and move on)
2. Current rung + eval evidence:      prompt / RAG / tune / distill
   (escalation requires a failing eval, not an opinion)
3. Knowledge or behavior problem?     changes weekly -> RAG
                                      must be identical -> tune
4. Provenance required?               yes -> RAG stays in the loop
5. Volume:                            calls/mo + tokens/day
   vs break-evens (100K-1M calls; ~100M+ tok/mo)
6. TCO honest?                        x3-5 the naive self-host
                                      number; +3-5x yr-1 tune cost
7. Compliance/latency walls?          name the regulation or the ms
8. Reversibility:                     what does switching back cost?
   (adapter on open base = cheap; bespoke stack = career)

8. Honest tradeoffs and reversibility

• Buying carries drift you don't control. Vendor retunes change behavior under your feet (our model point-of-view post covers the audit); mitigations are eval suites on every model update and keeping the swap cheap - which is also the anti-lock-in play.
• Fine-tuning couples you to a base model's lifecycle. When the base is deprecated, your adapter and its eval history need a migration plan. Version everything; assume retraining twice a year.
• Building buys control and bills you for optionality. The stack you own is the stack you must evolve - every serving-layer innovation (new attention kernels, cheaper quantization, better batching) arrives on your roadmap instead of in a vendor changelog.
• The ladder is descendable. The best property of prompt-to-distill sequencing is that each rung is cheap to retreat from - which is exactly what makes escalating safe. Protect that property: the moment a rung becomes irreversible (bespoke infra, undocumented adapters), you have converted a decision into a commitment, and commitments need a much higher bar.

9. Takeaways

• This is a differentiation-placement decision, not a technology preference. Buy the commodity; invest only where you are genuinely different.
• Climb the ladder on evidence: prompt, then RAG, then fine-tune, then distill - each escalation justified by a failing eval at the current rung.
• RAG for knowledge that changes, fine-tuning for behavior that shouldn't, provenance always retrieval-shaped.
• Know the numbers: tuning breaks even around 100K-1M calls/month and costs 3-5x its training price in year one; self-hosting breaks even around 100M+ tokens/month and is underestimated 3-5x by naive math.
• "Build" means tuned open weights behind your firewall for compliance, latency, volume, or moat reasons - training from scratch is for frontier labs.
• Whatever you choose: cost attribution and evals first, and keep every choice reversible. The hybrid (API for general, tuned open model for the core) is the 2026 default for a reason.

References

1. The build-vs-buy LLM infrastructure decision most teams get wrong (TianPan)
2. RAG vs fine-tuning in 2026: a decision framework (Winder.AI)
3. Fine-tuning LLMs in 2026: when RAG isn't enough (BigData Boutique)
4. Enterprise LLM training: build, buy, or hybrid (Digital Divide Data)
5. Local LLMs vs cloud APIs: 2026 TCO analysis (SitePoint)
6. Self-host LLM vs API: real cost breakdown (DevTk.AI)
7. LLM fine-tuning: a practical guide for teams considering it (AlphaCorp)
8. Buy versus build an LLM: a decision framework for governments (arXiv)

Hero image: Felix Mittermeier on Unsplash.

We help teams choose deliberately, then ship it. Explore our other insights or get in touch if you would like to talk it through.