Fine-tuning is usually the wrong answer. It's slower, more expensive, and harder to maintain than prompting plus RAG. But for some problems it's the only answer, and when it works the gains are enormous.
This is a playbook from a project where it was the right call: fine-tuning Llama 3 on 50K domain-specific examples for an enterprise NLP workload.
When fine-tuning is worth it
Three conditions, all of them: the task has a consistent shape (input format, output format, expected reasoning), you have or can produce high-quality labeled data, and prompt engineering has plateaued.
If you haven't squeezed prompts hard, don't fine-tune. If your task drifts (one user wants summary, the next wants extraction, the next wants classification), don't fine-tune. If your dataset is messy, fine-tuning will encode the mess.
Dataset curation
50K examples sounds like a lot until you start checking quality. We threw out roughly 30% of our initial dataset for one of three reasons: label noise, leaked test data, or examples that were technically correct but stylistically inconsistent.
Stylistic inconsistency is the killer. A fine-tuned model converges on the average of your training data. If half your training answers are terse and half are verbose, the model will produce something nobody asked for.
LoRA vs full fine-tune
LoRA wins for most enterprise use cases. You get 80% of the gain at 5% of the cost, and the resulting adapter is small enough to swap quickly.
Full fine-tune wins when you need to change the model's behavior significantly: new domain vocabulary, new task structure, new output format. Those are rare.
Training infrastructure
P3 and G4 instances on EC2 with NVIDIA CUDA. Distributed training across 4 GPUs. SageMaker for orchestration so we could track experiments and avoid the "which checkpoint was that?" problem.
Total training cost for the production model: under $500. The cost math against commercial APIs paid back in three weeks.
Evaluation
Standard supervised metrics (accuracy, F1, precision, recall) plus a human eval set of 200 edge cases that I curated personally. The auto metrics tell you the model is working on the easy stuff. The human evals tell you whether it's working on the stuff that matters.
We hit 94% accuracy versus 82% for the base model on the same eval set. The hard cases drove the gain.
Cost math
The base option used a commercial API at roughly $0.01 per request. Our fine-tuned Llama 3 cost approximately $0.005 per request including infrastructure amortization. At our request volume, that's 50% operational cost reduction in the first year, paying back development effort in three weeks.
Fine-tuning isn't free. But when the math works, it works hard.