CyberFreezeDev
Loading
Further training on your examples so a model follows your format and decisions consistently.
Missing knowledge needs RAG and vague instructions need better prompts. Fine-tuning pays off for strict formats, high volume, private data or specialised vocabulary.
Most teams who ask for LLM fine-tuning actually need something else. If the model does not know your products, policies or prices, that is a knowledge problem and a RAG chatbot solves it better, because documents change and retrieval picks up the change on the next index. If the model gets a task wrong because the instructions are vague, better prompts and a few examples usually fix it in a day.
Fine-tuning earns its cost when:
We start every project by trying the cheaper options against your test set. If they pass, you save the training budget.
LoRA trains a small adapter on a frozen base; several adapters share one server.
LoRA (low rank adaptation, Hu et al. 2021) freezes the base model and trains small extra matrices. The adapter is a few hundred megabytes at most, and several adapters can share one base model on one server.
With a 4 bit base model, QLoRA trains mid-sized models on one workstation GPU.
QLoRA (Dettmers et al. 2023) loads the base model in 4 bit precision while training the adapter, which lets mid sized models train on a single workstation GPU rather than a cluster.
The base model stays untouched; a new base means retraining only the adapter.
The base model is untouched. If a new base model is released, you retrain the adapter on the same dataset rather than starting over.
One task, a held-out test set of real cases and a scoring method.
One task, written down, with a held out test set of real inputs and correct outputs and a way to score them: exact match, field accuracy, a rubric checked by a reviewer, or a mix.
A strong hosted model and untrained open models set the score to beat.
Run the test set through a strong hosted model with a good prompt, and through the candidate open models without training. This is the number fine-tuning has to beat.
Examples collected, cleaned and agreed with your experts, which usually takes longer than training.
Collect and clean examples from your records, remove personal data that is not needed, fix inconsistent labels and agree on edge cases with your domain experts. This step usually takes longer than training.
Adapters trained with a few settings and scored against the baseline, failures included.
Train adapters with a few settings, score each on the test set, and inspect the failures by hand. We report the results against the baseline, including where the tuned model is worse.
The adapter served behind a logged API, quantized for your hardware and wired in.
Merge or serve the adapter, quantize if needed for your hardware, put it behind an API with logging, and connect it to the workflow that uses it.
Corrections made in production become new examples for the next training run.
Corrections your team makes in production are collected as new examples, so the next training run starts from better data.
Small models and quick experiments run on our 8 GB laptop GPU, results published.
Our lab machine includes an RTX 3070 Ti laptop GPU with 8 GB of memory, which suits small models and quick experiments. We publish what we run on it, with versions and dates, in local LLMs.
Rented cloud GPUs for a one-off run, or your own server for frequent retraining.
For bigger models we size the training hardware with you: rented cloud GPUs for a one-off run, or a server you own if you will retrain often. See GPU servers and AI hardware.
Train on rented GPUs, serve on a smaller office machine, both planned up front.
A model can be trained on rented GPUs and served on a smaller machine in your office. We plan both before training starts.
It depends on the task. A narrow formatting or extraction task can show clear gains with a few hundred good examples; a task that needs judgement needs more and needs them consistent. We decide after the baseline, when we can see where the model fails.
Some hosted providers offer fine-tuning for some of their models, with the trained model staying on their platform. We use those where they fit, but most of our work is on open-weight models because you own the result and can run it anywhere.
Not on its own. Fine-tuning shapes behaviour and format; it is a poor way to store facts. For factual answers combine a tuned model with retrieval from your documents.
You do. The dataset, adapter weights, training configuration and evaluation results are handed over. Check the base model's licence as well; we flag any restrictions before training.
Last updated