CyberFreezeDev
Loading
Fine-tuning on the cheap: train a small adapter beside the frozen model, not every weight.
LoRA freezes the original weights and trains two small matrices on chosen layers. The adapter is often tens of megabytes; QLoRA cuts memory further.
LoRA (low-rank adaptation) is a parameter-efficient way to fine-tune a model, introduced by Hu and colleagues at Microsoft in 2021. The original weights are frozen. For selected layers, LoRA adds two small matrices whose product represents the change you want. Only those small matrices are trained.
The result is an adapter: often tens of megabytes, compared with many gigabytes for the base model. QLoRA (Dettmers and colleagues, 2023) goes further by loading the frozen base in 4-bit quantized form while training the adapter, which cuts memory again.
Start with an open-weight model such as Llama, Qwen or Mistral.
An open-weight model such as Llama, Qwen or Mistral, in full or quantized precision.
Small matrices go on chosen layers; the rank sets their size and capacity.
Small matrices are added to chosen layers, usually the attention projections. You set the rank, which controls adapter size and capacity.
Only adapter weights change, so memory needs fall sharply.
Gradients flow through the frozen model but only adapter weights change, so memory needs fall sharply. Libraries such as Hugging Face PEFT handle this.
Merge the adapter for speed, or load the right one per task or client.
Merge the adapter into the base weights for speed, or keep it separate and load the right adapter per task or per client.
Small models can be adapted on one workstation GPU, so data stays in the building.
Small models can be adapted on a single workstation GPU, so training data never leaves the building.
Several adapters, such as a classifier and a tone adapter, share one base model.
A claims classifier, an email tone adapter and an extraction adapter can share one base model in memory.
Short runs make it practical to try, measure and discard versions.
Short training runs make it practical to try, measure and discard versions until one is clearly better.
An agency trains one LoRA adapter per brand on approved copy, runs one base model, and adds a brand with one short training run.
A marketing agency writes product copy for several brands, each with its own tone rules. Rather than host one tuned model per brand, it trains a LoRA adapter per brand on approved past copy. One base model runs on the agency's server; the request for brand A loads adapter A. Adding a new brand means one more short training run, not a new deployment.
It shapes behaviour and format. Facts that change belong in RAG.
Like any fine-tuning, it shapes behaviour and format. Facts that change belong in RAG.
Too low cannot learn the task; too high overfits. Test a few values.
Too low and the adapter cannot learn the task; too high and it overfits small datasets. Test a few values against a held-out set.
An adapter works only with the exact base model and version it was trained on.
An adapter only works with the exact base model and version it was trained on.
For many narrow business tasks the results are close, and the cost is far lower. For large shifts in what a model can do, full fine-tuning can still win. Measure on your own test set.
No. It is widely used with image diffusion models to teach a style, product or character from a small set of pictures.
Last updated