LoRA

LoRA, short for low-rank adaptation, is the most common way builders fine-tune open-weight models. Instead of updating every weight, it freezes the original model and trains small extra matrices alongside some of its layers. Together these form an adapter, typically a tiny fraction of the model's size, that steers the model toward your task.

That changes the practical shape of fine-tuning. Training fits on much smaller GPUs, and QLoRA goes further by loading the frozen base model in 4-bit precision during training. The output is a small file you can version, store with your releases, and roll back. Serving stacks such as vLLM can attach several adapters to one base model, so different features or customers share the same hardware. For laptops or on-premises boxes, you can merge an adapter into the base weights and quantize the result.

LoRA has knobs, such as the rank and which layers get adapters, but for product work the bigger levers are data quality and the choice of base model. A better base model often beats a better training run, so evaluate a few candidates with plain prompting first. An adapter only works with the exact base model it was trained on: upgrading the base means retraining the adapter and rerunning your evals.