LoRA (Low-Rank Adaptation)

A parameter-efficient fine-tuning method that freezes base weights and trains small low-rank matrices, drastically cutting cost.

A LoRA adapter is often only a few megabytes, so you can train it on a single consumer GPU and swap adapters to switch behaviors without duplicating the whole model. Widely used for Stable Diffusion styles and custom LLMs.

Related resources

Hands-on LLM Fine-Tuning

A practical, hands-on guide to fine-tuning large language models, covering parameter-efficient approaches like LoRA and QLoRA alongside traditional full fine-tuning. It walks through dataset preparation, hyperparameter tuning, and evaluation metrics with real code examples. Aimed at ML engineers who want to adapt base models to specific domains without the cost of training from scratch.