Getting Started with Fine-Tuning: LoRA and QLoRA
When prompt engineering and RAG cannot meet your needs, fine-tuning can make a model reliably master a specific style or task. This tutorial introduces the most mainstream efficient fine-tuning methods: LoRA and QLoRA.
ByAI Resource Hub
When Do You Actually Need Fine-Tuning
If you only need to supplement knowledge, prefer RAG. Fine-tuning is more appropriate when you need a fixed output style or format, or want a small model to learn a specific task.
Fine-tuning requires a certain amount of high-quality labeled data, so think through your goals and evaluation approach before investing.
The Cost of Full Fine-Tuning
Full-parameter fine-tuning updates all of the model weights, incurring enormous memory and compute costs—for large models this is nearly unaffordable for individuals and small teams.
The Core Idea of LoRA
LoRA (Low-Rank Adaptation) freezes the original weights and inserts a pair of small low-rank matrices alongside them to learn the "delta". The number of trainable parameters is often just one percent of full fine-tuning, or even less.
After training you get a very small adapter that can be loaded on demand, making it easy to maintain multiple sets for different tasks.
QLoRA and Quantization
Building on LoRA, QLoRA first quantizes the base model to 4-bit before training, further reducing memory usage dramatically and making it feasible to fine-tune fairly large models on a single consumer GPU.
Data Preparation and Practical Tips
Data quality matters more than quantity: construct clear, consistent instruction-answer pairs that cover your target scenarios.
Start with a small learning rate and a few epochs, watch validation performance closely to avoid overfitting, and use libraries such as Hugging Face PEFT to get started quickly.
Merging and Deploying Adapters
After training, you can either merge the LoRA weights back into the base model for a single standalone file, or keep them separate and load them on demand. Merging is simpler for deployment but means you lose the ability to swap adapters without re-exporting. Keep separate adapters when you need to serve multiple tasks from the same base model; merge when you want a single artifact for a single purpose.
One adapter step
This call solves the step that attaches a LoRA adapter before any training loop. The rank and alpha in the snippet are a starting sketch, not a measured result. Failure: if target_modules names are not in this model, print_trainable_parameters reports 0 and the loss never moves the layers you meant to train.
from peft import LoraConfig, get_peft_model
config = LoraConfig(
r=8,
lora_alpha=16,
target_modules=["q_proj", "v_proj"],
task_type="CAUSAL_LM",
)
model = get_peft_model(base_model, config)
model.print_trainable_parameters()