Why Fine-Tune?

Pre-trained models are powerful but may not match your domain. Fine-tuning adapts them to your specific needs without training from scratch.

The Cost Problem

Full fine-tuning of a 7B parameter model requires ~160GB VRAM. That's 2-4 A100 GPUs at $2-3/hour each.

Parameter-Efficient Methods

  • LoRA: Add small trainable matrices to frozen layers. Reduces trainable parameters by 100x.
  • QLoRA: Quantize the base model to 4-bit, then apply LoRA. Fits a 65B model on a single 48GB GPU.
  • Prefix Tuning: Only train small prefix vectors. Minimal memory overhead.
  • Adapter Layers: Insert small trainable modules between frozen layers.

Practical Setup

  • Hardware: A single RTX 4090 (24GB) or A10G (24GB) handles most QLoRA jobs.
  • Cloud options: RunPod, Lambda Labs, Vast.ai at $0.3-0.8/hour.
  • Frameworks: Hugging Face PEFT, Unsloth, Axolotl.

Step-by-Step

1. Prepare your dataset (JSONL format, 1K-10K examples).

2. Choose a base model matching your task domain.

3. Configure QLoRA with rank 16-64 and alpha 16-32.

4. Train for 1-3 epochs, monitoring validation loss.

5. Merge adapters and evaluate on held-out test data.

Tips

  • Start with 1K high-quality examples before scaling to 10K.
  • Use instruction-following format even for completion tasks.
  • Always evaluate on data the model hasn't seen during training.

Cost Optimization Strategies

  • Start with LoRA/QLoRA: Full fine-tuning requires enormous GPU memory. LoRA reduces trainable parameters to <1% while maintaining 90-95% of full fine-tuning quality.
  • Use smaller base models: Fine-tuning Llama 3.1 8B costs ~10x less than 70B. Start small, scale only if needed.
  • Cloud GPU pricing: A100 40GB ~$1.50/hr, H100 ~$3.00/hr. A typical LoRA fine-tune takes 2-4 hours.
  • Data quality over quantity: 1000 high-quality examples often outperform 10,000 mediocre ones. Curate your dataset carefully.

Common Pitfalls

  • Overfitting: Monitor validation loss. Use early stopping and dropout.
  • Catastrophic forgetting: Mix in general conversation data (10-20%) to maintain base capabilities.
  • Evaluation gaps: Don’t just check loss curves. Build a held-out test set and evaluate task-specific metrics.
  • Ignoring data formatting: The chat template format matters enormously. Match the format the base model was trained on.