예산 내에서 LLM을 파인튜닝하는 방법

Why Fine-Tune?

Pre-trained models are powerful but may not match your domain. Fine-tuning adapts them to your specific needs without training from scratch.

The Cost Problem

Full fine-tuning of a 7B parameter model requires ~160GB VRAM. That's 2-4 A100 GPUs at $2-3/hour each.

Parameter-Efficient Methods

• LoRA: Add small trainable matrices to frozen layers. Reduces trainable parameters by 100x.
• QLoRA: Quantize the base model to 4-bit, then apply LoRA. Fits a 65B model on a single 48GB GPU.
• Prefix Tuning: Only train small prefix vectors. Minimal memory overhead.
• Adapter Layers: Insert small trainable modules between frozen layers.

Practical Setup

• Hardware: A single RTX 4090 (24GB) or A10G (24GB) handles most QLoRA jobs.
• Cloud options: RunPod, Lambda Labs, Vast.ai at $0.3-0.8/hour.
• Frameworks: Hugging Face PEFT, Unsloth, Axolotl.

Step-by-Step

1. Prepare your dataset (JSONL format, 1K-10K examples).

2. Choose a base model matching your task domain.

3. Configure QLoRA with rank 16-64 and alpha 16-32.

4. Train for 1-3 epochs, monitoring validation loss.

5. Merge adapters and evaluate on held-out test data.

Tips

• Start with 1K high-quality examples before scaling to 10K.
• Use instruction-following format even for completion tasks.
• Always evaluate on data the model hasn't seen during training.