서버리스 AI 추론: 서버 없이 모델 배포

Why Serverless AI?

Traditional GPU servers are expensive and idle most of the time. Serverless inference lets you pay only for actual usage.

Platforms

### Modal

• Python-native serverless cloud.
• GPU access with code changes.
• Great for ML pipelines and batch inference.
• Pricing: Per-second GPU billing.

### Replicate

• Run open-source models via API.
• No infrastructure management.
• Huge model library (Llama, Stable Diffusion, Whisper).
• Pricing: Per-second GPU time.

### Cloudflare Workers AI

• Edge-deployed models globally.
• Low latency for end users.
• Limited model selection but growing.
• Pricing: Per-request.

### Banana / Together AI

• Specialized in LLM inference.
• Auto-scaling from zero to thousands of requests.
• Competitive pricing for high-volume workloads.

Architecture Patterns

• Cold Start Mitigation: Use model caching and pre-warming.
• Request Queuing: Buffer requests during scale-up.
• Edge + Cloud: Simple models at edge, complex in cloud.

When NOT to Use Serverless

• Consistent high-throughput workloads (>80% GPU utilization).
• Models too large for available serverless GPU memory.
• Latency-sensitive applications requiring <50ms response.