·Par AI Resource Hub Team
Exécuter des LLM locaux avec Ollama : Guide complet
Apprenez à installer et utiliser Ollama pour exécuter des LLM open source localement.
Why Run LLMs Locally?
- Privacy: Your data never leaves your machine.
- Cost: Zero per-token costs after setup.
- Speed: No network latency for inference.
- Offline: Works without internet connection.
Installing Ollama
- Download from [ollama.com](https://ollama.com) — available for macOS, Linux, and Windows.
- Run `ollama pull llama3` to download a model.
- Run `ollama run llama3` to start chatting.
Popular Models
- Llama 3 8B: Good general-purpose model, runs on 8GB RAM.
- Llama 3 70B: Much smarter, needs 48GB+ RAM.
- Qwen 2.5: Excellent for Chinese and multilingual tasks.
- Mistral 7B: Fast and efficient for coding tasks.
- Phi-3 Mini: Microsoft's small but capable model.
Using the API
Ollama provides a local REST API at `http://localhost:11434`.
- `POST /api/generate` — text generation.
- `POST /api/chat` — chat completions.
- Compatible with OpenAI SDK via base URL override.
Integration Tips
- Use with LangChain via `OllamaLLM` class.
- Use with Open WebUI for a ChatGPT-like interface.
- GPU acceleration: Works with NVIDIA (CUDA) and Apple Silicon (Metal).
Performance Expectations
- 7B model on M2 MacBook: ~30 tokens/second.
- 7B model on RTX 4090: ~60 tokens/second.
- 70B model on 48GB RAM (CPU): ~3 tokens/second.
TutorielLLM local