Best tools to run LLMs on your own machine

These tools let you download and run open-weight models locally with full data privacy, from a simple CLI to a polished chat UI.

Recommended tools

Run large language models locally with Ollama, supporting open-source models such as Llama and Mistral.

A desktop app that downloads and runs open-source LLMs locally with one click, offering a graphical interface and an OpenAI-compatible API.

A feature-rich, self-hosted AI chat interface that supports Ollama and OpenAI-compatible APIs and can run fully offline.

A high-throughput inference and serving framework for large language models that significantly improves deployment efficiency via PagedAttention.