Skip to content

GGUF

A file format for storing quantized models, widely used to run LLMs locally with tools like llama.cpp and Ollama.

Related resources

Deploying Local LLMs with Ollama
Run large language models locally with Ollama, supporting open-source models such as Llama and Mistral.
LM Studio Local Model Client
A desktop app that downloads and runs open-source LLMs locally with one click, offering a graphical interface and an OpenAI-compatible API.