Open-Source LLM Stack
Run and build with open-weight models: local runtimes, inference servers, and app frameworks.
Deploying Local LLMs with Ollama
A cross-platform tool that lets you download and run open-source LLMs like Llama, Mistral, and Phi locally on your own machine. It provides a CLI, a local API server, and an OpenAI-compatible endpoint, so you can experiment with models privately — no GPU or cloud account required. Ideal for developers who want offline AI capabilities or need to test prompts before deploying to production.
vLLM High-Performance Inference Framework
vLLM is a high-throughput inference and serving engine for large language models, using PagedAttention to dramatically improve memory efficiency and throughput. It supports continuous batching, tensor parallelism, and OpenAI-compatible API serving, making it a top choice for deploying LLMs at scale.
Open WebUI
Open WebUI is a feature-rich, self-hosted AI chat interface that supports Ollama and OpenAI-compatible APIs out of the box. It runs fully offline, offers a polished multi-model conversation experience, and includes role management, markdown support, and plugin extensibility for teams and individuals.
LM Studio Local Model Client
LM Studio is a desktop application that lets you download and run open-source LLMs locally with a single click. It provides a polished graphical interface for chatting with models, an OpenAI-compatible local API server, and support for GGUF/GGML formats — no command line or GPU expertise required.
Dify LLM App Development Platform
Dify is an open-source LLM application development platform that combines visual workflow orchestration, a built-in RAG engine, and Agent capabilities. It lets developers and non-technical users quickly build, test, and deploy AI-powered apps with support for multiple model providers and seamless API integration.
PyTorch Deep Learning Framework
PyTorch is a leading open-source deep learning framework developed by Meta, known for its dynamic computation graph and Pythonic interface. It powers both cutting-edge AI research and production deployments, with a rich ecosystem of tools for training, debugging, and scaling models across GPUs and TPUs.
Hugging Face Transformers Tutorial
The official tutorial for Hugging Face Transformers, the industry-standard library for NLP and computer vision. It covers text classification, tokenisation, translation, image segmentation, and more, with code examples for both PyTorch and TensorFlow backends. Aimed at practitioners who want to leverage pre-trained models or fine-tune them on custom datasets for production use.
Meta Llama
Llama is Meta's family of open-weight language models — the backbone of the open-source AI ecosystem, deployable locally or on any cloud. Available in multiple sizes from lightweight to frontier-class, Llama models power countless fine-tuned derivatives and give organizations full control over their AI stack.
llama.cpp
llama.cpp is the lightweight C/C++ inference engine that runs open LLMs efficiently on CPUs and consumer GPUs, serving as the foundation beneath many local AI tools like Ollama and LM Studio. Its GGUF quantization formats let large models fit into modest hardware without dedicated accelerators.