Skip to content

Transformer

The neural network architecture behind modern LLMs, using self-attention to process all tokens in parallel.

Introduced in the 2017 paper "Attention Is All You Need," it replaced recurrent networks and enabled training at massive scale. Nearly every well-known LLM today is a Transformer variant.

Related resources

Hugging Face Transformers Tutorial
The official tutorial for Hugging Face Transformers, the industry-standard library for NLP and computer vision. It covers text classification, tokenisation, translation, image segmentation, and more, with code examples for both PyTorch and TensorFlow backends. Aimed at practitioners who want to leverage pre-trained models or fine-tune them on custom datasets for production use.
PyTorch Deep Learning Framework
PyTorch is a leading open-source deep learning framework developed by Meta, known for its dynamic computation graph and Pythonic interface. It powers both cutting-edge AI research and production deployments, with a rich ecosystem of tools for training, debugging, and scaling models across GPUs and TPUs.