Large language model (LLM)

Modern LLMs are built on the transformer architecture, which was introduced in Google’s 2017 research paper "Attention Is All You Need".

The core innovation of the transformer architecture is the attention mechanism. This is what allows a model to weigh the relevance of every token against every other token in the current context when predicting the next token. This enables transformers to capture long-range dependencies in text far more effectively than earlier architectures. See transformer architecture for a full explanation.

In the development of LLMs, training is the computationally enormous process of adjusting a model’s weights by exposing the model to vast amounts of data.

Training a frontier model requires data centers full of specialized hardware running for weeks, or even months. Training demands maximum compute throughput, and so benefits enormously from features like high-bandwidth inter-GPU connections.

Post-training refers to all the additional steps applied to a model after pre-training is complete, to make it more useful, safe, and aligned with human intentions. Key post-training techniques include fine-tuning and RLHF, both described below.

Fine-tuning is the process of taking a pre-trained model and continuing to train it on a smaller, task-specific dataset to adapt its behavior for a particular domain or use case. Rather than training from scratch – which requires enormous compute and data – fine-tuning starts from an existing model’s weights and adjusts them incrementally.

Fine-tuning is how general-purpose base models are turned into specialized assistants. A model fine-tuned on medical literature will perform better on clinical tasks than the same base model used out-of-the-box.

A specialized form of fine-tuning is instruction tuning, which is training a base model on examples of instructions paired with desired responses. Most frontier models also undergo RLHF as part of their post-training pipeline – see below.

LoRA (Low-Rank Adaptation) is a fine-tuning technique for large neural networks introduced by Microsoft researchers in 2021. Instead of updating the billions of weights in a model – which is expensive, slow, produces huge files – LoRA freezes the original weights and trains small "adapter" matrices alongside them. The practical upshot is that a LoRA for a 7B-parameter model might be 10-200 MB instead of 14 GB, and can be trained in hours on a single consumer GPU rather than days on a cluster. The LoRA can also be swapped in and out at inference time. This technique is commonly used now to fine-tune open-weight models such as Llama and Qwen. Variants include QLoRA (Quantized LoRA) and merged LoRAs (where the changes are baked back into the base weights).

A lightweight alternative to fine-tuning is prompt-based adaptation. This is the process of crafting system prompts to steer model behavior without changing its weights. This is simpler but less powerful than fine-tuning. (The system prompt is a set of instructions provided to the model before a conversation begins. The system prompt is typically hard-coded by application developers, rather than being something the end user inputs. Ollama’s Modelfiles are an example of prompt-based adaptation.)

Reinforcement learning from human feedback (RLHF) is a post-training technique that aligns a model’s behavior with human preferences — making it safer, more helpful, and less likely to produce harmful outputs.

The process works in two stages. First, human evaluators compare pairs of model outputs and select which is better. This data is used to train a separate reward model that learns to predict human preferences. Second, the main model is trained via reinforcement learning to maximize scores from the reward model. Because the second stage optimizes against the reward model rather than humans directly, it scales without requiring continuous human input.

RLHF is how base models — which simply predict the next token — are transformed into helpful assistants that follow instructions. Most frontier models use RLHF or a close variant (such as constitutional AI, RLAIF, or DPO) as part of their post-training pipeline.

Because training is a one-time process, a model’s knowledge is frozen at the date its training data was collected. The model has no awareness of events, publications, or other changes in the world that occurred after that date. This is known as the knowledge cutoff. It is one of the core limitations of LLMs, and the primary motivation for augmenting LLMs with techniques such as retrieval-augmented generation (RAG) – see below.

LLMs do not process text character-by-character or word-by-word. Instead, their base unit is a token.

A token is a chunk of text, typically a word, part of a word, or a punctuation character. Common words usually fit into a single token. Rarer or longer words may be split into multiple tokens. As a rough guide, one token is approximately four characters, or three-quarters of the average word in English.

Tokens are the unit of measurement for both input (the prompt) and output (the response). LLM pricing is typically quoted per million tokens (eg. $1.50/1M input, $3.25/1M output). A model’s context window (see below) and other capabilities are also measured in tokens.

Before a model can process tokens, the tokens must first be converted into a numerical form. An embedding is this numerical representation of a token. An embedding is a vector – a list of floating-point numbers – where each value captures some aspect of the token’s meaning.

Tokens with similar meanings produce vectors that are close together in this multi-dimensional space. This is how a model captures semantic and syntactic relationships between words.

Weights refer to a model’s learned parameters – the billions of numbers that encode everything the model learned during training. Weights determine how a model transforms inputs into output.

Small changes in weights, eg. through fine-tuning, can dramatically alter the behavior of a model.

The embedding lookup table is one part of a model’s weights.

An LLM with 100 billion parameters has roughly that many weights, stored either in memory or on disk.

Inference is what an LLM does to generate output. This process is entirely distinct from training, fine-tuning, and quantization, which are processes to create a model in the first place. Inference is how a model, once made, works.

The term comes from logic and statistics, where "inference" means drawing a conclusion from available information. In LLMs, inference is the process of calculating the next token based on past learning.

When you type a prompt into an LLM and get a response back, the entire process that happens behind-the-scenes – loading the model, tokenizing your input, generating each token of the output – is inference.

The model’s weights – the billions of numerical parameters that encode everything it learned during training – are fixed. Inference is the process of running data through those fixed weights to produce an output.

Compared to training, inference is relatively cheap. Depending on the model size, it can be done on a single workstation GPU. Available VRAM is the primary constraint on the size of the model you can run, and the performance – the throughput, measured in tokens per second – of that model.

Quantization is a technique for reducing the numerical precision of a model’s weights, for example representing values with 4-bit integers instead of 16-bit or 32-bit floats. This reduces memory footprint and speeds up inference, at the cost of a small reduction in output quality.

Quantization is what makes it practical to run large models on consumer hardware with limited VRAM.

Distillation (or knowledge distillation) is a technique for transferring the capabilities of a large, capable model into a smaller, cheaper one. The large model is the teacher and the smaller model is the student. Rather than training the student from scratch on raw data, it is trained to imitate the teacher — either by matching the teacher’s full output probability distribution (the "soft labels", which carry more information than a single correct answer alone), or simply by training on large volumes of synthetic data generated by the teacher.

The result is a model that is far smaller and faster to run, yet retains much of the teacher’s capability. This is how many light models are produced (see Light models, below): Google’s Gemini Flash models, for example, are distilled from the larger Gemini Pro models.

Distillation can be done in two broad settings:

Black-box distillation of a competitor’s proprietary model is contentious. It generally violates the terms of service of the model being copied, and has become a flashpoint in the industry. DeepSeek was widely suspected of distilling OpenAI’s models; the US has characterized the practice as industrial espionage when done by Chinese labs; and in April 2026, during the Musk v. Altman trial, Elon Musk acknowledged that xAI had distilled OpenAI’s models while training Grok. Once "a common practice in the AI world", distilling others' models is becoming less and less accepted.

The context window – or simply the "context" – is the maximum number of tokens an LLM can process in a single interaction. It is the combined total of the system prompt, conversation history, and the current input and output. Text outside the context window is not visible to the model.

Context window size is one of the most impactful characteristics of models on the user experience. A larger context window allows a model to consider more of a document, a longer conversation history, or a larger codebase at once.

Modern frontier models support context windows of 128K tokens or more. Some extend to 1M tokens and beyond.

During inference, the attention mechanism requires each token to be compared against every other token in the context. Doing this from scratch for every new token generated would be enormously wasteful. The keys and values for all previously processed tokens would have to be recomputed identically at each step.

A key-value cache solves this by storing the computed key and value vectors for every token already processed, so they only need to be calculated once and can be reused as each subsequent token is generated. This trades memory for compute, and is what makes token-by-token generation practical at interactive speeds.

The cost is that KV caches grow linearly with context length and model depth. A large context window requires proportionally more VRAM to hold its KV cache. This is one reason why large context inference is memory-intensive even when the model itself fits comfortably in VRAM.

This is also the problem that vLLM’s PagedAttention addresses (see the section on inference engines, below). It manages KV cache memory more efficiently to support higher request concurrency.

When generating a response, an LLM does not simply pick the most probable next token at every step. That would produce repetitive, deterministic output. Instead, the model samples from a probability distribution of possible next tokens.

Temperature is the parameter that controls how sharp or flat this distribution is. At low temperatures (close to 0), the model almost always picks the highest-probability token, producing focused and predictable output. At higher temperatures, lower-probability tokens become more likely to be chosen, producing more varied and creative – but also less reliable – output.

For factual or code-generation tasks, a low temperature is usually preferable. For creative writing, a higher temperature produces more interesting results.

Most LLMs expose temperature as a configurable parameter.

LLMs generate text by predicting the most statistically plausible next token, not by reasoning from facts. As a result, they can produce outputs that are fluent and confident but factually incorrect – a phenomenon known as hallucination.

A model may fabricate citations, invent plausible-sounding names, or state incorrect information as though it were certain. These are all examples of hallucinations.

Hallucinations are an inherent limitation of how LLMs work, not a bug that can be fully fixed. The practical mitigations are:

Evals (short for evaluations) are structured tests used to measure how well an LLM or LLM-powered application performs on specific tasks. They are the AI equivalent of unit tests and regression tests.

An eval runs the model through a predefined set of inputs and compares its responses against expected outputs or criteria — such as factual correctness, safety, helpfulness, tone, or task completion. This allows developers to quantify progress, catch regressions when switching models or changing prompts, and guide improvements over time.

Evals are particularly important in production applications, where the quality of model outputs directly affects user experience. They are often cited as the most critical, yet most overlooked, practice in building reliable AI products.

Teams typically write their own evals tailored to their use case, but open-source frameworks supply ready-made harnesses and shareable test suites. OpenAI Evals, for example, is a framework for evaluating LLMs paired with an open registry of reusable evals that anyone can contribute to. The public benchmarks described under Leaderboards and benchmarks (below) are, in effect, standardized evals run across many models.

A core limitation of LLMs is that their knowledge is frozen at the point of training. By default, models cannot access new information or browse the web. In addition, LLMs are prone to hallucination — generating fluent, confident-sounding responses that are factually incorrect — particularly when asked about topics outside their training data or at the edges of their knowledge.

Retrieval-augmented generation (RAG) is a technique that addresses these issues. It combines an LLM with an external knowledge source, which is retrieved at inference time and loaded into the context window.

In a RAG system, a user’s query is used to search a knowledge base (a vector database, a search index, or live web results). The most relevant results are fetched and injected into the model’s context alongside the original query. The model then generates a response grounded in that retrieved content, rather than relying solely on what it learned during training.

RAG is widely used to build knowledge-aware applications – internal document Q&A systems, customer support bots, and AI search engines. Perplexity is a prominent consumer example. It uses RAG over live web search to answer questions with cited sources in real time. See AI assistants for other examples.

Frameworks such as LangChain and LlamaIndex are commonly used to build RAG pipelines, providing the retrieval, chunking, and orchestration plumbing so developers do not have to wire it together from scratch.

Both training and inference are computationally intensive and rely heavily on GPUs, which excel at the massively parallel matrix operations that underpin neural networks.

The dominant frameworks for programming GPUs are listed below. These are the software stacks that allow programs like llama.cpp or vLLM (see below) to access GPUs for general compute, rather than graphics workloads.

Inference engines are the low-level runtimes that actually execute a model – loading weights into memory and running the computations that produce each token. Model managers like Ollama and LM Studio (see below) are built on top of them.

The two most widely used inference engines for open-weight models are:

Several independent sites aggregate model rankings, pricing, speed, and availability – useful for comparing options before committing to a model:

Specific benchmark suites measure particular capabilities:

The model tables on this page also cite academic benchmarks, including GPQA Diamond (graduate-level science reasoning), AIME (competition mathematics), MMLU / MMMU (broad multi-subject knowledge), ARC-AGI-2 (abstract reasoning), Humanity’s Last Exam (HLE) (expert-level frontier questions), and GDPval (economically-valuable knowledge work).

The single most useful question when choosing a model is: what kind of task is this? Each task type stresses a different capability — language quality, step-by-step reasoning, code quality, factual grounding, raw speed — and that dominant capability, not the model’s headline benchmark score, is what should drive the choice. The aim is to match the model to the work, rather than defaulting to the biggest model for everything (wasteful) or the cheapest (unreliable).

Two principles underpin the table:

Beyond size and cost, models differ significantly in what they are designed to do. The main capability types are:

Base models are the raw, pre-trained models before any instruction tuning or RLHF is applied. They predict the next token without any conversational or helpfulness alignment, and are rarely used directly. They serve as the starting point for fine-tuning. Examples: Llama 3 (base), Mistral 7B (base).

Chat models are instruction-tuned to follow natural-language instructions and hold multi-turn conversations. These are the general-purpose workhorses — the models behind ChatGPT, Claude, and Gemini. Almost all frontier and mid-tier models fall into this category. Examples: GPT-4o, Claude Sonnet 4.6, Gemini 2.5 Pro, Llama 3.3 70B Instruct.

Flash models: These are optimized for producing a response in a fraction of the time it takes other models. They often rely on shortcuts like caching or pre-computing certain results to speed up generation.

Code completion models are optimized for programming tasks and typically deployed as IDE autocomplete backends rather than chat interfaces. A distinctive capability is fill-in-the-middle (FIM): generating code that fits between an existing prefix and suffix, which is how IDE autocomplete works. Examples: Codestral, Code Llama, Qwen2.5-Coder.

Reasoning models spend extra compute working through a problem step-by-step — producing an internal chain of thought — before arriving at a final answer. They trade higher latency and token cost for substantially better performance on complex mathematical, logical, and multi-step problems. See the Reasoning models section below. Examples: OpenAI o3, DeepSeek-R1, Claude with extended thinking.

Embedding models convert text into dense numerical vectors that represent the semantic meaning of the input. Unlike generative models, they produce a fixed-length vector rather than text. These vectors are used to power semantic search, RAG pipelines, clustering, and classification. Examples: OpenAI text-embedding-3, Cohere Embed, Nomic Embed, BGE.

Reranker models are used as a second pass in RAG pipelines. After an initial retrieval step fetches candidate documents, a reranker scores each candidate’s relevance to the query more precisely, so the most useful results are surfaced before being passed to the generative model. Examples: Cohere Rerank, BGE Reranker, cross-encoder models.

Multimodal models accept inputs — or produce outputs — beyond text, such as images, audio, and video. Most modern frontier models are multimodal to some degree. Examples: GPT-4o (image input), Gemini 3 Pro (image, video, and audio input and output), Claude Opus 4.7 (image input).

A frontier model is a model – or, more commonly, a family of models – that represents the current state of the art.

Frontier models are typically the largest models, trained on the most data, with the best performance across a broad range of tasks. They are developed by well-resourced AI labs and accessed via paid APIs rather than downloaded and run locally.

Frontier models are usually proprietary – which means their weights are not publicly released. The companies that develop them invest heavily in safety evaluation and alignment – the process of steering model behavior to be helpful, honest, and to avoid harmful outputs.

Modern frontier models are increasingly multimodal – capable of processing not just text but also images, audio, and video as inputs, and in some cases generating them as outputs.

Examples of frontier model families include:

Other notable proprietary providers include Cohere, whose Command family targets enterprise use cases with strong retrieval-augmented generation support (Cohere also produces the widely-used Embed and Rerank models – see Capability types, below). The leading Qwen (Alibaba), DeepSeek, Kimi (Moonshot AI), and GLM (Z.ai) families also reach near-frontier performance while releasing open weights – covered under Open-weight models, below.

As of March 2026, OpenAI GPT 5.2 is the flagship model in OpenAI’s GPT lineup. It’s a very well-rounded model – it can do a bit of everything (multimodality) and is good at chaining together multiple actions. (The following stats are sourced from Artificial Analysis.)

For Anthropic, as of May 2026 its Claude Opus 4.7 is its flagship model. It’s another all-rounder, but specializes in text and code generation, rather than aiming for multimodality. Unlike GPT 5.2, it cannot do image generation directly. It is also currently the most expensive and slowest of the frontier models, but it is highly regarded for its coding abilities.

But, as of May 2026, it is Grok 4.1 that is widely considered to be the most capable of the frontier models. It is fast, has a very large context window, and is cheap relative to other frontier models. It is noted for strong reasoning across diverse tasks and a highly permissive output style.

As of May 2026, Gemini 3 Pro is the current flagship in Google’s Gemini series. Its performance is pretty on par with the rival flagship models, but it has two things going for it: a large context window, and (what really makes it stand out) multimodality capabilities. It can analyze and generate all kinds of media, not only text.

The mid-tier models are the workhorses that you will probably use 80% of the time for general tasks. They offer a good balance between capability, size, speed, and cost. These are the defaults for a lot of agentic workflows.

Examples include:

Claude Sonnet 4.6 is a mid-tier model from Anthropic that is particularly strong at coding tasks. It’s cheaper than Opus 4.7, the flagship model, but still perfectly capable at advanced programming tasks, including building greenfield projects from scratch.

Light models are faster and cheaper, but less capable than the frontier/flagship models. Yet some of these are still incredibly capable. Gemini 3 Flash, for example, maintains most of the capabilities of the flagship Gemini 3 Pro model. This is achieved by knowledge distillation, whereby a smaller student model is trained to replicate the behaviour of a larger teacher model, producing a model that is smaller in size but retains much of the larger model’s capability.

Reasoning models are a category of LLM that spend additional compute "thinking" through a problem before producing a final response. Rather than generating an answer immediately, they produce an internal chain-of-thought – working through steps, checking logic, and reconsidering – before arriving at a conclusion. This makes them substantially better at complex mathematical, logical, and multi-step problems, at the cost of higher latency and token usage.

Notable reasoning models include OpenAI’s o1 and o3 series and the open-weight DeepSeek-R1.

Where reasoning models can think before answering, an effort level (sometimes "reasoning effort" or "thinking level") is the dial that controls how much they do so. It drives adaptive reasoning: rather than a fixed amount of thinking, the model decides whether and how much to reason on each step, based on the complexity of the task at hand. Lower effort is faster and cheaper for straightforward work; higher effort buys deeper reasoning on complex problems.

An effort level is a behavioral signal, not a strict token cap. It nudges the model toward more or less deliberation — trading token spend against capability — rather than hard-limiting the number of tokens it may use.

The dial is exposed differently by different tools — for example as a reasoning_effort parameter on some APIs, or as a configurable effort level in coding agents such as Claude Code. As a rough guide for picking a level:

This is the same idea as the thinking level column in the task-matching table above: effort is a separate dial from model tier. Turning it up helps on problems with a verifiable chain of logic (maths, code, planning), while adding latency and cost for little benefit on language tasks.

Specialist models are fine-tuned for a narrow domain or task, rather than general-purpose use. Perplexity’s Sonar model is an example.

Open-weight models have publicly released weights, meaning anyone can download and run them. This is distinct from open-source, which would also require the training data and code to be public – most open-weight models do not meet that bar.

Licensing terms vary. Some permit commercial use, others restrict it.

The main consideration when choosing between open-weight models is the model size. Model size is commonly expressed as the number of trainable parameters, in billions – for example, a 7B model has 7 billion parameters. More parameters generally means greater capability but higher memory and compute requirements. A 7B model can run comfortably on a consumer GPU; a 70B model requires a high-end workstation or server; while a 405B model requires multiple high-end GPUs.

Open-weight models, which can be downloaded and run on your own computer or server, are a good choice for privacy-sensitive use cases, offline environments, or experimenting with open models without incurring API costs.

Open-weight models can also be customized through further fine-tuning and through RAG. Perplexity’s Sonar model is an example of an open-weight model that has been fine-tuned for specific use cases.

Hugging Face is the primary hub for discovering and downloading open-weight models, hosting over two million models contributed by individuals, research labs, and major organizations. The other major resource in this space is Ollama’s model database, which is a more curated library of models that are proven to be stable with the Ollama model manager.

There are five broad patterns for running and accessing LLMs, spanning a spectrum from full local control to fully managed services:

Model gateways provide a single, unified API endpoint, and/or a GUI or other user interface, to access models from multiple hosted LLM providers. Rather than integrating directly with each provider’s SDK and managing separate authentication, response formats, and failover logic, developers integrate once with the gateway, which handles routing, fallback, cost optimization, and observability across providers.

Model managers are primarily designed for running open-weight models locally. They handle downloading of models, managing hardware resources, and serving a local inference endpoint – typically an OpenAI-compatible API on localhost. Essentially, model managers wrap an inference engine with all the additional tools needed to managing and running large-language models.

By contrast, model API gateways (above) are middleware for routing requests to remote, hosted models across multiple cloud providers.

The two most popular model managers are:

Ollama is designed to run LLMs as a background service (daemon) on your machine. It provides a clean CLI for managing models and a local API (compatible with OpenAI’s API) that allows other applications — like IDE plugins, web UIs, or custom scripts — to communicate with the models you’ve downloaded. Ollama is also a model hosting service. It maintains a curated library of models, many of which have been optimized using quantization, allowing large models to run on consumer hardware.

LM Studio is a desktop GUI application that provides an "all-in-one" visual interface for discovering, downloading, and chatting. While it also provides a local server for API access, like Ollama, its primary value is its user-friendly dashboard that allows you to tweak hardware settings (like GPU offloading), monitor system resources in real-time, and chat with models without needing a terminal or other front-end integration. LM Studio integrates with the Hugging Face model library, allowing easy access with every available open-weight model.

Both Ollama and LM Studio use llama.cpp as their underlying inference engine.

A related but distinct tool is Open WebUI — a self-hosted, extensible web front-end for LLMs rather than a model manager in its own right. It provides a polished, ChatGPT-like browser interface and connects to a backend (most commonly Ollama, or any OpenAI-compatible API) to serve the actual models. It is often run alongside Ollama to give a local model stack a clean, multi-user web UI.

For training data, Argilla (now part of Hugging Face) is an open-source data curation and labelling platform for AI engineers and domain experts to maintain high-quality data sets, while distilabel is for synthetic data generation.

Other supporting tools include Weights & Biases, a software-as-a-service for tracking experiments and monitoring model checkpoints during the iterative development of models. MLFlow is similar but is fully self-hostable.