Model access layer

Although it is possible to interact with a large language model directly, for most use cases you reach it via an access layer, which handles authentication, response formatting, and other details specific to a particular provider or use case.

There are two useful lenses on your options. The first is where the model runs, and who operates the infrastructure — the deployment patterns described next. The second is the tooling you use to reach the model: broadly, model gateways (for hosted models) and model managers (for running open-weight models locally), each covered in its own section below.

Deployment patterns

There are five broad patterns for running and accessing LLMs, spanning a spectrum from full local control to fully managed services:

• Local: The model runs on your own machine, with direct access to your data. Model managers such as Ollama LM Studio, and Open Web UI (see Model managers, below) handle downloading, hardware management, and serving a local API. Best for privacy, offline use, and experimentation, with no per-token cost — but you are limited by your own hardware.
• VPS + rented GPU: You run your own models on infrastructure you rent, then expose them over an API (for example, Ollama behind an Nginx reverse proxy). Hosting providers include DigitalOcean and RunPod. You keep full control over the model and the data path while offloading the hardware, but you own the operations — scaling, uptime, and security. Higher up-front costs, but cheaper in the long term than managed cloud.
• Managed cloud: Your model runs on a cloud provider’s managed ML platform, which handles provisioning, scaling, and serving. Examples include Amazon SageMaker, AWS Bedrock, Google Vertex AI, Azure ML, Modal, and Beam. Well suited to organizations already invested in a particular cloud, and to enterprise governance and data-residency requirements. Also good for unpredictable workloads, and for deploying fine-tune custom models with autoscaling. Trade-offs: complex setup, steep learning curve, and high costs and low scale.
• Managed inference API: You call a provider’s hosted endpoint with just an API key, choosing from their model library and paying per use or by subscription. Examples include Groq, Together AI, Fireworks AI, Replicate, OpenRouter, and the Hugging Face Inference API. There is no infrastructure to manage, but you have the least control over where and how the model runs. Great for indie hackers and shipping fast. Free tiers are available, and you can experiment with lots of model variants. (Model gateways, below, are the tooling for this pattern.)
• Edge / on-device: The model is embedded directly into your application and ships to your users' devices, running entirely client-side with no server. Tooling includes WebLLM, Apple MLX, MLC LLM, llama.cpp, ONNX Runtime, and Transformers.js. Best for privacy, low latency, and offline support, but constrained by the capabilities of the user’s device — typically, you’e looking at ultra-light models, 1-3B parameters maximum.

Model gateways

Model gateways provide a single, unified API endpoint, and/or a GUI or other user interface, to access models from multiple hosted LLM providers. Rather than integrating directly with each provider’s SDK and managing separate authentication, response formats, and failover logic, developers integrate once with the gateway, which handles routing, fallback, cost optimization, and observability across providers.

• OpenRouter — Unified API for 300+ models across multiple providers, with per-request routing, cost tracking, and usage rankings.
• Anannas — OpenAI-compatible API gateway with intelligent routing, automatic failover, and real-time cost dashboards.
• Hugging Face Inference API — Managed inference endpoints for models hosted on Hugging Face, with serverless and dedicated deployment options.
• Nvidia’s Build site includes free endpoints to access open-weight models.
• Perplexity — AI-native search engine and general-purpose AI assistant. Provides access to a large number of models, and you can even experiment with multiple models simultaneously, feeding them the same inputs.
• LiteLLM — Open-source gateway that exposes 100+ providers (OpenAI, Anthropic, Gemini, Bedrock, Azure, and more) through a single OpenAI-format API. Available as a Python SDK for direct integration, or as a self-hosted proxy server (its "AI Gateway") for organization-wide access, with virtual API keys, per-project cost tracking, automatic fallbacks, and load balancing. Unlike the hosted services above, you run it yourself.
• OpenCode Zen — Curated gateway from the OpenCode team offering a vetted set of coding-benchmarked models (OpenAI, Anthropic, Google, Qwen, DeepSeek, and others) with pay-as-you-go per-token pricing and bring-your-own-key support. Integrates as an optional provider inside the OpenCode coding agent.

Model managers

Model managers are primarily designed for running open-weight models locally. They handle downloading of models, managing hardware resources, and serving a local inference endpoint — typically an OpenAI-compatible API on localhost. Essentially, model managers wrap an inference engine with all the additional tools needed to managing and running large-language models.

By contrast, model API gateways (above) are middleware for routing requests to remote, hosted models across multiple cloud providers.

The two most popular model managers are:

• Ollama
• LM Studio

Ollama is designed to run LLMs as a background service (daemon) on your machine. It provides a clean CLI for managing models and a local API (compatible with OpenAI’s API) that allows other applications — like IDE plugins, web UIs, or custom scripts — to communicate with the models you’ve downloaded. Ollama is also a model hosting service. It maintains a curated library of models, many of which have been optimized using quantization, allowing large models to run on consumer hardware.

LM Studio is a desktop GUI application that provides an "all-in-one" visual interface for discovering, downloading, and chatting. While it also provides a local server for API access, like Ollama, its primary value is its user-friendly dashboard that allows you to tweak hardware settings (like GPU offloading), monitor system resources in real-time, and chat with models without needing a terminal or other front-end integration. LM Studio integrates with the Hugging Face model library, allowing easy access with every available open-weight model.

Both Ollama and LM Studio use llama.cpp as their underlying inference engine.

A related but distinct tool is Open WebUI — a self-hosted, extensible web front-end for LLMs rather than a model manager in its own right. It provides a polished, ChatGPT-like browser interface and connects to a backend (most commonly Ollama, or any OpenAI-compatible API) to serve the actual models. It is often run alongside Ollama to give a local model stack a clean, multi-user web UI.

See also

• Large language model (LLM): The broader concept these access patterns and tools serve.
• Open-weight model: Models with publicly released weights, most commonly run through the model managers described above.