LLM model families: closed APIs, open-weight, small and reasoning models
When you build with LLMs, you choose between four main kinds of model. Closed API models (like GPT, Claude and Gemini) give you the strongest general performance with nothing to host, but your data goes to the provider and you pay per token. Open-weight models (like Llama, Qwen, DeepSeek, Mistral and Gemma) can run on your own infrastructure, which gives you control and privacy, but you need the hardware and the team. Small language models are fast and cheap for narrow tasks, and reasoning models think before they answer, for hard multi-step problems. Many production systems combine several of them, routing each request to the right one.
Why this choice matters
Picking a model family is not only a technical decision. It decides how fast you can ship, what each request costs as you grow, where your users' data goes, how much control you have when something changes, and what team you need. Changing it later is possible, but it means re-testing your prompts, your evaluations and often your architecture, so it is worth understanding the options before you commit.
Closed API models
These are models you access only through the provider's API or app: OpenAI's GPT models, Anthropic's Claude, Google's Gemini, xAI's Grok and others. You can't download them, and you don't see how they were trained.
What they are good at
Speed to market is the biggest advantage. All you need is an API key and a payment method, and the provider takes care of the hardware, the infrastructure and the model itself. You can build and ship a first version of an application in days, and the strongest closed models are usually the best available for general tasks: writing, summarising, coding, reasoning and following complex instructions.
Snapchat's My AI is an example of this path. When it launched in 2023, Snapchat built it on OpenAI's models instead of training its own, and every message to My AI was answered by calling the provider's model.
What to watch for
- Your data goes to the provider. Read the data usage terms, and check whether they fit your industry's rules.
- Cost scales with usage. Pay-per-token is cheap at small scale and can become a large monthly bill when thousands of users send long requests.
- You don't control changes. Providers update and retire models, and a model update can change how your prompts behave.
- You depend on their availability and rate limits.
The data question is real. In 2023, Samsung restricted employees' use of generative AI tools after engineers pasted internal source code into ChatGPT. Whatever a provider's policy says about using data for training, the safest assumption is that sensitive data shouldn't go to a public tool unless your company has an agreement that covers it.
Open-weight models
Open-weight models publish their trained parameters, so you can download them and run them anywhere: on your laptop, your own servers, or a cloud GPU. Well-known families include Meta's Llama, Alibaba's Qwen, DeepSeek, Mistral, Google's Gemma and Microsoft's Phi.
People often call these open-source models, but most are more accurately open-weight: you get the weights and a license to use them, but usually not the training data or the full training code. The licenses also differ, and some restrict commercial use above a certain size or for certain purposes, so read the license before you build a product on one.
What they are good at
- Control and privacy. The data never leaves your infrastructure, which is often required in healthcare, finance, defence, government and law. For on-premises deployments, open-weight models are usually the only option.
- Customisation. You can fine-tune them on your own data, which a closed API either doesn't allow or allows only in limited ways.
- Predictable cost at high volume. Once you pay for the hardware, the cost per request doesn't grow with every token the way an API bill does.
- No surprise changes. The model doesn't change unless you change it.
What to watch for
The hidden cost is everything around the model.
- Hardware: serious models need GPUs. A single high-end data centre GPU like an NVIDIA H100 costs tens of thousands of dollars to buy, or a significant hourly rate to rent, plus networking, storage, power and cooling if you run your own.
- Talent: you need engineers who can deploy, optimise, monitor and update the model, and they are expensive and in demand.
- Operations: monitoring, security patches, scaling, and handling the model's behaviour over time all become your job.
So open-weight models are usually more expensive at the start and can become cheaper at high, steady volume. At low volume, the API is almost always cheaper.
Running one on your laptop
You can try open-weight models locally with tools like Ollama, which downloads and runs a model with one command, and Open WebUI, which gives it a chat interface in your browser.
ollama pull llama3.1:8b
ollama run llama3.1:8b
An 8 billion parameter model, compressed to 4 bits per parameter, is about 5 GB and runs on a modern laptop, slowly on a CPU and much faster with a good GPU or Apple Silicon. It won't match the best closed models, but for many narrow tasks it is good enough, and nothing leaves your machine.
Small language models
Small language models (SLMs) are models with fewer parameters, roughly under 10 billion, sometimes under 1 billion.
They are fast, cheap to run, can run on a phone or a laptop, and are easy to fine-tune. On broad tasks they are weaker than large models, but on a narrow, well-defined task, especially after fine-tuning, a small model can match or beat a much larger general model at a fraction of the cost.
Good uses for small models:
- Classification and routing: deciding what kind of request this is and where to send it.
- Filters and guardrails: checking inputs and outputs for prompt injection, personal data or policy violations before and after the main model runs.
- Extraction of a fixed set of fields from a known type of document.
- On-device features, where latency and privacy matter.
Reasoning models
Reasoning models are trained to think through a problem step by step before giving the final answer, using extra tokens for that thinking. OpenAI's o1 in 2024 brought them into wide use, and DeepSeek-R1 in early 2025 showed an open-weight version, and many model families now have a reasoning mode.
They are much better at multi-step problems like maths, complex code, planning and analysis, and slower and more expensive for everything else. The reasoning models page covers when they are worth it.
Other kinds you will meet
- Embedding models turn text into vectors for search and retrieval, and they are separate from the models that generate text. See embeddings.
- Multimodal models read images, audio, video and documents as well as text. See multimodal models.
- Specialised models are trained or fine-tuned for one domain, like code, medicine or law.
Base, instruct and chat models
Open-weight families usually publish more than one version of each model.
A base model has only been pre-trained on text, so it continues text rather than following instructions. An instruct or chat model has been further trained to follow instructions and hold a conversation, and it is what you want for almost every application. Base models are mostly useful as a starting point for your own fine-tuning.
How to read a model name
Open model names pack a lot of information. In a name like Qwen3-235B-A22B-Instruct, 235B means 235 billion parameters in total, A22B means about 22 billion are active for each token, and Instruct means it is tuned to follow instructions.
The active-parameter part tells you it is a mixture-of-experts (MoE) model. Instead of using every parameter for every token, an MoE model has many expert sub-networks and a router that sends each token to a few of them. It needs enough memory to hold all the parameters, but each token only costs the compute of the active ones, which is why MoE models can be large and still fairly fast.
You will also see quantization labels like Q4 or 4-bit, which mean the weights are stored with fewer bits to save memory, usually with a small loss of quality.
Proprietary data is your advantage
There are two kinds of advantage that push companies toward their own models.
The first is defensive: compliance. If you work under rules like HIPAA for health data in the US or GDPR in Europe, or with government or defence clients, you may not be allowed to send data to a public API at all.
The second is offensive: customisation with data nobody else has. A firm with 20 years of its own case files, research or customer conversations can adapt a model to that knowledge in a way no generic API can match. BloombergGPT is a well-known example: in 2023, Bloomberg trained a 50 billion parameter model on a mix of general text and its own large archive of financial data, aiming for a model that understood finance better than general models of the time. It also shows the cost of this path, because general models improved quickly afterwards, and keeping a custom model ahead of them is a continuous investment.
Using more than one: the hybrid fleet
You don't have to choose only one. Many mature systems use a mix and route each request to the model that fits it.
For example, a legal assistant might send simple, general questions to a cheap API model, send anything involving client documents to a fine-tuned model running on-premises, and send the hardest drafting tasks to a strong reasoning model. A router in front decides where each request goes, using simple rules, a small classifier model, or both.
The goal is to use the right model for each job, balancing cost, quality and security, instead of paying for the most powerful model for every "hello".
From the floor
I ran a webinar on this exact decision, cloud APIs or open-source LLMs, for people who make these choices in their companies. We went through it as a strategy question rather than a technical one, because the deciding factors are speed to market, cost as you scale, control and privacy, and the performance you need, and I asked the attendees which of those was their primary concern, because that answer decides the rest.
In my own work, we chose open-weight models for a knowledge engine product built for enterprises with large internal knowledge bases. The whole solution runs on-premises, so the client's data never leaves their infrastructure, and that need for control is what decided the model family.