What Is a Foundation Model? A Simple Guide
GPT, Claude, Llama… the name that spans the models at the center of today’s AI is “foundation model.” Here’s this often-used but fuzzy term, from concept to use.
What is a foundation model?
A foundation model is an AI model that undergoes large-scale pretraining on broad data and can then be adapted to a wide range of tasks. The term was first defined in 2021 by Stanford’s CRFM (Center for Research on Foundation Models).
The key is the “foundation” in the name. It isn’t built for one specific task — it’s a model that serves as the base for many applications. Just as you lay a foundation before adding various rooms, you train a foundation model first, then build translation, summarization, chatbots, and more on top of it.
How it’s made: pretraining and adaptation
A foundation model works in two broad stages.
Stage 1 — Pretraining. The model learns from vast data (web text, images, etc.). Instead of humans labeling every answer, it mainly uses self-supervised learning, learning from the structure of the data itself — for example, “predict the next word,” using patterns already in the data as signal. This is the most expensive stage, demanding enormous compute and data.
Stage 2 — Adaptation. The pretrained model is adjusted for a specific use: fine-tuning with additional data, prompting with examples, or transfer learning. This stage takes far fewer resources than pretraining.
Foundation models and LLMs
To untangle two often-confused terms:
An LLM (large language model) is a type of foundation model. Foundation model is the broader concept — spanning language, images, audio, and multimodal — and the language-specialized subset is the LLM. Stanford’s researchers coined “foundation model” precisely because “LLM” was too narrow, being limited to language.
Term | Scope |
|---|---|
Foundation model | Broad — language, image, audio, etc. |
LLM | The language-specialized subset |
How enterprises use foundation models
Most enterprises don’t build foundation models from scratch. Pretraining needs enormous GPUs and data, so only a few large companies and labs do it. Instead, enterprises typically take an existing (e.g., open-source) foundation model and fine-tune it on their own data, apply it via prompting, or connect their own documents. In other words, most real work happens at the “adaptation” stage, not “pretraining.”
But whether fine-tuning or serving inference, working with a foundation model ultimately requires GPU infrastructure. The larger the model, the more GPU memory and compute it takes — and when many teams fine-tune and serve different models, GPU resource management gets complex.
FAQ
Q. Are foundation models and LLMs the same?
No. An LLM is the language-specialized type of foundation model. Foundation model is broader, including images, audio, and more.
Q. Do we need to build a foundation model ourselves?
Usually not. Pretraining takes enormous resources and few do it; enterprises typically adapt existing models via fine-tuning and prompting.
Q. What is self-supervised learning?
Learning from the data’s own structure (e.g., the next word) without human labels — letting you use vast unlabeled data.
Q. How much GPU do we need to use foundation models?
It depends on the use. Fine-tuning and inference take less than pretraining, but still need substantial GPU resources depending on model size and concurrent users.
Conclusion: in the age of use, the key is infrastructure operations
Foundation models are now the “base” for most AI applications. And for most enterprises, what matters isn’t building one from scratch but efficiently fine-tuning and serving existing models for their own purposes.
Where many teams work with different foundation models, GPU resource management becomes the key challenge. TEN’s AIPub partitions GPUs into blocks so multiple models and workloads share resources, isolates resources and permissions per team and project, and measures usage so you can manage who uses which GPU and how much. What an organization using foundation models ultimately needs is the GPU operations layer beneath them.
To efficiently operate multi-team foundation-model fine-tuning and serving, explore AIPub.
👉 Learn more about AIPub: https://ten1010.io/
References
Bommasani et al., “On the Opportunities and Risks of Foundation Models,” Stanford CRFM (2021)
Stanford HAI, Center for Research on Foundation Models
Stanford CRFM, Foundation Model Transparency Index