Foundation Models

The building blocks of generative AI

Overview

Foundation models refer to large AI models that are trained using vast datasets to perform various tasks. These models mark a departure from the smaller task-specific models that previously dominated the AI landscape. 

First popularized by the Stanford Institute for Human-Centered Artificial Intelligence, the term “foundation models” refers to how these large models serve as the foundation for developing more complex and refined models across diverse applications. For example, GPT serves as a foundation model and successors like GPT-3 and GPT-4 are harnessed by hundreds of startups and established companies to develop task-specific models for applications ranging from content creation to marketing plan generation. 

This hub includes organizations that have built large AI models, whether for in-house product development, to drive AI research, or for commercialization via third-party API access and other means. Due to the prohibitive costs of training and running large AI models, the space remains strongly incumbent-driven and is led by Big Tech firms, followed by AI research labs and communities.

Among startups, OpenAI is a frontrunner in terms of commercializing foundation models. Big Tech firms like Google and Meta had previously refrained from releasing foundation models for public use and, instead, leveraged them to enhance the functionality of in-house products. However, this posture has since shifted, with Google recently making its foundation models available via the PaLM API and Vertex AI platform, and Meta open-sourcing models like LLaMA. AI research labs and communities, such as EleutherAI and BigScience, tend to focus on open-sourcing AI development.

Note: Additional sections (market sizing, incumbents, etc.) can be provided on request.

Industry Updates

View all updatesicon

Use cases


Foundation models are currently deployed across a variety of industries to develop new applications and introduce new AI-powered features on existing products. As the first player to commercialize its foundation models, OpenAI's models remain the most widely used. For instance, OpenAI’s GPT-3 reportedly has over 300 applications under its belt, while its fine-tuned, coding-focused descendant, Codex, has over 70 applications. Other foundation model builders like Google, AI21 Labs, Anthropic, and Midjourney are now quickly following suit with the commercialization of their models.

Large language models, fine-tuned language models, and multimodal models remain the most popular among users. Many providers have yet to disclose customer use cases for newer model types like audio, video, and speech models. Some foundation models are also still at the research stage (for example, Meta's ImageBind model).

We have identified key use cases below:

Market Mapping


All foundation models are primarily segmented based on the distinctions in data type, specifically the type of input data (prompt) leveraged by a user and the type of output data generated by the model. For instance, LLMs use textual inputs to generate textual outputs, while video models use textual inputs to generate video outputs. LLM segments dominate the AI landscape as they are relatively easier to train and deploy than multimodal models, which must manage diverse data types. However, companies are increasingly developing fine-tuned, industry-specific, multimodal models capable of more complex and comprehensive tasks.

The Disruptors


The industry is dominated by key players like OpenAI and Anthropic, both backed by major incumbents. For instance, OpenAI received a massive USD 10 billion investment from Microsoft. In contrast, Anthropic has received investment from Google (committed to investing USD 2 billion in October 2023) and Amazon (invested USD 4 billion as of March 2024). Startups like Mistral AI, too, have raised substantial funds and established partnerships with incumbents like Microsoft. However, OpenAI is the only player that operates across multiple segments, offering models for text, images, videos, speech, and music.

Funding History

Competitive Analysis


Filter by a segment or companies of your choice
expand
 
Loading...
Loading...
Loading...
Loading...
Product Overview
-
Loading...
Loading...
Loading...
Loading...
-
Loading...
Loading...
Loading...
Loading...
-
Loading...
Loading...
Loading...
Loading...
-
Loading...
Loading...
Loading...
Loading...
-
Loading...
Loading...
Loading...
Loading...
Product Metrics
-
Loading...
Loading...
Loading...
Loading...
-
Loading...
Loading...
Loading...
Loading...
-
Loading...
Loading...
Loading...
Loading...
-
Loading...
Loading...
Loading...
Loading...
-
Loading...
Loading...
Loading...
Loading...
Company profile
-
Loading...
Loading...
Loading...
Loading...
-
Loading...
Loading...
Loading...
Loading...
-
Loading...
Loading...
Loading...
Loading...
-
Loading...
Loading...
Loading...
Loading...
-
Loading...
Loading...
Loading...
Loading...

Notable Investors


?
Funding data are powered by Crunchbase
arrow
menuarrow
close

Contact us

Gain access to all industry hubs, market maps, research tools, and more
Get a demo

Overview

What are foundation models?

Foundation models are AI models that have been trained on historical training data and have grown in complexity to reach billions of parameters, enabling them to approach tasks with greater skill and accuracy than their smaller counterparts and predecessors. These models tend to be tens of gigabytes (GB) in size and are trained on vast amounts of data—sometimes at the petabyte (PB) scale. 
What makes a foundation model a “large” AI model is its number of parameters. Parameters are values or “settings” in machine-learning (ML) algorithms (i.e., the weights and coefficients that the model extracts and estimates from training data and uses to develop outputs). For example, language models train by adjusting parameters, blanking out words, and comparing their predictions with reality. As such, parameters represent the part of the model that has learned from historical training data. 
Generally, the more parameters a model has, the more information it can digest from its training data, the more accurate its predictions, and hence, the more sophisticated the model. Some large AI models may even be described as “neural networks” in terms of how they try to mimic the human brain through a set of deep-learning algorithms. For example, OpenAI’s GPT-3 (175 billion parameters) was the largest in the current generation of LLMs when it first emerged in 2020, but its successor—GPT-4, which was released in March 2023—is reportedly 10x larger, at 1.8 trillion parameters
Additionally, while most foundation models tend to have billions of parameters, a higher number of parameters does not necessarily lead to better performance outcomes, especially if models are undertrained. Google’s Chinchilla, a 70 billion-parameter model, is a case in point. Although 4x smaller than Google’s Gopher, it was trained on 4x more data. Researchers found that Chinchilla outperforms Gopher, GPT-3, Jurassic-1, and Megatron-Turing NLG across several language benchmarks. 
Click here to learn more
Get a demo

By using this site, you agree to allow SPEEDA Edge and our partners to use cookies for analytics and personalization. Visit our privacy policy for more information about our data collection practices.