EleutherAI

EleutherAI, formerly known as LibreAI, started as a Discord group of programmers which later transitioned into a non-profit research institute in early 2023. While its primary focus has shifted toward AI ethics, alignment, and interpretability, it also aims to continue maintaining and supporting its existing foundation models inspired by GPT-3, each with unique capabilities:

1. GPT-Neo: Released in March 2021, GPT-Neo is an open-source GPT-3-style language model, which serves as a foundational tool for natural language processing and generation tasks. GPT-Neo models are trained on the Pile, a 825 GiB open-source dataset of human-authored text compiled by EleutherAI for use in large-scale language modeling. It is available in parameter sizes ranging from 125 million to 2.7 billion.

2. GPT-J: Also released in March 2021, GPT-J is another open-source GPT-3-style language model, notable for its substantial parameter size of six billion. The solution is trained on a combination of the Pile dataset and other sources and is ideal for applications requiring larger language models.

3. GPT-NeoX: Launched in February 2022, GPT-NeoX is reportedly the first model trained on CoreWeave GPUs using the internally developed GPT-NeoX framework. IT has been trained with The Pile. The 20-billion parameter model claimed to be the largest public-access language model.

The organization primarily relied on the TPU Research Cloud from Google for training its models. It received computational resources from CoreWeave, a cryptocurrency miner, to support its research.

Additionally, there are a variety of specialized tools, libraries, and models created by EleutherAI for specific AI and machine-learning tasks. These models serve specific purposes, such as optimizing AI training on GPUs (OSLO), supporting controlled research on large language models and model analysis (Pythia), text-to-image generation (VQGAN-CLIP), and  multilingual models (Korean-language Polyglot-Ko).

Funding and financials 

In March 2023, the company stated that it was funded by donations and grants from backers, including AI startups Hugging Face and Stability AI, former GitHub CEO Nat Friedman, Lambda Labs, and Canva.