Deep Infra

DeepInfra is a cloud platform that enables businesses to run machine learning models through scalable inference API. The company provides infrastructure for deploying and running various open-source AI models, including Meta's Llama 2, CodeLlama, and their variants. 

DeepInfra's platform handles servers, GPUs, scaling, and monitoring, allowing businesses to focus on their applications. The company differentiates itself through its pricing model, charging USD 1 per 1 million tokens for input and output, compared to higher rates charged by competitors.

The company’s infrastructure is optimized to efficiently manage concurrent users on the same servers and model simultaneously, addressing the challenge of token computation and memory bandwidth requirements. 

The service is accessible through REST, Python, or JavaScript APIs and supports OpenAI API compatibility for easy migration.

The company generates revenue on a pay-as-you-go scheme, with pricing ranging from USD 0.055 to USD 0.24 per 1 million input tokens, based on the type of model used. Deep Infra also allows enterprises to deploy their models on its hardware infrastructure, priced at USD 1.5 to USD 3 per GPU per hour (based on GPU model).

Key customers and partnerships

DeepInfra primarily serves small-to-medium-sized businesses (SMBs) that seek access to open-source language models.