Portkey AI

Portkey.ai is an enterprise tech startup focused on generative AI applications. The company provides a unified interface for integrating multiple Large Language Model (LLM) providers, simplifying the process of building and deploying generative AI apps. Portkey's AI gateway offers features such as request routing, response caching, and tracking capabilities. The platform enables tech teams to deploy AI applications rapidly, claiming to ship AI apps in as little as two minutes.

Portkey's technology addresses key challenges in Retrieval Augmented Generation (RAG) applications, including monitoring RAG calls, reducing latency, and ensuring reliability. The company's unified API signature works with major LLM providers, allowing consistent request handling across different models. Portkey also offers load balancing functionality, distributing network traffic across various LLM APIs based on assigned weights. Additionally, the platform utilizes a cache system to expedite responses to previously served queries, employing both semantic and simple caching techniques.

Portkey reported serving millions of requests daily. The company's product integrates with platforms such as OpenAI, Anthropic, Langchain, and LlamaIndex, offering full-stack LLMOps tools for effective LLM connection and prompt management. Portkey plans to expand its offerings by experimenting with and releasing products in areas such as fine-tuning models and real-time canary testing.

Key customers and partnerships

Portkey has established partnerships with notable companies in the tech industry. Some of its key partners include Postman, Jio Haptik, and Springworks.

In July 2024, the company partnered with Aporia to integrate Aporia's LLM Guardrails into Portkey's Gateway to enhance the security and reliability of GenAI applications for Portkey's users. Earlier in May 2024, Portkey AI partnered with F5 Inc to reinforce security, stability, and efficiency in deploying and managing LLM applications.