Groq

Groq is a manufacturer of advanced chips engineered for inference processing, enhancing systems for real-time AI and high-performance computing (HPC). Groq provides hardware and software solutions, including GroqChip processors, GroqCard accelerators, GroqNode servers, GroqRack compute clusters, and GroqWare suites.

Notably, Groq’s chips feature its proprietary tensor streaming processor (TSP) architecture, which delivers consistent, low-latency, and high-throughput performance built for machine-learning applications and other computationally intensive tasks.

Groq also developed the Language Processing Unit (LPU) Inference Engine, which is a novel processing system designed to efficiently handle computationally intensive applications with a sequential component, like LLMs. These LPU Inference Engines address the primary bottlenecks for LLMs, offering substantial compute power and memory bandwidth, significantly improving the speed of text sequence generation compared to Graphics Processors (GPUs). Groq was the first to achieve 100 tokens per second (T/s) per user for Llama-2 70B, and by August 2023, its LPU system was generating more than 240 T/s per user.

In March 2024 , Groq acquired Definitive Intelligence to scale its cloud platform and provide on-demand access to its AI chips. Further, the company stated in July 2024 that plans to deploy 1.7 million of its AI processors by end 2025, as it aims to capture half of the global AI inference market.

Key customers and partnerships

In July 2024, the company disclosed that it attracted 280,000 developers to its platform in a span of four months. Overall, Groq's products cater to various sectors, such as finance, industrial automation, cybersecurity, and scientific research. Notable customers include Argonne National Laboratory and In-Q-Tel, a non-profit investor dedicated to expediting the advancement and implementation of state-of-the-art technologies for US government agencies.

Key partnerships include its collaboration with OneNano to create a next-generation CEX (cryptocurrency exchange) with scalable, ultra-low latency AI acceleration solutions (December 2022) and a partnership with Samsung's Foundry business, where it would be Groq’s next-gen silicon partner (August 2023).