Moreh

Moreh is a US-Korean AI software tools provider that optimizes and creates AI models. Its flagship product is AI software, MoAI, which provides access to AI inference servers for training and deploying large language models (LLMs). The MoAI platform is powered by an on-the-fly IR constructor, a graph-level compiler, and a distributed runtime system. The MoAI platform primarily supports AMD ROCm and it has the capability to train and deploy multi-billion or multi-trillion parameter models,and  program anything that is needed for pretraining, fine-tuning, compression. Furthermore, it has the ability to  serve, scale to many GPUs/NPUs by automatic parallelization and optimization, virtualize all GPU/NPUs in a cluster for higher utilization and failover, and de-couple AI infrastructure from the specific hardware vendor. 

Moreh claims its platform is similar to NVIDIA’s CUDA and it is compatible with existing machine-learning frameworks like Meta’s Pytorch, Google’s TensorFlow, applications, and AI models that are not compatible with NVIDIA. Korean telecommunication KT claims that Moreh’s technologies work and perform above NVIDIA’s DGX in terms of performance speed and GPU memory capacity. As per Moreh, AMD’s MI250 Instinct accelerator powered by the MoAI platform has shown a 116% higher GPU throughput than NVIDIA’s A100, and it is also claimed that AI developers can reduce a tenth of the total time required to initiate large AI models via its software. 

Moreh completed the training of Korean language-based LLM with 211 billion parameters and aimed to release the model as open-source in the latter part of 2023.

As of October 2023, Moreh claimed to have begun generating revenue in 2021 and aimed to have reached ~USD 30 million by the end of 2023.

Key customers and partnerships

In November 2024, Moreh partnered with AI semiconductor company Tenstorrent to integrate Tenstorrent's AI semiconductors (NPUs) with Moreh's software to support various AI applications, including LLM training and inference.  The combined solution is expected to be commercialized by the first half of 2025.

Korean telecommunication KT has been working with Moreh since 2021 to build a cost-effective, scalable AI infrastructure powered by AMD GPUs and MoAI software. 

MoAI is currently powering over 2,000 GPUs for commercial laaS/PaaS clouds and on-premise LLM training systems.