LastMile AI, a developer platform for customizing and debugging AI applications, introduced AutoEval, a fine-tuning platform for evaluator models. The platform is accessible through the company's website for virtual private cloud deployments and will soon be available on major public cloud marketplaces.

AutoEval helps create customizable evaluator models to test AI applications. It features synthetic data label generation to augment training datasets and includes alBERTa, a small language model that assesses metrics like relevance, answer equivalence, faithfulness, toxicity, and summarization. The platform runs on CPUs and processes inference requests in under 300 milliseconds.

The company claims AutoEval helps enterprises properly evaluate AI applications before deployment, reduces the time needed for subject matter experts to curate training data, and enables developers to adopt an "Eval-Driven Development" approach similar to traditional software development cycles.

Analyst QuickTake: LastMile AI continues to expand its suite of tools for optimizing AI development with the launch of AutoEval. The release follows the beta launch of its RAG Workbench ( June 2024 ), a tool for debugging and optimizing retrieval-augmented generation (RAG) systems, including a specialized hallucination detector for RAG evaluation.