Arize AI, an ML observability platform, has launched a suite of tools to evaluate and monitor production deployments of large language models (LLMs). It is available as part of the company’s free product.

The tools helps developers and engineers evaluate LLM responses, identify areas of improvement with prompt engineering, and fine-tune the generative model using vector similarity search. 

Analyst QuickTake: Last month Arize AI launched Phoenix , an open-source library to monitor LLMs for hallucinations. It monitors false or misleading results in applications built on top of LLMs such as Google’s Bard and OpenAI’s GPT-4. Since the launch of ChatGPT (a chatbot based on OpenAI’s GPT model), there has been an influx of applications built leveraging LLMs for various applications such as generative and conversational AI , language translation and summarization, as well as synthetic data generation.