Distributional

Founded in September 2023, Distributional is an AI testing platform that helps organizations evaluate and ensure  reliability of their AI applications. The platform enables AI engineering and product teams to proactively detect, understand, and address AI risks before deploying applications to production. Distributional's platform focuses on testing the consistency of any AI/ML application, with particular emphasis on GenAI which is prone to non-determinism (varying outputs from the same input) and non-stationarity (shifting components outside developer control).

The platform features an extensible test framework that allows teams to collect and augment data, run tests, and respond to alerts through adaptive calibration or debugging while integrating with existing datastores, workflow systems, and alerting platforms. Through its configurable test dashboard, teams can collaborate on test repositories, analyze results, triage failed tests, and capture audit trails. The platform's intelligent test automation capabilities help to scale AI testing by automating data augmentation, test selection, and calibration in an adaptive preference learning process.