Mistral AI, an open-source developer of foundation models, has released 8x22B, an LLM built on a mixture of expert (MoE) architecture.
This approach aims to provide users with a combination of different models, specialized in different tasks, to optimize performance and costs by requiring only 44 billion active parameters per forward pass.
The model has a 65,000-token context window and 176 billion parameters. Additionally, the model is available under a permissive Apache 2.0 license.
By using this site, you agree to allow SPEEDA Edge and our partners to use cookies for analytics and personalization. Visit our privacy policy for more information about our data collection practices.