All Updates

All Updates

icon
Filter
Product updates
Partnerships
Allen Institute for AI releases OLMoE, an open mixture-of-experts model
Foundation Models
Sep 4, 2024
This week:
Product updates
Pimax announces compact Dream Air VR headset for USD 1,900
Extended Reality
Dec 23, 2024
Funding
HuLoop raises Series A funding from Mighty Capital
Workflow Automation Platforms
Dec 23, 2024
M&A
Fiserv to acquire Payfare for undisclosed sum to enhance embedded finance solutions
Financial Wellness Tools
Dec 23, 2024
M&A
Fiserv to acquire Payfare for undisclosed sum to enhance embedded finance solutions
FinTech Infrastructure
Dec 23, 2024
Funding
Meight raises EUR 3.4 million in funding led by Cusp Capital
Smart Mobility Information
Dec 23, 2024
Regulation/policy
Desktop Metal sues Nano Dimension over merger compliance
Additive Manufacturing
Dec 23, 2024
Management news
Nexa3D scales back operations amid financial crisis
Additive Manufacturing
Dec 23, 2024
Funding
Fraction AI raises USD 6 million in pre-seed funding to scale hybrid data labeling
Machine Learning Infrastructure
Dec 23, 2024
Last week:
Partnerships
Rocket Lab launches sixth Synspective radar satellite as part of ongoing partnership
Space Travel and Exploration Tech
Dec 21, 2024
Partnerships
EHang partners with Changan Automobile to develop eVTOL aircraft for personal transportation
Passenger eVTOL Aircraft
Dec 21, 2024
Foundation Models

Foundation Models

Sep 4, 2024

Allen Institute for AI releases OLMoE, an open mixture-of-experts model

Product updates
Partnerships

  • Allen Institute for AI (AI2) has partnered with Contextual AI to introduce OLMoE, a model claimed to be performance and size-efficient.

  • The mixture-of-experts model has 1 billion active and 7 billion total parameters. Additionally, it was trained on 5 trillion tokens using a data mix based on AI2's Dolma and DataComp-Baseline. The model incorporates MoE details like routing algorithms, auxiliary loss functions, and sparse upcycling.

  • The model, along with its data, code, evaluations, logs, and intermediate training checkpoints, is freely available.

  • The institute has also released multiple variants and checkpoints of the model for future research. Additionally, AI2 is previewing its new Tulu 3 post-training pipeline, which incorporates additional instruction data and shows improvements in math, code, and instruction-following evaluations.

Contact us

Gain access to all industry hubs, market maps, research tools, and more
Get a demo
arrow
menuarrow

By using this site, you agree to allow SPEEDA Edge and our partners to use cookies for analytics and personalization. Visit our privacy policy for more information about our data collection practices.