GLaM
Google Brain's trillion-weight sparse mixture-of-experts (MoE) model; competitive with GPT-3, more efficiently.
2021
2023
Mixtral 8x7B
Sparse mixture-of-experts (MoE) model with open weights under Apache 2.0; beats Llama 2 70B on most benchmarks.
2024
Gemini 1.5
Gemini 1.5 Pro, a mixture-of-experts (MoE) model, matches 1.0 Ultra with less compute and offers up to 1M tokens.
DeepSeek-V3
Open-weights mixture-of-experts (MoE) model with 671B parameters, 37B active, trained on 14.8T tokens.
No events match these filters.