Meta Opens Muse Spark Weights: 816B MoE, 1M Context

On October 15, 2026, Meta Superintelligence Labs released the weights for Muse Spark Open — an 816-billion-parameter sparse mixture-of-experts model that activates 42 billion parameters per token, with a 1,048,576-token context window. Meta calls it “the most capable US open-weight model to date,” and the release is as much a hardware story as a benchmark one: Meta says the checkpoint runs on two Apple M5 Ultra Mac Studios linked with MLX.

Advanced

Key points

On October 15, 2026, Meta Superintelligence Labs released the weights for Muse Spark Open, an 816-billion-parameter sparse mixture-of-experts model that activates 42 billion parameters per token with a 1,048,576-token context window.
Meta calls Muse Spark Open “the most capable US open-weight model to date” and says the checkpoint runs on two Apple M5 Ultra Mac Studios linked with MLX.
The Muse Model License dated October 15, 2026 grants free use, modification, fine-tuning, distillation, redistribution and sale but requires prominent “Built with Muse Spark Open” attribution and a separate agreement for Model-as-a-Service providers above US$20 million group revenue.
Artificial Analysis independently scores it 39 on its Intelligence Index, 9th of 118 models, while Meta’s own table places Qwen3.8-2.4T-A95B at 40, Kimi K3 at 44 and GLM 5.3 at 45.
Memory is described as the real barrier: BF16 weights total about 1.63 TB, the model card asks for at least 2 TB of VRAM, and Unsloth’s 2-bit GGUFs target 250-316 GB.

Meta Muse Spark Open announcement graphic — Image credit: Meta AI

What Meta Released

Muse Spark Open is the open-weight counterpart of Muse Spark, the proprietary frontier model Meta launched in April 2026, and shares the same base model as Muse Spark 1.3. It is natively multimodal — text, image, video and audio in, text out — and trained on data from more than 100 languages, with a separate 1.8B-parameter perception encoder alongside the language model.

The licence is not Apache 2.0. Muse Spark Open ships under the Muse Model License, dated October 15, 2026, which grants free use, modification, fine-tuning, distillation, redistribution and sale but attaches two conditions: products built on it must display “Built with Muse Spark Open” prominently, and anyone offering it as a Model-as-a-Service whose group gross revenue exceeds US$20 million in any 12-month period must notify Meta and sign a separate written agreement. Meta claims no ownership of outputs, and downloads on Hugging Face are gated behind a registration form requiring a legal name, date of birth and organisation.

Weights are on Hugging Face, Amazon SageMaker AI, Google Cloud Model Garden and Microsoft Foundry Models, with hosted APIs from seven partners including Together AI and Fireworks, and support across SGLang, vLLM, TensorRT-LLM, Apple MLX and llama.cpp.

Inside the Architecture

Diagram of the Muse Spark Open mixture-of-experts architecture — Image credit: vLLM project

All 62 decoder layers are MoE layers, each holding 256 experts with top-8 routing. The vLLM team’s breakdown puts almost all parameter mass in expert weights: 799B in experts, 13.5B in attention, 4.2B in MoE latent projections and 3.3B in the embedding and output layers. Attention uses 128 query heads and 16 KV heads (grouped-query) over a hidden dimension of 8192. Two details differ from a vanilla decoder: “sandwich” normalisation, with a norm layer before and after each block, and latent MoE projections that pool expert inputs into a smaller space.

Meta’s new Helion kernel DSL ships auto-tunable GPU kernels in SGLang, reporting 1.03–3.95× kernel-level speedups on NVIDIA and AMD hardware. vLLM added day-0 support with expert parallelism, PD disaggregation and a trained DSpark draft model for speculative decoding.

Benchmarks and Hardware

Meta's benchmark scorecard comparing Muse Spark Open with other open-weight models — Image credit: Meta AI

At xhigh reasoning effort Meta reports Terminal-Bench 2.1 of 76.4, MCP-Atlas 89.2, JobBench 61.8, GDPVal-AA v2 of 1401, GPQA Diamond 89.0, HLE-Diamond 26.1 without tools and 50.0 on ZeroBench with tools. Artificial Analysis independently scores it 39 on its Intelligence Index — 9th of 118 models — at about US$1.25 per million input tokens and US$4.25 per million output tokens, at 102 output tokens per second.

Against other open models in Meta’s own table that placement is competitive rather than dominant: Qwen3.8-2.4T-A95B scores 40, Kimi K3 44 and GLM 5.3 45, while Muse Spark Open leads the group on JobBench and MCP-Atlas. Memory is the real barrier: BF16 weights total about 1.63 TB and the model card asks for at least 2 TB of VRAM, or 700 GB for the NVFP4 release. Unsloth’s Dynamic GGUFs shrink that to a 2-bit quant targeting 250–316 GB of total memory, or 180–212 GB at 1-bit.

What This Means

The interesting part is the shape of the release, not the headline score. Meta is shipping a frontier-adjacent checkpoint under terms that are permissive for researchers and small developers but explicitly commercial-restrictive at the Model-as-a-Service scale — a middle path between Apache 2.0 and a closed API. Gating downloads behind an identity form departs further from open-weight convention.

For local-inference developers, a 2-bit quant that fits in 256 GB of unified memory is the first realistic route to running a 256-expert model on a workstation, and it keeps pressure on the assumption that only Chinese labs publish competitive open weights. Meta also reports extra safety post-training for the open release, claiming 99.2% refusal on its internal jailbreak benchmark, and states the model is not intended for users under 18.

Related Coverage

Meta Hasn’t Given Up on Open Source: Muse Spark Launches as Open-Weight Plans Continue — April 2026, when Meta first promised open-weight versions of its frontier models.
Meta Releases Muse Glimmer, a 30B Agent Model for a Single GPU — the small open-weight model distilled from Muse Spark.
Meta Releases Muse Wonder, Its First Frontier Agent Models — the closed models Meta shipped two days earlier.
Qwen3.8-2.4T-A95B: Alibaba Open-Weights Its Max-Tier Flagship — the open-weight benchmark Meta is measuring itself against.

This post was drafted with AI assistance and reviewed by RITS staff.