China’s social media giant Xiaohongshu (known internationally as Rednote) has officially released dots.llm1, their groundbreaking open‑source large language model (LLM), developed by their Humane Intelligence Lab (hi lab) in Shanghai. Here’s what makes this model noteworthy:
Key points
- China’s Xiaohongshu, known internationally as Rednote, released the open-source model dots.llm1 through its Humane Intelligence Lab (hi lab) in Shanghai.
- dots.llm1 is a mixture-of-experts model with 142 billion total parameters, of which only 14 billion activate during inference.
- It has a 32,768-token context window, which the article says can process entire books, codebases or lengthy documents in a single pass.
- The model was pre-trained on 11.2 trillion tokens of natural, non-synthetic data and reaches benchmark-level performance without generative augmentation.
- According to the article, it achieves MMLU-level performance competitive with larger models such as Alibaba’s Qwen2.5-72B and DeepSeek-V3.
⚙️ What is dots.llm1?
- MoE architecture: dots.llm1 is a mixture‑of‑experts model with a total of 142 billion parameters, but only 14 billion activate during inference—offering state‑of‑the‑art performance with far lower computational cost (scmp.com).
- Long context support: with a 32,768‑token context window, it can process entire books, codebases, or lengthy documents in a single pass (dotsllm.dev).
- Pure training data: pre‑trained on 11.2 trillion tokens of natural (non‑synthetic) data, it achieves benchmark-level performance without generative augmentation (github.com).
- Open checkpoints: intermediate weights at every 1 trillion token milestone are published, supporting researchers in studying training dynamics and fine‑tuning earlier stages (github.com).
- MIT license: the model is released under a truly permissive open-source license—a rarity in recent LLM releases (linkedin.com).
🏆 Performance & Benchmarks
- Achieves MMLU-level performance competitive with larger models like Alibaba’s Qwen2.5‑72B and DeepSeek‑V3, despite activating fewer parameters (scmp.com).
- Early community tests report strong reasoning, coding, and world‑knowledge abilities, earning praise across the AI developer community (linkedin.com).
🚀 Getting Started
Try it now via the live Hugging Face demo
🔗 dots.llm1 demo – Hugging Face Space (linkedin.com)
Download the model and access code examples:
- GitHub: rednote‑hilab/dots.llm1 (github.com)
- Supports Hugging Face Transformers, vLLM, and sglang for Docker-based serving (github.com)
Key parameters and usage:
from transformers import AutoTokenizer, AutoModelForCausalLM
tok = AutoTokenizer.from_pretrained("rednote-hilab/dots.llm1.inst")
model = AutoModelForCausalLM.from_pretrained("rednote-hilab/dots.llm1.inst", device_map="auto")
Launch via vLLM serve or Docker for high-performance deployment (dotsllm.dev, github.com).
🔧 Why It Matters
- Efficiency at scale: MoE design keeps compute manageable—ideal for users with limited GPU capacity (dotsllm.dev).
- Transparency for research: open checkpoints demystify LLM training progression (github.com).
- Fully open-source: MIT license maximizes flexibility for commercial and academic applications .
- China’s entry into open LLMs: signifying a shift from closed behind-the‑scenes AI to sharing foundational models publicly .
🗣 Community Buzz
From r/LocalLLaMA:
“Notably, they are releasing a true base model (with no synthetic data)… with intermediate checkpoints… meaning it can be customized for just about any data distribution” (reddit.com)
On LinkedIn:
“dots.llm is released as a true base model trained on 11.2T high‑quality tokens… MIT license… intermediate training checkpoints… a gift to the AI community.” (linkedin.com)
🧭 What’s Next
- Fine‑tuning: Users can tailor the model by training from intermediate checkpoints—ideal for niche domains like legal, medical, or code generation.
- Quantization: Community projects (e.g., via llama.cpp or GGUF) are already exploring 4‑bit quantizations to enable local deployment on consumer-grade hardware.
- Benchmarking: Larger studies will compare it to Qwen3-235B, GPT‑4‑equivalents, and other benchmarks.
📌 Summary
Xiaohongshu’s release of dots.llm1 represents a major milestone in open AI: an efficient, powerful, and transparent LLM available under a permissive license. With strong benchmarks and a commitment to openness, it’s poised to accelerate research and real-world applications in China and worldwide.
Explore it now:
- Try live demo: Hugging Face Space
- Download models and docs: GitHub repo
- Dig deeper: Download the tech report from arXiv (
dots1_tech_report.pdf) (linkedin.com, github.com, x.com)
Leave a Reply