MiMo-Embodied: X-Embodied Foundation Model Technical Report
MiMo-Embodied: X-Embodied Foundation Model Technical Report
Section titled “MiMo-Embodied: X-Embodied Foundation Model Technical Report”⚠️ AI 生成 · 建议对照原文 本页为自动整理的学习笔记；关键数据与引用如需引用，请回查 PDF / 官方版本。
学习档位 中文笔记
类型 文献 · 更新 2026-07-19
所属 具身基础模型与智能体
中文学习笔记（自动生成，需核验）
Section titled “中文学习笔记（自动生成，需核验）”Topic: embodied-foundation-models · Tier: watch · Year: 2025 · Venue: —
Evidence level: partial · 本地全文: 是 · 建议阅读: ~75 分钟
Paper: https://arxiv.org/abs/2511.16518
Code: —
Generator: grok
为什么值得读
Section titled “为什么值得读”首个开源跨具身基础模型，成功统一自动驾驶与具身智能两大领域并均达SOTA；通过多阶段训练、数据策展与CoT/RL展现强正迁移，为物理世界交互的通用VLM提供可复现范式与详细设计分析。
MiMo-Embodied是首个开源并在自动驾驶与具身智能双领域均达SOTA的跨具身基础模型，证明两域可通过多阶段学习相互强化。
现有专用具身VLM受限于单一领域（室内具身或户外驾驶），存在显著域差距，缺乏统一跨具身VLM与全面跨具身能力评估，导致空间理解与推理难以在动态室内外场景间泛化。
Vision-Language Models（VLMs）基础、具身智能核心能力（affordance、任务规划、空间理解）、自动驾驶感知-预测-规划范式、多阶段SFT、Chain-of-Thought、强化学习（如GRPO）、以及从MiMo-VL继承的ViT+LLM架构知识。
- 提出首个开源统一VLM（MiMo-Embodied），整合自动驾驶与具身智能，支持单图/多图/视频输入。
- 在17个具身AI基准（任务规划、Affordance预测、空间理解）和12个自动驾驶基准（环境感知、状态预测、驾驶规划）上达SOTA，显著优于开源/闭源/专用基线。
- 构建涵盖通用、具身与自动驾驶的多元数据集，并设计四阶段渐进训练策略，验证两域强正迁移与相互强化。
- 开源代码与模型（GitHub），并提供模型设计、数据构建与训练方法的详细分析以促进后续研究。
基于MiMo-VL初始化的架构（ViT编码视觉输入 + MLP投影对齐LLM潜在空间 + LLM推理），处理单图/多图/视频。数据覆盖通用多模态 + 具身（affordance/规划/空间）+ 自动驾驶（感知/预测/规划）。四阶段训练：Stage1 通用+具身SFT建立核心能力；Stage2 混合自动驾驶与具身SFT实现跨域；Stage3 CoT推理SFT增强复杂推理；Stage4 基于GRPO的RL微调优化任务性能。
关键模块和设计取舍
Section titled “关键模块和设计取舍”核心组件：1）从MiMo-VL继承的ViT处理高分辨率单图/多图/视频，自注意力提取特征；2）MLP投影器将视觉token映射到LLM输入空间并保留关键特征；3）LLM骨干负责文本理解、思考过程与最终答案生成。设计取舍：继承预训练对齐与推理能力以降低从零训练成本；渐进四阶段（先具身后驾驶再CoT/RL）促进正迁移而非简单混合；支持跨具身指令与多模态输入以统一室内外物理交互。
数据集、实验设置与指标
Section titled “数据集、实验设置与指标”通用数据集：继承MiMo-VL语料（高分辨率图像、视频、长文本/文档、合成推理数据），覆盖视觉定位、文档图表理解、视频时序推理等。具身AI数据集：Affordance预测、高层任务规划、空间理解。自动驾驶数据集：环境感知、状态预测、驾驶规划。评估：17个具身AI基准 + 12个自动驾驶基准，涵盖上述核心能力；另有真实世界导航/操作与驾驶定性评估及消融。具体指标数值与详细实验设置待来源核验（摘录仅描述SOTA与优于基线）。
主要结果（需原文证据）
Section titled “主要结果（需原文证据）”MiMo-Embodied在17个具身AI基准（任务规划、Affordance预测、空间理解）和12个自动驾驶基准（环境感知、状态预测、驾驶规划）上达到SOTA，显著优于现有开源、闭源与专用VLM基线；两域通过多阶段学习、策展数据与CoT/RL表现出强正迁移并相互强化。真实世界具身导航/操作与自动驾驶任务有定性验证，另有消融研究。具体数值指标与对比表待来源核验。
局限、失败场景与适用边界
Section titled “局限、失败场景与适用边界”摘录主要强调挑战（域差距、评估不全面）与模型优势，未详细列出模型自身局限、失败场景或适用边界；结论提及Next Steps但内容未提供。待来源核验。
与前序 / 同期 / 后续方法的关系
Section titled “与前序 / 同期 / 后续方法的关系”针对专用具身VLM如RoboBrain、VeBrain（侧重规划与空间）和自动驾驶专用如RoboTron-Drive、DriveLMM-o1（侧重感知/预测/规划）的碎片化局限；基于通用VLM进展与MiMo-VL作为初始化基础；填补统一跨具身模型空白，并通过全面基准评估推动领域。
官方代码与复现建议
Section titled “官方代码与复现建议”推荐阅读顺序
Section titled “推荐阅读顺序”- Abstract与Introduction（问题与总览）；2. Architecture（第2节，核心组件与输入处理）；3. Training Dataset（第3节，数据构成）；4. Training Strategy（第4节，四阶段细节）；5. Evaluation（第5节，定量/定性/消融）；6. Conclusion与Appendix（可视化示例与补充）。
- Q: MiMo-Embodied解决的两个主要挑战是什么？ A: 缺乏统一桥接自动驾驶与具身AI的VLM；缺乏全面的跨具身能力评估。
- Q: 模型架构的三个核心组件是什么？如何初始化？ A: Vision Transformer（视觉编码）、MLP投影器、LLM；均从MiMo-VL预训练权重初始化。
- Q: 四阶段训练策略分别是什么？ A: Stage1：具身AI监督微调（通用+具身）；Stage2：自动驾驶监督微调（混合）；Stage3：CoT推理监督微调；Stage4：强化学习微调（GRPO）。
- Q: 具身AI与自动驾驶各自评估的核心能力各是哪三项？ A: 具身：Affordance预测、任务规划、空间理解；自动驾驶：环境感知、状态预测、驾驶规划。
- Q: 数据与训练如何实现两域正迁移？ A: 策展多元数据集（通用+具身+驾驶）配合渐进四阶段（先具身后混合驾驶再CoT/RL），使两域相互强化。
- Abstract (page 1): We open-source MiMo-Embodied, the first cross-embodied foundation model to successfully integrate and achieve state-of-the-art performance in both Autonomous Driving and Embodied AI. MiMo-Embodied sets new records across 17 embodied AI benchmarks… while also excelling in 12 autonomous driving benchmarks… these two domains exhibit strong positive transfer and mutually reinforce one another.
- Section 1 Introduction (page 3): To address these challenges, we present MiMo-Embodied, a unified VLM that merges the tasks of autonomous driving and embodied AI into a single model. To our knowledge, MiMo-Embodied is the first open-source VLM to integrate these critical areas…
- Section 2.1 Architecture (page 4): The MiMo-Embodied architecture consists of three main components: (1) a Vision Transformer (ViT) for encoding visual inputs; (2) a projector that transforms these visual encodings into a latent space aligned with the Large Language Model (LLM); and (3) the LLM…
- Section 4 Training Strategy overview (page 3 & 10+): we develop a progressive four-stage training strategy… Stage 1: Embodied AI Supervised Fine-tuning… Stage 2: Autonomous Driving Supervised Fine-tuning… Stage 3: Chain-of-Thought Reasoning Supervised Fine-tuning… Stage 4: Reinforcement Learning Fine-Tuning.
- Abstract & Figure 1 caption (page 1): Code and models are available at https://github.com/XiaomiMiMo/MiMo-Embodied.
Discovery evidence
Section titled “Discovery evidence”- topic:
embodied-agents - sources:
arxiv - retrieved_at: 2026-07-20
- query: embodied AI agents foundation models
- arxiv:
2511.16518 - score_total: 41
- suggested_tier:
watch
Relevance
Section titled “Relevance”(no prose relevance explanation — numeric score only or HTTP source)
Snippets
Section titled “Snippets”(no snippet evidence in candidate pool)
延伸解读与背景补充
Section titled “延伸解读与背景补充”生成：2026-07-21 · 来源条数 2 · 模型
heuristic· 需人工核验数字
围绕「MiMo-Embodied: X-Embodied Foundation Model Technical Report」的核心问题与动机（待结合全文核验）。
方法要点补强
Section titled “方法要点补强”- 见原文方法章节；以下为基于摘要/摘录的要点提示。
- Xiaomi EV Xiaomi MiMo-Embodied MiMo-Embodied: X-Embodied F…
与相近工作的关系
Section titled “与相近工作的关系”与相近工作的关系待核验；请对照 related work。
- 勿仅凭摘要推断未给出的数值指标。
- 这篇工作的输入/输出表示是什么？（MiMo-Embodied: X-Embodied Foundation Model Technical Report）
- 训练目标与评测协议各是什么？
- 主要失败模式或局限是什么？
外部解读索引
Section titled “外部解读索引”- [2511.16518] MiMo-Embodied: X-Embodied Foundation Model Technical Report — 2026-07-21 — 官方摘要/二次页面（自动抓取）
- [2511.16518] MiMo-Embodied: X-Embodied Foundation Model Technical Report — 2026-07-21 — 官方摘要/二次页面（自动抓取）
方法结构（重绘）
Section titled “方法结构（重绘）”flowchart LR A["输入 / 观测"] --> B["表示 / 编码"] B --> C["推理 / 解码"] C --> D["输出 / 动作或检测"] %% method sketch for: MiMo-Embodied: X-Embodied Foundation Model Technical Report方法结构示意（重绘；细节以原论文为准，待 PDF 核验）。
论文摘录图/表
Section titled “论文摘录图/表”
来源：原论文约 p.1（arch）；学习用途摘录。
来源：原论文约 p.11（table）；学习用途摘录。
英文自动分析（可折叠）
Section titled “英文自动分析（可折叠）”展开英文 Paper Card / AI deep analysis
Paper Card
Section titled “Paper Card”| Field | Content |
|---|---|
| Year | 2025 |
| Authors | Xiaoshuai Hao, Lei Zhou, Zhijian Huang, Zhiwen Hou, Yingbo Tang, Lingfeng Zhang, Guang Li, Zheng Lu, Shuhuai Ren, Xianhui Meng, Yuchen Zhang, Jing Wu |
| arXiv | 2511.16518 |
| DOI | — |
| Topics | embodied-foundation-models |
| Paper | https://arxiv.org/abs/2511.16518 |
原文摘录与素材（可折叠）
Section titled “原文摘录与素材（可折叠）”展开 Extract / Selections / Local assets
Selections
Section titled “Selections”embodied-foundation-models: tier=watch rank=4 score=53 — auto refresh 2026-07-19 sources=arxiv
Extract excerpt
Section titled “Extract excerpt”Xiaomi EV Xiaomi MiMo-Embodied
MiMo-Embodied: X-Embodied Foundation Model Technical Report
Xiaomi Embodied Intelligence Team
See Contributions and Acknowledgments section for a full author list.
arXiv:2511.16518v2 [cs.RO] 28 Apr 2026 Abstract We open-source MiMo-Embodied, the first cross-embodied foundation model to successfully integrate and achieve state-of-the-art performance in both Autonomous Driving and Embodied AI. MiMo-Embodied sets new records across 17 embodied AI benchmarks in Task Planning, Affordance Prediction and Spatial Understanding, while also excelling in 12 autonomous driving benchmarks across Environmental Perception, Status Prediction, and Driving Planning. Across these tasks, MiMo-Embodied significantly outperforms existing open-source, closed-source, and specialized baselines. Our results indicate that through multi-stage learning, curated data construction, and CoT/RL fine-tuning, these two domains exhibit strong positive transfer and mutually reinforce one another. We provide a detailed analysis of our model design and training methodologies to facilitate further research. Code and models are available at https://github.com/XiaomiMiMo/ MiMo-Embodied.
Embodied AI Benchmarks Autonomous Driving Benchmarks
Figure 1 Performance Comparison in Autonomous Driving and Embodied AI Benchmarks. MiMo-Embodied achieves state-of-the-art performance on both ben