DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Driving

DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Driving

⚠️ AI 生成 · 建议对照原文 本页为自动整理的学习笔记；关键数据与引用如需引用，请回查 PDF / 官方版本。

学习档位 中文笔记

类型 文献 · 更新 2026-07-19

标签 autonomous-driving · planning · end-to-end-learning · diffusion-models · navsim

所属 端到端驾驶 · 扩散模型

Migration status: imported from ad_projs@a823662; source anchors and claims remain needs-source-verification.

• Paper: https://arxiv.org/abs/2411.15139
• Code: https://github.com/hustvl/DiffusionDrive at 9b52ed0ec06b073d82d6f392ab084c7b301c8681

中文学习笔记（自动生成，需核验）

Topic: ad-end-to-end-driving · Tier: recent · Year: 2024 · Venue: —
Evidence level: partial · 本地全文: 是 · 建议阅读: ~35 分钟
Paper: https://arxiv.org/abs/2411.15139
Code: —
Generator: grok

为什么值得读

将扩散模型成功引入实时端到端自动驾驶，通过截断扩散策略与锚定高斯分布解决了直接迁移时的模式坍塌与高去噪步数开销问题，实现仅2步去噪的高质量多模态轨迹生成，并在NAVSIM上刷新记录，兼具多样性、质量与实时性。

一句话结论

提出截断扩散策略（锚定高斯+截断日程）与级联扩散解码器，使DiffusionDrive仅需2步去噪即可生成多样高质量驾驶轨迹，在NAVSIM达88.1 PDMS并以45 FPS实时运行。

问题定义

端到端自动驾驶需建模多模态驾驶动作分布，但主流单模态回归忽略不确定性；固定大词汇锚点（如VADv2）受限于锚点数量/质量且计算重。直接迁移机器人领域的vanilla扩散策略虽可建模多模态，但存在两大挑战：1）众多去噪步数（如DDIM 20步）导致推理开销大、难实时；2）从纯高斯噪声采样在动态开放交通场景中易模式坍塌、轨迹严重重叠，难以生成多样合理动作。

前置知识

扩散模型基础（DDPM/DDIM前向加噪与反向去噪、条件生成）；端到端自动驾驶范式（Transfuser/UniAD/VAD等单模态回归、VADv2词汇采样）；BEV/透视特征与可变形注意力；Transformer解码器与场景条件交互。

核心贡献

• 首次将扩散模型引入端到端自动驾驶，提出截断扩散策略：引入先验多模态锚点并截断扩散日程，从锚定高斯分布去噪到多模态驾驶动作分布，解决模式坍塌与计算开销。
• 设计高效级联transformer扩散解码器，以级联方式与感知结构化查询、BEV/PV特征交互，增强条件场景上下文利用与轨迹重建。
• 在对齐ResNet-34骨干下无额外技巧达NAVSIM navtest 88.1 PDMS新纪录，45 FPS实时，显著优于先前SOTA（含Hydra-MDP）；在nuScenes开环也优于VAD。
• 定性展示在挑战场景中生成更高质量、更多样的多模态合理驾驶轨迹。

方法流程

原始传感器输入经感知模块（地图/检测等）编码为场景表示与条件（Agent/Map查询、BEV/PV特征）；从锚定高斯分布采样噪声轨迹（截断扩散日程，向先验锚点加少量高斯噪声）；输入级联扩散解码器，在条件场景上下文引导下迭代去噪（推理仅2步），每步通过交叉注意力与调制进行轨迹重建与评分，最终输出多模态轨迹及其置信度分数。

关键模块和设计取舍

1）截断扩散策略（核心）：对比vanilla（全T步从纯高斯到数据），截断至T_trunc，仅向锚点加小部分噪声形成锚定高斯分布作为起点；训练从该分布重构GT轨迹，推理从更好初始样本截断去噪，将步数从20降至2（约10×加速），同时借助扩散多模态表达力覆盖动作空间而无需超大固定词汇。2）级联扩散解码器：transformer层结构，对噪声轨迹做空间/时间处理，与Agent/Map交叉注意力、BEV/PV可变形注意力交互，FFN+调制，输出轨迹与score；级联机制在每步迭代精炼。取舍：牺牲完整去噪灵活性换取实时性与多样性，锚点先验提供强引导但需合理设计。

数据集、实验设置与指标

主要在规划导向NAVSIM数据集（navtest split，非反应式仿真与闭环评估），指标PDMS；对齐ResNet-34骨干，无额外技巧。辅以nuScenes开环评估（L2误差、碰撞率），ResNet-50骨干。推理速度：NVIDIA 4090上FPS。对比Transfuser、VADv2/Hydra-MDP、TransfuserDP（vanilla扩散变体）等。

主要结果（需原文证据）

在NAVSIM navtest对齐ResNet-34下达88.1 PDMS新纪录（无额外技巧），45 FPS实时；相对Hydra-MDP-V8192-W-EP（8192锚点+后处理+额外监督）仍高1.6 PDMS且直接从人类演示学习、无后处理。去噪步数相对vanilla约10×减少，仅2步即可获优多样性与质量。nuScenes同ResNet-50下比VAD快1.8×，L2误差低20.8%，碰撞率低63.6%。定性：挑战场景中top-1匹配GT，top-10展现高质量换道等多样合理动作，优于Transfuser与TransfuserDP的模式坍塌。

局限、失败场景与适用边界

提取中未详述完整局限章节；直接适配vanilla时的模式坍塌与开销已通过截断解决，但仍依赖先验锚点设计，开放世界极端OOD场景的覆盖与泛化边界待进一步验证；截断可能限制极端长程去噪灵活性。适用边界：侧重实时多模态规划的端到端系统，依赖较好感知条件。

与前序 / 同期 / 后续方法的关系

端到端：对比单模态回归（Transfuser/UniAD/VAD等）与固定大词汇采样评分（VADv2、Hydra-MDP）；提出生成式扩散新范式。交通仿真扩散（MotionDiffuser/CTG等）多用抽象感知GT，本工作面向原始传感器实时E2E。机器人扩散策略（Diffusion Policy/Diffuser等）启发但直接迁移不适合动态开放交通与实时要求，本工作提出截断与锚定创新。图像生成（DDIM/TDPM等）提供截断/少步采样参考，但本工作引入显式驾驶先验。

官方代码与复现建议

官方Code & Model & Demo: hustvl/DiffusionDrive。复现建议：对齐ResNet-34/50骨干与NAVSIM设置，关注锚定高斯与截断日程超参、级联解码器注意力实现；从提供demo验证2步推理与多样性；注意条件特征交互与训练噪声调度一致性。

推荐阅读顺序

先读Abstract与Fig.1范式对比；再Intro问题分析与Fig.2/3定性+截断示意；然后Method 3.1初步与Fig.4整体架构/解码器；接着贡献列表与Related Work定位；最后结果描述与定性。有表格时对照实验。

自测问题

1. Q: Vanilla扩散策略直接用于E2E AD的两个主要问题是什么？ A: 1）去噪步数多（如20步）计算开销大难实时；2）从纯高斯采样导致轨迹模式坍塌、严重重叠，多样性差。
2. Q: 截断扩散策略如何形成锚定高斯分布并加速推理？ A: 截断扩散日程，仅向先验多模态锚点加小部分高斯噪声；推理从更好初始样本开始截断去噪，步数从20降至2。
3. Q: DiffusionDrive在NAVSIM上的主要定量结果（对齐ResNet-34）是什么？ A: 88.1 PDMS新纪录，45 FPS（NVIDIA 4090），无额外技巧，优于含后处理的Hydra-MDP约1.6。
4. Q: 级联扩散解码器如何增强条件交互？ A: Transformer结构，与Agent/Map查询交叉注意力、BEV/PV可变形注意力交互，级联机制在每去噪步迭代精炼轨迹与评分。
5. Q: 与VADv2固定词汇范式相比，本方法的优势是什么？ A: 借助扩散多模态表达力，用锚定高斯覆盖潜在动作空间而无需超大固定锚点集，减少计算并更好处理OOV场景。

原文证据

• page 1 Abstract: we propose a novel truncated diffusion policy that incorporates prior multi-mode anchors and truncates the diffusion schedule, enabling the model to learn denoising from anchored Gaussian distribution to the multi-mode driving action distribution. … demonstrates 10× reduction in denoising steps … delivering superior diversity and quality in just 2 steps. On the planning-oriented NAVSIM dataset, with aligned ResNet-34 backbone, DiffusionDrive achieves 88.1 PDMS … 45 FPS on an NVIDIA 4090.
• page 2: the numerous 20 denoising steps in the vanilla DDIM diffusion policy introduce heavy computational consumption … The trajectories sampled from different Gaussian noises severely overlap with each other … We can truncate the denoising process, reducing the required steps from 20 to just 2
• page 2 contributions: • We firstly introduce the diffusion model to the field of end-to-end autonomous driving and propose a novel truncated diffusion policy … • We design an efficient transformer-based diffusion decoder that interacts with the conditional information in a cascaded manner … • … achieving a record-breaking 88.1 PDMS on the NAVSIM navtest split … 45 FPS on an NVIDIA 4090.
• page 2: Even compared to the NAVSIM challenge-winning solution Hydra-MDP-V8192-W-EP [25] … DiffusionDrive still outperforms it by 1.6 PDMS … DiffusionDrive runs 1.8× faster than VAD and outperforms it [20] by 20.8% lower L2 error and 63.6% lower collision rate with the same ResNet-50 backbone
• page 4 Figure 3 caption / text: We truncate the diffusion process and only add a small portion of Gaussian noise to diffuse the anchor trajectories. Then, we train the diffusion model to reconstruct the ground-truth trajectory from the anchored Gaussian distribution with conditional scene context. During the inference, we also truncate the denoising process by starting from the better samples in the anchored Gaussian distribution than the pure Gaussian noise.
• page 5 Figure 4: DiffusionDrive can integrate various existing perception modules and sensor inputs. (b) The designed diffusion decoder takes the sampled noisy trajectories from anchored Gaussian distribution as input and progressively denoises them with enhanced interactions with the conditional scene context in a cascade manner

Evidence-backed Reading Notes

Topic	Evidence-backed note	Source	Short original cue
Problem	Diffusion policies can model multimodal driving but standard denoising is too slow for real-time planning.	[PDF p.1, Abstract]	Introduction
Representation	The note treats the method as trajectory anchors + truncated diffusion because the paper’s method pages introduce the relevant representation/module vocabulary.	[PDF p.4, Method]	Method
Core mechanism	An anchored Gaussian prior and cascade diffusion decoder denoise toward multi-modal driving trajectories with few steps.	[PDF p.4, Method]	truncated diffusion
Input / Output	Input: NAVSIM scene features. Output: candidate ego trajectories/actions.	[PDF p.9, Method]	NAVSIM
Training / Evaluation	The paper reports NAVSIM planning metrics such as PDMS; benchmark/devkit version must be recorded.	[PDF p.12, Method]	PDMS
Relationship	GTRS/SparseDriveV2 emphasize scoring; DiffusionDriveV2 adds RL constraints to truncated diffusion.	[PDF p.2, Related Work]	Related Work
Failure/Risk	NAVSIM cache/version, anchor files, checkpoint compatibility, and diffusion sampling settings are the main risks.	[PDF p.4, Method]	truncated diffusion
Reproduction boundary	Needs NAVSIM assets; do not substitute nuScenes mini for the default flow.	[PDF p.9, Method]	NAVSIM

Code Cross-References

Local path	Why it matters
9b52ed0ec06b073d82d6f392ab084c7b301c8681:README.md	Code/repo anchor for implementation cross-check.
9b52ed0ec06b073d82d6f392ab084c7b301c8681:navsim/agents/diffusiondrive/transfuser_config.py	Code/repo anchor for implementation cross-check.
9b52ed0ec06b073d82d6f392ab084c7b301c8681:navsim/planning/script/config/common/agent/diffusiondrive_agent.yaml	Code/repo anchor for implementation cross-check.
9b52ed0ec06b073d82d6f392ab084c7b301c8681:navsim/agents/diffusiondrive/modules/blocks.py	Code/repo anchor for implementation cross-check.
9b52ed0ec06b073d82d6f392ab084c7b301c8681:navsim/agents/diffusiondrive/modules/conditional_unet1d.py	Code/repo anchor for implementation cross-check.
9b52ed0ec06b073d82d6f392ab084c7b301c8681:navsim/agents/diffusiondrive/modules/multimodal_loss.py	Code/repo anchor for implementation cross-check.

Reproduction Notes

• Treat this note as paper/code reading material first; do not interpret mini-data smoke tests as paper reproduction. [PDF p.9, Method]
• Before running experiments, verify the local code entry points above against the paper method terminology and dataset protocol. [PDF p.4, Method]
• If a claim is not linked to a PDF page or code path in this note, treat it as an implementation hypothesis rather than established paper fact.

Local Asset Identity

Discovery evidence

• topic: ad-end-to-end-driving
• sources: asta
• retrieved_at: 2026-07-20
• query: Find foundational and recent research papers for the topic «端到端驾驶» (ad-end-to-end-driving). Prefer peer-reviewed or widely cited work with clear method contributions. Include open-source code when available. Exclude pure survey spam unless highly cited. Core concepts: end-to-end autonomous driving, planning-oriented, trajectory scoring, diffusion planning, vectorized scene. Search facets: end-to-end autonomous driving planning perception; trajectory vocabulary scoring NAVSIM; diffusion model end
• corpus_id: 274192736
• relevance_score: 0.7669155133811644
• score_total: 41
• suggested_tier: watch

Relevance

(no prose relevance explanation — numeric score only or HTTP source)

Snippets

(no snippet evidence in candidate pool)

延伸解读与背景补充

生成：2026-07-21 · 来源条数 2 · 模型 heuristic · 需人工核验数字

问题再表述

围绕「DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Driving」的核心问题与动机（待结合全文核验）。

方法要点补强

• 见原文方法章节；以下为基于摘要/摘录的要点提示。
• DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Driving Bencheng Liao1,2,⋄ Shaoyu Chen2,3 Haoran…

与相近工作的关系

与相近工作的关系待核验；请对照 related work。

常见误读

• 勿仅凭摘要推断未给出的数值指标。

自测问题

1. 这篇工作的输入/输出表示是什么？（DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Driving）
2. 训练目标与评测协议各是什么？
3. 主要失败模式或局限是什么？

外部解读索引

• [2411.15139] DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Driving — 2026-07-21 — 官方摘要/二次页面（自动抓取）
• [2411.15139] DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Driving — 2026-07-21 — 官方摘要/二次页面（自动抓取）

关键图表

方法结构（重绘）

flowchart LR
  A["输入 / 观测"] --> B["表示 / 编码"]
  B --> C["推理 / 解码"]
  C --> D["输出 / 动作或检测"]
  %% method sketch for: DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Driving

方法结构示意（重绘；细节以原论文为准，待 PDF 核验）。

论文摘录图/表

来源：原论文约 p.1（arch）；学习用途摘录。

来源：原论文约 p.7（table）；学习用途摘录。

英文自动分析（可折叠）

展开英文 Paper Card / AI deep analysis

Paper Card

Field	Content
One-line takeaway	DiffusionDrive uses prior multi-mode anchors and a truncated diffusion schedule to make diffusion-style planning fast enough for E2E driving.
Problem	Diffusion policies can model multimodal driving but standard denoising is too slow for real-time planning.
Representation	trajectory anchors + truncated diffusion
Input / Output	Input: NAVSIM scene features. Output: candidate ego trajectories/actions.
Core Mechanism	An anchored Gaussian prior and cascade diffusion decoder denoise toward multi-modal driving trajectories with few steps.
Training / Evaluation	The paper reports NAVSIM planning metrics such as PDMS; benchmark/devkit version must be recorded.
Reproduction Status	Needs NAVSIM assets; do not substitute nuScenes mini for the default flow.
Compare With	GTRS/SparseDriveV2 emphasize scoring; DiffusionDriveV2 adds RL constraints to truncated diffusion.
Failure/Risk	NAVSIM cache/version, anchor files, checkpoint compatibility, and diffusion sampling settings are the main risks.

原文摘录与素材（可折叠）

展开 Extract / Selections / Local assets

Source Anchors

Anchor	What to verify	Source	Short original cue
Title and abstract	Use to verify paper identity and top-level contribution.	[PDF p.1, Abstract]	DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Driving
Motivation	Use to verify the problem statement and why the work is needed.	[PDF p.1, Abstract]	Introduction
Core method	Use to verify the main modeling mechanism and module names.	[PDF p.4, Method]	Method
Key module terms	Use to verify exact component names before editing the note.	[PDF p.4, Method]	truncated diffusion
Dataset and protocol	Use to verify data dependencies: NAVSIM.	[PDF p.9, Method]	NAVSIM
Metrics and results	Use to verify metric names and reported benchmark context.	[PDF p.12, Method]	PDMS
Experiments or ablation	Use to verify which claims are experimentally supported.	[PDF p.9, Method]	Experiments
Position in related work	Use to verify the claimed relationship to neighboring methods.	[PDF p.2, Related Work]	Related Work
Conclusion or limits	Use to verify final claims and remaining constraints.	[PDF p.9, Method]	Conclusion

反向链接

• 扩散模型
• 端到端驾驶
• 自动驾驶数据集与评测矩阵
• 自动驾驶数据集与评测矩阵
• 自动驾驶方法谱系
• 自动驾驶方法谱系

---

学习导航

标记为已读

• 所属 Topic: ad-end-to-end-driving
• 学习路径: 学习路线 · 论文库
• 上一篇: Diffusion Models for End-to-End Autonomous Driving: A Survey of Perception, Prediction, Planning, and Control
• 下一篇: DiffusionDriveV2: Reinforcement Learning-Constrained Truncated Diffusion Modeling in End-to-End Autonomous Driving
• 返回 Topic