UniAD: Planning-oriented Autonomous Driving

UniAD: Planning-oriented Autonomous Driving

⚠️ AI 生成 · 建议对照原文 本页为自动整理的学习笔记；关键数据与引用如需引用，请回查 PDF / 官方版本。

学习档位 中文笔记

类型 文献 · 更新 2026-07-19

标签 autonomous-driving · planning · end-to-end-learning · nuscenes

所属 端到端驾驶 · 端到端学习

Migration status: imported from ad_projs@a823662; source anchors and claims remain needs-source-verification.

• Paper: https://arxiv.org/abs/2212.10156
• Code: https://github.com/OpenDriveLab/UniAD at 609ee083ea51c3521c323f1279dfc4cee0e60467

中文学习笔记（自动生成，需核验）

Topic: ad-end-to-end-driving · Tier: foundational · Year: 2022 · Venue: —
Evidence level: partial · 本地全文: 是 · 建议阅读: ~45 分钟
Paper: https://arxiv.org/abs/2212.10156
Code: —
Generator: grok

为什么值得读

论文提出以规划为导向的端到端自动驾驶统一框架UniAD，首次系统整合感知（检测/跟踪/建图）、预测（运动/占用）与规划任务，用query接口实现任务协同与全局交互建模，强调规划优先而非简单堆叠或多任务学习，在nuScenes上验证全面优越性，对理解目标驱动系统设计与端到端AD范式很有价值。

一句话结论

UniAD以规划为导向，用统一query连接全栈感知-预测-规划模块，实现任务协同并在nuScenes上全面超越先前SOTA。

问题定义

现代自动驾驶系统常采用模块化顺序任务（感知-预测-规划）的独立模型或多任务共享骨干+分头设计，易出现累积误差、特征错位、负迁移或任务协调不足；直接端到端规划则缺乏安全性与可解释性。需设计以规划为最终目标、合理组织前序任务的框架，使所有模块服务于规划。

前置知识

BEV表示与编码器（如BEVFormer）、Transformer解码器与query机制、多目标跟踪、全景分割建图、多模态运动预测、占用网格预测、规划基础；熟悉nuScenes场景与自动驾驶任务管线。

核心贡献

• 提出规划导向的自动驾驶框架新视角，证明有效任务协调的必要性，而非独立设计或简单多任务学习。
• 提出UniAD综合端到端系统，覆盖宽范围任务，核心是用query作为接口连接所有节点，实现灵活中间表示与多任务知识交换以服务规划。
• 在具有挑战性的真实场景基准上实例化UniAD，通过大量消融验证相对先前SOTA在所有方面的优越性。

方法流程

多相机图像输入特征提取器，经BEV编码器得到统一BEV特征B；TrackFormer用track/detection queries从B检测跟踪智能体；MapFormer用map queries对道路元素做全景分割；MotionFormer用agent与map queries建模交互并预测多智能体多模态轨迹（含ego-vehicle query）；OccFormer以B为query、智能体知识为key/value预测多步占用并保持身份；最后Planner用MotionFormer的ego-query预测路径点，并结合OccFormer占用避开碰撞。

关键模块和设计取舍

TrackFormer：query-based联合检测与MOT，无需非可微后处理，track queries跨帧建模并自注意力聚合时序。MapFormer：基于Panoptic SegFormer，稀疏map queries编码车道/分隔线/交叉等things与可行驶区域stuff，输出供下游交互。MotionFormer：场景中心多智能体预测，每层建模agent-agent、agent-map、agent-goal point三类交互（MHSA/MHCA/DeformAttn），motion queries含context与位置（场景级/智能体级锚点等）；训练时用非线性平滑器调整目标轨迹以处理上游不确定性并符合运动学。OccFormer：结合BEV与智能体知识预测未来占用。Planner：基于注意力的简单规划器，利用ego-query与占用避免碰撞。取舍：query提供更大感受野以软化上游误差并灵活建模交互；规划导向而非直接优化或残缺组件；训练时目标优化不影响推理。

数据集、实验设置与指标

在具有挑战性的nuScenes基准上实例化与评估；通过大量消融验证有效性。具体指标与设置细节待来源核验。

主要结果（需原文证据）

通过大量消融证明该哲学的有效性，在所有方面substantial outperforming先前state-of-the-arts。具体数值与对比细节待来源核验。

局限、失败场景与适用边界

摘录未详述UniAD自身局限、失败场景或适用边界；提及先前独立模型有信息丢失/误差累积/特征错位风险，MTL有负迁移，直接规划缺乏安全与可解释性（尤其高动态城市场景）。更细局限待来源核验。

与前序 / 同期 / 后续方法的关系

对比独立模型设计（工业常见）、多任务学习（共享骨干分头，如Transfuser、BEVerse等）、端到端范式。先前端到端或为直接规划（c.1，如部分仿真工作）、或含部分组件（c.2，如NMP、NEAT、BEVerse、PnPNet、ViP3D、P3、MP3、ST-P3、LAV）；UniAD（c.3）首次全面调查感知-预测-规划多任务联合协作，含检测/跟踪/建图/运动/占用/规划，以规划为导向组织前序任务。

官方代码与复现建议

官方代码与模型公开：https://github.com/OpenDriveLab/UniAD。摘录称Code and models are public，建议从该仓库复现；具体环境/预训练/训练细节待来源核验。

推荐阅读顺序

先读摘要与引言（含Fig.1框架对比与Table 1任务分类），明确规划导向动机；再读Methodology Overview与Fig.2 pipeline；然后逐模块（TrackFormer/MapFormer → MotionFormer交互与query/非线性平滑 → OccFormer → Planner）；最后看实验/消融（摘录无）与贡献总结。

自测问题

1. Q: UniAD的核心设计哲学是什么？与先前独立模型或多任务学习有何不同？ A: 规划导向：以规划为最终目标组织前序任务，使它们服务于规划；不同于独立模型的误差累积/信息丢失，或多任务的负迁移，也不同于直接规划或残缺组件的端到端。
2. Q: query在UniAD中起什么作用？ A: 作为统一接口连接所有节点；提供更大感受野软化上游误差，并灵活建模多agent交互、agent-map等关系，实现中间表示交换与任务协同。
3. Q: MotionFormer建模了哪些交互？如何处理上游不确定性？ A: agent-agent、agent-map、agent-goal point三类交互；训练时用非线性平滑器（含jerk/曲率等运动学代价）调整目标轨迹使其物理可行，不影响推理。
4. Q: UniAD包含哪些主要任务模块？ A: 感知：TrackFormer（检测+跟踪）、MapFormer（在线建图/全景分割）；预测：MotionFormer（运动预测，含ego）、OccFormer（占用预测）；规划：Planner。
5. Q: 为什么强调场景中心而非agent中心预测？ A: 场景中心范式一次前向即可输出多智能体轨迹，避免将整个场景对齐到每个agent坐标系的计算开销。

原文证据

• page 1 Abstract: Instead, we argue that a favorable framework should be devised and optimized in pursuit of the ultimate goal, i.e., planning of the self-driving car. … We introduce Unified Autonomous Driving (UniAD), a comprehensive framework up-to-date that incorporates full-stack driving tasks in one network.
• page 1-2 Introduction: the emergence of end-to-end autonomous driving unites all nodes from perception, prediction and planning as a whole. The choice and priority of preceding tasks should be determined in favor of planning. The system should be planning-oriented
• page 2 Table 1 / contributions: UniAD (ours) … Det. Track Map Motion Occ. Plan all ✓. … (a) we embrace a new outlook of autonomous driving framework following a planning-oriented philosophy… (b) we present UniAD… The key component… is the query design as interfaces connecting all nodes. (c) we instantiate UniAD on the challenging benchmark… substantially outperforming previous state-of-the-arts in all aspects.
• page 2-3 Methodology Overview: UniAD comprises four transformer decoder-based perception and prediction modules and one planner in the end. Queries Q play the role of connecting the pipeline to model different interactions of entities in the driving scenario. … Finally, Planner utilizes the expressive ego-vehicle query from MotionFormer to predict the planning result, and keep itself away from occupied regions predicted by OccFormer to avoid collisions.
• page 3-4 MotionFormer: MotionFormer captures interactions among agents and maps and forecasts per-agent future trajectories. … each layer captures three types of interactions: agent-agent, agent-map and agent-goal point. … we adopt a non-linear smoother to adjust the target trajectories and make them physically feasible given an imprecise starting point
• page 1: https://github.com/OpenDriveLab/UniAD … Code and models are public.

Evidence-backed Reading Notes

Topic	Evidence-backed note	Source	Short original cue
Problem	Optimizing perception tasks independently does not guarantee planning-oriented scene understanding.	[PDF p.1, Abstract]	Introduction
Representation	The note treats the method as dense BEV + task queries because the paper’s method pages introduce the relevant representation/module vocabulary.	[PDF p.3, Abstract]	Method
Core mechanism	Task queries and BEV features form a chained multi-task system whose outputs feed downstream motion and planning.	[PDF p.3, Abstract]	planning-oriented
Input / Output	Input: multi-camera temporal scenes and map/ego metadata. Output: boxes, tracks, maps, motion, occupancy, and ego plan.	[PDF p.6, Method]	nuScenes
Training / Evaluation	The paper reports multiple nuScenes task metrics; full data and staged artifacts are needed for meaningful results.	[PDF p.6, Method]	L2
Relationship	VAD compresses planning inputs into vectors; SparseDrive uses sparse scene tokens; SSR questions explicit perception-task necessity.	[PDF p.13, Related Work]	Related Work
Failure/Risk	Multi-stage checkpoints, map expansion, motion anchors, temporal infos, and old OpenMMLab stack are major risks.	[PDF p.3, Abstract]	planning-oriented
Reproduction boundary	Too long for first-stage reproduction; mini can only validate local pieces.	[PDF p.6, Method]	nuScenes

Code Cross-References

Local path	Why it matters
609ee083ea51c3521c323f1279dfc4cee0e60467:README.md	Code/repo anchor for implementation cross-check.
609ee083ea51c3521c323f1279dfc4cee0e60467:projects/mmdet3d_plugin/uniad/dense_heads/motion_head.py	Code/repo anchor for implementation cross-check.
609ee083ea51c3521c323f1279dfc4cee0e60467:projects/mmdet3d_plugin/uniad/dense_heads/motion_head_plugin/__init__.py	Code/repo anchor for implementation cross-check.
609ee083ea51c3521c323f1279dfc4cee0e60467:projects/mmdet3d_plugin/uniad/dense_heads/motion_head_plugin/base_motion_head.py	Code/repo anchor for implementation cross-check.
609ee083ea51c3521c323f1279dfc4cee0e60467:projects/mmdet3d_plugin/uniad/dense_heads/motion_head_plugin/modules.py	Code/repo anchor for implementation cross-check.
609ee083ea51c3521c323f1279dfc4cee0e60467:projects/mmdet3d_plugin/uniad/dense_heads/motion_head_plugin/motion_deformable_attn.py	Code/repo anchor for implementation cross-check.

Reproduction Notes

• Treat this note as paper/code reading material first; do not interpret mini-data smoke tests as paper reproduction. [PDF p.6, Method]
• Before running experiments, verify the local code entry points above against the paper method terminology and dataset protocol. [PDF p.3, Abstract]
• If a claim is not linked to a PDF page or code path in this note, treat it as an implementation hypothesis rather than established paper fact.

Local Asset Identity

延伸解读与背景补充

生成：2026-07-21 · 来源条数 2 · 模型 heuristic · 需人工核验数字

问题再表述

围绕「UniAD: Planning-oriented Autonomous Driving」的核心问题与动机（待结合全文核验）。

方法要点补强

• 见原文方法章节；以下为基于摘要/摘录的要点提示。
• Planning-oriented Autonomous Driving Yihan Hu1,2∗ , Jiazhi Yan…

与相近工作的关系

与相近工作的关系待核验；请对照 related work。

常见误读

• 勿仅凭摘要推断未给出的数值指标。

自测问题

1. 这篇工作的输入/输出表示是什么？（UniAD: Planning-oriented Autonomous Driving）
2. 训练目标与评测协议各是什么？
3. 主要失败模式或局限是什么？

外部解读索引

• [2212.10156] Planning-oriented Autonomous Driving — 2026-07-21 — 官方摘要/二次页面（自动抓取）
• [2212.10156] Planning-oriented Autonomous Driving — 2026-07-21 — 官方摘要/二次页面（自动抓取）

关键图表

方法结构（重绘）

flowchart LR
  A["输入 / 观测"] --> B["表示 / 编码"]
  B --> C["推理 / 解码"]
  C --> D["输出 / 动作或检测"]
  %% method sketch for: UniAD: Planning-oriented Autonomous Driving

方法结构示意（重绘；细节以原论文为准，待 PDF 核验）。

论文摘录图/表

来源：原论文约 p.1（arch）；学习用途摘录。

来源：原论文约 p.7（table）；学习用途摘录。

英文自动分析（可折叠）

展开英文 Paper Card / AI deep analysis

Paper Card

Field	Content
One-line takeaway	UniAD organizes perception, prediction, occupancy, and planning tasks around the final ego-planning objective.
Problem	Optimizing perception tasks independently does not guarantee planning-oriented scene understanding.
Representation	dense BEV + task queries
Input / Output	Input: multi-camera temporal scenes and map/ego metadata. Output: boxes, tracks, maps, motion, occupancy, and ego plan.
Core Mechanism	Task queries and BEV features form a chained multi-task system whose outputs feed downstream motion and planning.
Training / Evaluation	The paper reports multiple nuScenes task metrics; full data and staged artifacts are needed for meaningful results.
Reproduction Status	Too long for first-stage reproduction; mini can only validate local pieces.
Compare With	VAD compresses planning inputs into vectors; SparseDrive uses sparse scene tokens; SSR questions explicit perception-task necessity.
Failure/Risk	Multi-stage checkpoints, map expansion, motion anchors, temporal infos, and old OpenMMLab stack are major risks.

原文摘录与素材（可折叠）

展开 Extract / Selections / Local assets

Source Anchors

Anchor	What to verify	Source	Short original cue
Title and abstract	Use to verify paper identity and top-level contribution.	[PDF p.1, Abstract]	UniAD: Planning-oriented Autonomous Driving
Motivation	Use to verify the problem statement and why the work is needed.	[PDF p.1, Abstract]	Introduction
Core method	Use to verify the main modeling mechanism and module names.	[PDF p.3, Abstract]	Method
Key module terms	Use to verify exact component names before editing the note.	[PDF p.3, Abstract]	planning-oriented
Dataset and protocol	Use to verify data dependencies: nuScenes full trainval + CAN bus + map/motion artifacts.	[PDF p.6, Method]	nuScenes
Metrics and results	Use to verify metric names and reported benchmark context.	[PDF p.6, Method]	L2
Experiments or ablation	Use to verify which claims are experimentally supported.	[PDF p.5, Method]	Experiments
Position in related work	Use to verify the claimed relationship to neighboring methods.	[PDF p.13, Related Work]	Related Work
Conclusion or limits	Use to verify final claims and remaining constraints.	[PDF p.8, Method]	Conclusion

反向链接

• 端到端学习
• 端到端驾驶
• 自动驾驶数据集与评测矩阵
• 自动驾驶数据集与评测矩阵
• 自动驾驶方法谱系
• 自动驾驶方法谱系

---

学习导航

标记为已读

• 所属 Topic: ad-end-to-end-driving
• 学习路径: 学习路线 · 论文库
• 上一篇: SparseDriveV2: Scoring is All You Need for End-to-End Autonomous Driving
• 下一篇: End-to-End Autonomous Driving: Challenges and Frontiers
• 返回 Topic