SSR: Navigation-guided Sparse Scene Representation for End-to-End Autonomous Driving

SSR: Navigation-guided Sparse Scene Representation for End-to-End Autonomous Driving

⚠️ AI 生成 · 建议对照原文 本页为自动整理的学习笔记；关键数据与引用如需引用，请回查 PDF / 官方版本。

学习档位 中文笔记

类型 文献 · 更新 2026-07-19

标签 autonomous-driving · planning · end-to-end-learning · nuscenes

所属 端到端驾驶 · 移动机器人与导航

Migration status: imported from ad_projs@a823662; source anchors and claims remain needs-source-verification.

• Paper: https://arxiv.org/abs/2409.18341
• Code: https://github.com/PeidongLi/SSR at 9b17910e1f86c6f324583efd71045b3b8eb1819e

中文学习笔记（自动生成，需核验）

Topic: ad-end-to-end-driving · Tier: needs-review · Year: 2024 · Venue: —
Evidence level: partial · 本地全文: 是 · 建议阅读: ~45 分钟
Paper: https://arxiv.org/abs/2409.18341
Code: —
Generator: grok

为什么值得读

SSR提出导航引导的仅16个稀疏场景token表示，无需任何感知子任务监督与昂贵标注，结合时序自监督未来特征预测，在nuScenes与CARLA上显著降低L2误差与碰撞率的同时实现10倍级加速，为实时可扩展E2EAD提供新范式。

一句话结论

SSR用导航引导的16个稀疏token与时序自监督替代所有感知监督，实现高效高精度端到端自动驾驶。

问题定义

现有E2EAD大多继承BEV感知框架，依赖检测/建图/占用等监督子任务提取场景信息，导致标注昂贵、模型庞大、推理训练慢，难实时部署与扩展；如何在无辅助感知监督下高效聚焦导航相关关键场景元素？

前置知识

BEV特征构建（如BEVFormer）、Transformer/交叉注意力、TokenLearner与TokenFuser、E2EAD范式（UniAD/VAD等）、nuScenes开环规划指标（L2/碰撞率）、CARLA闭环指标（DS/RC/IS）、导航命令（左转/右转/直行）。

核心贡献

• 提出人类启发的导航引导稀疏场景表示框架，仅用16个自适应token聚焦关键元素，显著降低计算与标注成本。
• 引入未来特征预测器，利用时序上下文对动态场景变化进行自监督，消除感知子任务监督需求。
• 在开环（nuScenes）与闭环（CARLA Town05 Long）上达到SOTA性能与效率，建立实时E2EAD新基准。

方法流程

多视角图像经BEV编码器得密集BEV特征Bt；导航命令cmd经SE层编码得导航感知Bnavi；Scenes TokenLearner从中提取16个稀疏场景token St；waypoint queries与St交叉注意力后经MLP并按cmd选择输出规划轨迹T；训练时Future Feature Predictor用T与St预测未来BEV并与真实未来BEV做L2自监督，总损失为模仿损失+Lbev。

关键模块和设计取舍

Scenes TokenLearner：SE导航编码+BEV TokenLearner（空间注意力+全局平均池化）+自注意力，将密集BEV压缩为Ns=16稀疏token，取舍为聚焦导航意图相关区域以减冗余；Planning：waypoint queries（Nm×Nt）交叉注意力稀疏token后MLP选轨迹，仅L1模仿监督；Future Feature Predictor（仅训练）：MLN注入运动信息得dreaming queries，自注意力预测未来token，TokenFuser重建密集BEV后L2监督，取舍为用时序一致性替代感知标注且不直接监督稀疏token以避免焦点漂移。

数据集、实验设置与指标

开环：nuScenes，L2位移误差与碰撞率（CR，含行人；默认MAX协议，也报AVG），3s时域0.5s间隔，评估1/2/3s及平均，FPS；闭环：CARLA Town05 Long，Route Completion（RC）、Infraction Score（IS）、Driving Score（DS=RC×IS）；实现基于VAD-Tiny，ResNet-50（开环640×360）/ResNet-34（闭环），BEV 100×100→16×256，导航命令3类，开环8×3090训练12epoch约11h，闭环4×3090 60epoch。

主要结果（需原文证据）

相对UniAD：L2误差相对降低27.2%、碰撞率相对降低51.6%，推理快10.9×、训练快13×；相对VAD-Base在CARLA Town05 Long驾驶分数提升48.6点；nuScenes MAX协议SSR平均L2=0.75m、平均CR=0.15%、FPS=19.6；AVG协议平均L2=0.39m、平均CR=0.06%；表中优于多数带辅助任务方法且无标注。

局限、失败场景与适用边界

提取中未提供明确局限/失败场景/适用边界讨论，待来源核验。

与前序 / 同期 / 后续方法的关系

对比任务特定监督范式（UniAD级联多辅助任务、VAD向量化、PARA-Drive并行、GenAD生成式、OccNet等）与稀疏尝试（SparseDrive/UAD仍用数百任务查询、LAW视图latent、跳过BEV但仍依赖监督）；SSR完全消除感知子任务，用导航引导自适应稀疏token+时序自监督，更充分体现E2E优势并超越前述方法。

官方代码与复现建议

官方代码https://github.com/PeidongLi/SSR；基于VAD-Tiny设置，开环ResNet-50、BEV100×100→16×256、AdamW lr=5e-5、Limi+Lbev权重1.0、8×3090 batch1/GPU训练12epoch约11h；闭环ResNet-34+TCP头、目标点拼接命令、4×3090 batch32训练60epoch；建议严格按VAD-Tiny与文中导航命令/TokenLearner配置复现，注意MAX/AVG指标差异与行人纳入碰撞计算。

推荐阅读顺序

先读Abstract与Fig.1/2/3把握动机与范式对比→Introduction问题与贡献→Method 3.1-3.4（尤其STL与FFP结构Fig.4）→Experiments 4.1-4.3主结果与实现细节→Related Work定位→最后Table1与代码链接。

自测问题

1. Q: SSR用多少个导航引导token表示场景？为何可无需感知监督？ A: 仅16个；通过导航命令引导TokenLearner聚焦关键区域，并用未来特征预测器的时序自监督（预测BEV与真实未来BEV L2）替代标注。
2. Q: Scenes TokenLearner的核心步骤是什么？ A: SE将cmd编码到BEV得Bnavi，TokenLearner空间注意力+全局池化提取token，再自注意力增强。
3. Q: 相对UniAD的主要效率与精度提升数据（原文）？ A: L2相对降27.2%、碰撞率相对降51.6%，推理10.9×快、训练13×快。
4. Q: Future Feature Predictor如何工作且为何不直接监督稀疏token？ A: 轨迹T经MLN得dreaming queries，自注意力预测未来token，TokenFuser重建BEV后L2监督；因连续帧焦点可能不同，避免直接监督Ŝt+1。
5. Q: 开环评估默认用哪种指标协议？与VAD有何差异？ A: 默认MAX（最新帧），含行人；VAD用AVG（历史平均），文中两者均报。

原文证据

• Abstract: utilizes only 16 navigation-guided tokens as Sparse Scene Representation… achieves a 27.2% relative reduction in L2 error and a 51.6% decrease in collision rate to UniAD in nuScenes, with a 10.9× faster inference speed and 13× faster training time. Moreover, SSR outperforms VAD-Base with a 48.6-point improvement on driving score in CARLA’s Town05 Long benchmark.
• Section 1 Introduction: How can we efficiently identify and focus on the crucial parts of the scene without auxiliary perception supervision? … using just 16 tokens guided by navigation commands.
• Section 3.2: Scenes TokenLearner (STL) module to extract scene queries St = [si]Ns i=1 ∈ R Ns×C from the BEV feature, where Ns is the number of scene queries… Bnavi t = SE(Bt , cmd). St = T LBEV (Bnavi t ).
• Section 3.4 / Eq.(15): Ltotal = Limi + Lbev . … we supervise B̂t+1 directly using an L2 loss with the real future BEV feature Bt+1.
• Table 1 / Section 4.3: SSR (Ours) … None … 0.24 0.65 1.36 0.75 0.00 0.10 0.36 0.15 19.6 … SSR‡ (Ours) … 0.18 0.36 0.63 0.39 0.01 0.04 0.12 0.06 19.6
• Abstract / Code: Code is available at https://github.com/PeidongLi/SSR.

Evidence-backed Reading Notes

Topic	Evidence-backed note	Source	Short original cue
Problem	Explicit perception tasks can add training complexity and may not align perfectly with final planning objectives.	[PDF p.2, Method]	Published as a conference paper at ICLR 2025 obj
Representation	The note treats the method as navigation-guided sparse scene tokens because the paper’s method pages introduce the relevant representation/module vocabulary.	[PDF p.3, Method]	Method
Core mechanism	Navigation guidance selects or constructs sparse scene tokens that emphasize planning-relevant information.	[PDF p.3, Method]	Navigation-guided
Input / Output	Input: temporal camera features and navigation/planning context. Output: sparse planning-relevant representation and ego plan.	[PDF p.6, Method]	nuScenes
Training / Evaluation	The paper is published at ICLR 2025 and evaluates E2E driving/planning on nuScenes-style protocols.	[PDF p.6, Method]	L2
Relationship	VAD is the closest vectorized planning baseline; UniAD is heavier multi-task; SparseDrive focuses sparse system efficiency.	[PDF p.2, Method]	Published as a conference paper at ICLR 2025 obj
Failure/Risk	The central claim needs full-data evaluation; mini smoke tests cannot prove whether perception tasks are necessary.	[PDF p.3, Method]	Navigation-guided
Reproduction boundary	Best read after VAD; mini data can validate shapes, not the perception-task necessity claim.	[PDF p.6, Method]	nuScenes

Code Cross-References

Local path	Why it matters
9b17910e1f86c6f324583efd71045b3b8eb1819e:README.md	Code/repo anchor for implementation cross-check.
9b17910e1f86c6f324583efd71045b3b8eb1819e:projects/configs/SSR/SSR_e2e.py	Code/repo anchor for implementation cross-check.
9b17910e1f86c6f324583efd71045b3b8eb1819e:docs	Code/repo anchor for implementation cross-check.
9b17910e1f86c6f324583efd71045b3b8eb1819e:projects	Code/repo anchor for implementation cross-check.
9b17910e1f86c6f324583efd71045b3b8eb1819e:projects/__init__.py	Code/repo anchor for implementation cross-check.
9b17910e1f86c6f324583efd71045b3b8eb1819e:projects/configs	Code/repo anchor for implementation cross-check.

Reproduction Notes

• Treat this note as paper/code reading material first; do not interpret mini-data smoke tests as paper reproduction. [PDF p.6, Method]
• Before running experiments, verify the local code entry points above against the paper method terminology and dataset protocol. [PDF p.3, Method]
• If a claim is not linked to a PDF page or code path in this note, treat it as an implementation hypothesis rather than established paper fact.

Local Asset Identity

延伸解读与背景补充

生成：2026-07-21 · 来源条数 2 · 模型 heuristic · 需人工核验数字

问题再表述

围绕「SSR: Navigation-guided Sparse Scene Representation for End-to-End Autonomous Driving」的核心问题与动机（待结合全文核验）。

方法要点补强

• 见原文方法章节；以下为基于摘要/摘录的要点提示。
• Published as a conference paper at ICLR 2025 NAVIGATION -G UIDED S PARSE S CENE R EPRESENTA - …

与相近工作的关系

与相近工作的关系待核验；请对照 related work。

常见误读

• 勿仅凭摘要推断未给出的数值指标。

自测问题

1. 这篇工作的输入/输出表示是什么？（SSR: Navigation-guided Sparse Scene Representation for End-to-End Autonomous Driving）
2. 训练目标与评测协议各是什么？
3. 主要失败模式或局限是什么？

外部解读索引

• [2409.18341] Navigation-Guided Sparse Scene Representation for End-to-End Autonomous Driving — 2026-07-21 — 官方摘要/二次页面（自动抓取）
• [2409.18341] Navigation-Guided Sparse Scene Representation for End-to-End Autonomous Driving — 2026-07-21 — 官方摘要/二次页面（自动抓取）

关键图表

方法结构（重绘）

flowchart LR
  A["输入 / 观测"] --> B["表示 / 编码"]
  B --> C["推理 / 解码"]
  C --> D["输出 / 动作或检测"]
  %% method sketch for: SSR: Navigation-guided Sparse Scene Representation for End-to-End Autonomous Dri

方法结构示意（重绘；细节以原论文为准，待 PDF 核验）。

论文摘录图/表

来源：原论文约 p.1（arch）；学习用途摘录。

来源：原论文约 p.7（table）；学习用途摘录。

英文自动分析（可折叠）

展开英文 Paper Card / AI deep analysis

Paper Card

Field	Content
One-line takeaway	SSR argues for navigation-guided sparse scene representation to reduce reliance on explicit perception tasks in E2E driving.
Problem	Explicit perception tasks can add training complexity and may not align perfectly with final planning objectives.
Representation	navigation-guided sparse scene tokens
Input / Output	Input: temporal camera features and navigation/planning context. Output: sparse planning-relevant representation and ego plan.
Core Mechanism	Navigation guidance selects or constructs sparse scene tokens that emphasize planning-relevant information.
Training / Evaluation	The paper is published at ICLR 2025 and evaluates E2E driving/planning on nuScenes-style protocols.
Reproduction Status	Best read after VAD; mini data can validate shapes, not the perception-task necessity claim.
Compare With	VAD is the closest vectorized planning baseline; UniAD is heavier multi-task; SparseDrive focuses sparse system efficiency.
Failure/Risk	The central claim needs full-data evaluation; mini smoke tests cannot prove whether perception tasks are necessary.

原文摘录与素材（可折叠）

展开 Extract / Selections / Local assets

Source Anchors

Anchor	What to verify	Source	Short original cue
Title and abstract	Use to verify paper identity and top-level contribution.	[PDF p.1, Abstract]	SSR: Navigation-guided Sparse Scene Representation for End-to-End Autonomous Driving
Motivation	Use to verify the problem statement and why the work is needed.	[PDF p.2, Method]	Published as a conference paper at ICLR 2025 obj
Core method	Use to verify the main modeling mechanism and module names.	[PDF p.3, Method]	Method
Key module terms	Use to verify exact component names before editing the note.	[PDF p.3, Method]	Navigation-guided
Dataset and protocol	Use to verify data dependencies: nuScenes full trainval + CAN bus + VAD-style infos.	[PDF p.6, Method]	nuScenes
Metrics and results	Use to verify metric names and reported benchmark context.	[PDF p.6, Method]	L2
Experiments or ablation	Use to verify which claims are experimentally supported.	[PDF p.6, Method]	Experiments
Position in related work	Use to verify the claimed relationship to neighboring methods.	[PDF p.2, Method]	Published as a conference paper at ICLR 2025 obj
Conclusion or limits	Use to verify final claims and remaining constraints.	[PDF p.10, Paper text]	Limitations

反向链接

• 移动机器人与导航
• 端到端驾驶
• 自动驾驶方法谱系
• 自动驾驶方法谱系

---

学习导航

标记为已读

• 所属 Topic: ad-end-to-end-driving
• 学习路径: 学习路线 · 论文库
• 上一篇: RAD: Training an End-to-End Driving Policy via Large-Scale 3DGS and Reinforcement Learning
• 下一篇: Senna-2: Aligning VLM and End-to-End Driving Policy for Consistent Decision Making
• 返回 Topic