CFTrack: Center-based Radar and Camera Fusion for 3D Multi-Object Tracking
CFTrack: Center-based Radar and Camera Fusion for 3D Multi-Object Tracking
Section titled “CFTrack: Center-based Radar and Camera Fusion for 3D Multi-Object Tracking”⚠️ AI 生成 · 建议对照原文 本页为自动整理的学习笔记；关键数据与引用如需引用，请回查 PDF / 官方版本。
学习档位 中文笔记
类型 文献 · 更新 2026-07-19
所属 时序感知、跟踪与记忆
中文学习笔记（自动生成，需核验）
Section titled “中文学习笔记（自动生成，需核验）”Topic: temporal-perception-tracking · Tier: watch · Year: 2021 · Venue: —
Evidence level: partial · 本地全文: 是 · 建议阅读: ~40 分钟
Paper: https://arxiv.org/abs/2107.05150
Code: —
Generator: grok
为什么值得读
Section titled “为什么值得读”首个雷达-相机端到端深度融合的3D多目标检测与跟踪框架，用深度+速度增强关联，对遮挡/重叠鲁棒，实时性适合自动驾驶感知。
基于CenterFusion中心点雷达-相机融合检测，配合深度/速度/2D位移贪心关联，实现在线3D多目标跟踪。
自动驾驶3D多目标跟踪需处理遮挡与相似外观交互；现有方法多单模态（相机或LiDAR）或融合仅用于检测、关联未充分利用多模态（如雷达速度/深度）。
CenterNet/CenterFusion中心点检测与雷达-相机融合、跟踪-by-检测或联合检测跟踪范式、贪心/匈牙利关联、nuScenes与AMOTA指标。
- 提出首个雷达与相机传感器融合、深度网络端到端训练的3D多目标检测与跟踪框架（据作者所知）。
- 提出贪心关联算法，融合物体深度、速度与2D位移，提升遮挡/重叠场景关联准确性。
输入当前/前一帧图像与雷达点云及前帧检测热图；修改CenterFusion骨干提取融合特征并回归中心热图、尺寸、深度、速度、旋转、属性及位移；用基于像素位移+深度差+速度差的加权代价贪心算法关联轨迹ID，输出3D框与跟踪ID及全局速度。
关键模块和设计取舍
Section titled “关键模块和设计取舍”检测：扩展CenterFusion，加入前帧图像/雷达/检测热图输入，新增位移回归头（L1损失），融合特征预测位移；雷达以柱状展开后视锥关联到初步3D检测再融合深度/速度；关联：同类物体代价=α·Lpixel+β·Ldepth+δ·Lvelocity（否则∞），在半径r内匹配最低代价先验检测，未匹配则新建轨迹。取舍：在线局部关联（不重关联离开帧物体）、贪心而非全局优化以保实时。
数据集、实验设置与指标
Section titled “数据集、实验设置与指标”nuScenes（1000序列：训练700/验证150/测试150，每序列40帧，含相机/雷达/LiDAR）；主指标AMOTA（不同召回阈值下MOTA加权平均，含IDS/FP/FN）；在线实时（约35ms/图像或28FPS）。
主要结果（需原文证据）
Section titled “主要结果（需原文证据）”nuScenes上达20.0 AMOTA，优于所有视觉基3D跟踪方法及基线LiDAR方法AB3DMOT；比CenterTrack提升约4倍；运行时35ms/图像（或28帧/秒）。
局限、失败场景与适用边界
Section titled “局限、失败场景与适用边界”待来源核验（摘录未详述失败场景、遮挡极限、超参数敏感性或适用边界）。
与前序 / 同期 / 后续方法的关系
Section titled “与前序 / 同期 / 后续方法的关系”属联合检测跟踪在线方法，基于CenterFusion检测与CenterTrack点基框架/贪心关联；关联改进自仅2D位移，加入深度/速度；对比DeepSORT、Tracktor、AB3DMOT、Hu et al. LSTM等2D/3D MOT及地图增强方法。
官方代码与复现建议
Section titled “官方代码与复现建议”待来源核验（摘录未提及官方代码或复现细节）。
推荐阅读顺序
Section titled “推荐阅读顺序”摘要与引言贡献→相关工作定位→Preliminaries（CenterFusion）→方法（检测网络修改+关联代价）→实验设置与主结果→图表（架构与位移可视化）。
- Q: CFTrack的核心输入与输出是什么？ A: 输入：当前/前一帧图像与雷达点云、前帧检测热图；输出：3D检测框、跟踪ID与绝对速度（车辆坐标系）。
- Q: 关联代价函数包含哪些项？如何处理类别不一致？ A: Lpixel（中心2D位移平方和）+Ldepth（深度差平方）+Lvelocity（速度分量差平方），加权α/β/δ；类别不同则代价为∞。
- Q: 与CenterTrack的主要区别是什么？ A: 加入雷达深度/速度信息到检测与关联，融合特征预测位移，代价不再仅2D位移。
- Q: 在nuScenes上的主要定量结果是什么？ A: 20.0 AMOTA，优于视觉方法与AB3DMOT基线，比CenterTrack高约4倍，实时约28-35ms。
- Q: 为什么雷达对跟踪有帮助？ A: 提供远距检测、全天候鲁棒性、准确速度（利于路径/位移预测）与低处理开销。
- Abstract (page 1): We evaluate our method on the challenging nuScenes dataset, where it achieves 20.0 AMOTA and outperforms all vision-based 3D tracking methods in the benchmark, as well as the baseline LiDAR-based method. Our method is online with a runtime of 35ms per image
- Introduction (page 1-2): the first to propose a radar and camera sensor fusion framework for 3D multi-object detection and tracking using a deep network trained end-to-end. Second, we propose a greedy algorithm that incorporates objects’ depth and velocity in addition to their 2D displacement
- Introduction (page 2): Our fusion-based object tracking algorithm achieves 20.0% AMOTA, outperforming CenterTrack [4] by a factor of 4, while running at 28 frames per second.
- Section IV-C Object Association (page 4): Costt,t−1 = α · Lpixel + β · Ldepth + δ · Lvelocity if ct=ct−1 ; ∞ otherwise. Lpixel=(xt−xt−1)2+(yt−yt−1)2 ; Ldepth=(dt−dt−1)2 ; Lvelocity=(vxt−vxt−1)2+(vyt−vyt−1)2
- Fig. 1 caption (page 3): An additional regression head (“Dis”) is added to the model, which uses the fused radar and image features to predict objects displacement in consecutive frames. The greedy algorithm in the association step uses the displacement, depth and velocity of each object to associate it to previous detections.
Discovery evidence
Section titled “Discovery evidence”- topic:
autonomous-driving - sources:
arxiv - retrieved_at: 2026-07-20
- query: autonomous driving survey end-to-end perception planning
- arxiv:
2107.05150 - score_total: 39
- suggested_tier:
watch
Relevance
Section titled “Relevance”(no prose relevance explanation — numeric score only or HTTP source)
Snippets
Section titled “Snippets”(no snippet evidence in candidate pool)
延伸解读与背景补充
Section titled “延伸解读与背景补充”生成：2026-07-21 · 来源条数 2 · 模型
heuristic· 需人工核验数字
围绕「CFTrack: Center-based Radar and Camera Fusion for 3D Multi-Object Tracking」的核心问题与动机（待结合全文核验）。
方法要点补强
Section titled “方法要点补强”- 见原文方法章节；以下为基于摘要/摘录的要点提示。
- CFTrack: Center-based Radar and Camera Fusion for 3D Multi-Object Tracking …
与相近工作的关系
Section titled “与相近工作的关系”与相近工作的关系待核验；请对照 related work。
- 勿仅凭摘要推断未给出的数值指标。
- 这篇工作的输入/输出表示是什么？（CFTrack: Center-based Radar and Camera Fusion for 3D Multi-Object Tracking）
- 训练目标与评测协议各是什么？
- 主要失败模式或局限是什么？
外部解读索引
Section titled “外部解读索引”- [2107.05150] CFTrack: Center-based Radar and Camera Fusion for 3D Multi-Object Tracking — 2026-07-21 — 官方摘要/二次页面（自动抓取）
- [2107.05150] CFTrack: Center-based Radar and Camera Fusion for 3D Multi-Object Tracking — 2026-07-21 — 官方摘要/二次页面（自动抓取）
方法结构（重绘）
Section titled “方法结构（重绘）”flowchart LR A["输入 / 观测"] --> B["表示 / 编码"] B --> C["推理 / 解码"] C --> D["输出 / 动作或检测"] %% method sketch for: CFTrack: Center-based Radar and Camera Fusion for 3D Multi-Object Tracking方法结构示意（重绘；细节以原论文为准，待 PDF 核验）。
论文摘录图/表
Section titled “论文摘录图/表”
来源：原论文约 p.1（arch）；学习用途摘录。
来源：原论文约 p.5（qualitative）；学习用途摘录。
英文自动分析（可折叠）
Section titled “英文自动分析（可折叠）”展开英文 Paper Card / AI deep analysis
Paper Card
Section titled “Paper Card”| Field | Content |
|---|---|
| Year | 2021 |
| Authors | Ramin Nabati, Landon Harris, Hairong Qi |
| arXiv | 2107.05150 |
| DOI | 10.1109/ivworkshops54471.2021.9669223 |
| Topics | temporal-perception-tracking |
| Paper | https://arxiv.org/abs/2107.05150 |
原文摘录与素材（可折叠）
Section titled “原文摘录与素材（可折叠）”展开 Extract / Selections / Local assets
Selections
Section titled “Selections”temporal-perception-tracking: tier=watch score=56 — auto refresh 2026-07-19 sources=arxiv,openalex
Extract excerpt
Section titled “Extract excerpt”CFTrack: Center-based Radar and Camera Fusion for 3D Multi-Object Tracking Ramin Nabati, Landon Harris, and Hairong Qi
Abstract— 3D multi-object tracking is a crucial component in cle is essential for many of the tasks crucial to autonomous the perception system of autonomous driving vehicles. Tracking navigation, such as path planning and obstacle avoidance [2]. all dynamic objects around the vehicle is essential for tasks such To increase reliability and accuracy, the perception system as obstacle avoidance and path planning. Autonomous vehicles are usually equipped with different sensor modalities to improve in an autonomous vehicle is usually equipped with multiple accuracy and reliability. While sensor fusion has been widely sensors with different sensing modalities such as cameras,
arXiv:2107.05150v1 [cs.CV] 11 Jul 2021 used in object detection networks in recent years, most existing radars and LiDARs. Incorporating the multi-modal sensory multi-object tracking algorithms either rely on a single input data into an object tracking framework for autonomous modality, or do not fully exploit the information provided by driving applications is not a trivial task. It requires an multiple sensing modalities. In this work, we propose an end- to-end network for joint object detection and tracking based efficient, accurate and reliable fusion algorithm capable of on radar and camera sensor fusion. Our proposed method utilizing the information embedded in different modalities uses a center-based radar-camera fusion algorithm for object in real time. Most multi-modal MOT methods use multiple detection and utilizes a greedy