Evaluating Large Language Models Trained on Code
Evaluating Large Language Models Trained on Code
Section titled “Evaluating Large Language Models Trained on Code”⚠️ AI 生成 · 建议对照原文 本页为自动整理的学习笔记；关键数据与引用如需引用，请回查 PDF / 官方版本。
学习档位 自动卡
类型 文献 · 更新 2026-07-19
所属 LLM 与语言推理
Discovery evidence
Section titled “Discovery evidence”- topic:
llm-language-reasoning - sources:
openalex - retrieved_at: 2026-07-20
- query: large language model reasoning robotics planning
- doi:
10.48550/arxiv.2107.03374 - score_total: 62
- suggested_tier:
recent
Relevance
Section titled “Relevance”(no prose relevance explanation — numeric score only or HTTP source)
Snippets
Section titled “Snippets”(no snippet evidence in candidate pool)
AI analysis (heuristic fallback, needs-source-verification)
Section titled “AI analysis (heuristic fallback, needs-source-verification)”Status: metadata_only · Source: metadata
One-line takeaway
Section titled “One-line takeaway”Metadata-only card for Evaluating Large Language Models Trained on Code.
Problem & motivation
Section titled “Problem & motivation”—
Inputs & outputs
Section titled “Inputs & outputs”—
Core representation
Section titled “Core representation”—
Method pipeline & key modules
Section titled “Method pipeline & key modules”—
Training objectives
Section titled “Training objectives”—
Datasets & metrics
Section titled “Datasets & metrics”—
Main results
Section titled “Main results”—
Ablation findings
Section titled “Ablation findings”—
Limitations & failure modes
Section titled “Limitations & failure modes”—
Related work positioning
Section titled “Related work positioning”—
Open-source / code anchors
Section titled “Open-source / code anchors”—
Reproduction risks
Section titled “Reproduction risks”—
Practical value
Section titled “Practical value”—
Evidence
Section titled “Evidence”—
延伸解读与背景补充
Section titled “延伸解读与背景补充”生成：2026-07-21 · 来源条数 2 · 模型
heuristic· 需人工核验数字
围绕「Evaluating Large Language Models Trained on Code」的核心问题与动机（待结合全文核验）。
方法要点补强
Section titled “方法要点补强”- 见原文方法章节；以下为基于摘要/摘录的要点提示。
- 本地摘录暂缺。
与相近工作的关系
Section titled “与相近工作的关系”与相近工作的关系待核验；请对照 related work。
- 勿仅凭摘要推断未给出的数值指标。
- 这篇工作的输入/输出表示是什么？（Evaluating Large Language Models Trained on Code）
- 训练目标与评测协议各是什么？
- 主要失败模式或局限是什么？
外部解读索引
Section titled “外部解读索引”- [2107.03374] Evaluating Large Language Models Trained on Code — 2026-07-21 — 官方摘要/二次页面（自动抓取）
- [2107.03374] Evaluating Large Language Models Trained on Code — 2026-07-21 — 官方摘要/二次页面（自动抓取）
方法结构（重绘）
Section titled “方法结构（重绘）”flowchart LR A["输入 / 观测"] --> B["表示 / 编码"] B --> C["推理 / 解码"] C --> D["输出 / 动作或检测"] %% method sketch for: Evaluating Large Language Models Trained on Code方法结构示意（重绘；细节以原论文为准，待 PDF 核验）。
英文自动分析（可折叠）
Section titled “英文自动分析（可折叠）”展开英文 Paper Card / AI deep analysis
Paper Card
Section titled “Paper Card”| Field | Content |
|---|---|
| Year | 2021 |
| Authors | Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Pondé de Oliveira Pinto, Jared Kaplan, Harrison Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri |
| arXiv | — |
| DOI | 10.48550/arxiv.2107.03374 |
| Topics | llm-language-reasoning |
原文摘录与素材（可折叠）
Section titled “原文摘录与素材（可折叠）”展开 Extract / Selections / Local assets
Selections
Section titled “Selections”llm-language-reasoning: tier=watch rank=4 score=62 — auto refresh 2026-07-19 sources=openalex
Extract excerpt
Section titled “Extract excerpt”(no PDF text available; metadata-only card)