PaperGym: Rubric-Centered Evolution for Research-Plan Generation

Abstract

Rubric-as-reward training is a prevalent strategy for improving research-plan generation in LLMs, but existing rubric-based datasets suffer from data contamination, evaluation bias, and insufficient domain specialization. We propose PaperGym, a low-leakage benchmark that turns each arXiv paper into a training environment: a four-stage extraction pipeline decouples research questions from reference answers, and dual-dimensional rubrics evaluate proposals along methodological innovation and experimental design. The same rubrics are reused in a two-stage rubric-centered scheme — rubric-conditioned OPSD followed by GRPO with rubric-as-rewards.

PaperGym keeps criterion leakage at only 3.7%. Models trained on it improve by +5.6 / +5.0 / +4.8 over their bases on Qwen3-1.7B / 4B / 8B, and the fine-tuned Qwen3-8B reaches 73.48 on ResearchQA, surpassing the much larger Kimi K2.6 (73.19).

Highlights

Existing rubric-based benchmarks suffer from contamination, evaluation bias, and limited domain specialization. PaperGym addresses all three at their source.

3.7%
Criterion leakage — roughly 3–9× lower than existing rubric-based benchmarks (12%–34%).

20,000
Instances in PaperGym-20k across five domains, with held-out Innov / Design test sets.

+5.6 / +5.0 / +4.8
Average gains over base models for Qwen3-1.7B / 4B / 8B — beating SFT, either stage alone, and the reversed ordering.

73.48
ResearchQA score with fine-tuned Qwen3-8B, surpassing the much larger Kimi K2.6 (73.19).

58.1%
Win rate in three-way blind comparisons on ResearchPlanGen-ML, vs. 28.2% for a RubricHub-trained baseline.

Method · Data Generation

Each paper is decomposed into disjoint sections; questions and reference answers are synthesized from non-overlapping sources — eliminating contamination at its origin.

Overview of the PaperGym framework: (a) data generation from arXiv papers into four stages; (b) two-stage policy training with rubric-based OPSD followed by rubric-as-rewards GRPO.

The PaperGym framework. (a) Data Generation: arXiv papers are parsed into four stages to synthesize questions and answers; rubrics are generated, merged, ranked, and filtered. (b) Two-Stage Policy Training: rubric-based OPSD followed by rubric-as-rewards GRPO.

PaperGym operates in four stages:

Parse. Each arXiv paper is decomposed into four stages — Research Goal, Background, Research Method, and Experimental Design (excluding concrete numerical results) — via a map-reduce procedure using Qwen3-235B-A22B.
Decouple. The research question is synthesized from Research Goal + Background; the reference answer from Research Method + Experimental Design — disjoint sections, so contamination is prevented at the source.
Rubricify. Specialized rubrics (R_Q + R_A) are merged, deduplicated, ranked, and filtered to the top n = 10 criteria per instance, covering both methodological innovation and experimental design; a validated general rubric is applied uniformly.
Output. The resulting PaperGym-20k dataset (20,000 instances) and the held-out PaperGym-Innov / PaperGym-Design test sets.

Rubric-Centered Training

Both training stages are supervised by the same rubrics and together form a rubric-centered paradigm: OPSD warms up with dense token-level guidance, then GRPO explores toward high-reward regions using the rubrics as rewards.

Stage 1 · Warm-up

Rubric-Conditioned OPSD

A rubric-conditioned teacher guides the unconditioned student on on-policy rollouts; the student matches the teacher by minimizing token-level KL divergence:

ℒ_OPSD = 𝔼 Σ_n KL( π_θ(· | x, ℛ, ŷ_<n) ∥ π_θ(· | x, ŷ_<n) )

The rubric keeps a broader set of valid continuations than answer-conditioned distillation, aligned with the stage-2 reward.

Stage 2 · Explore

GRPO with Rubric-as-Rewards

A frozen copy of the base model self-grades each candidate response criterion by criterion; the reward combines the specialized and general rubric scores as:

r_i = α · r_i,spec + (1 − α) · r_i,gen · α = 0.7

Setting α = 0.7 emphasizes task-specific scientific fit while keeping pressure on general proposal quality; GRPO then optimizes the policy.

Implementation. Qwen3 models at the 1.7B / 4B / 8B scales (bf16). OPSD fine-tunes with LoRA (r = 64, α = 128) at a learning rate of 5×10⁻⁶ and batch size 8; GRPO uses verl with 8 responses per prompt via vLLM, a KL penalty of 0.01, and a learning rate of 1×10⁻⁶. The 4B and 8B models serve as their own rubric verifiers, while the 1.7B model is scored by the 4B model.

Dataset

PaperGym-20k is built from publicly available arXiv papers (plain-text LaTeX, cleaner than rendered PDFs) — ~10,000 computer-science papers from January 2015 to December 2025. Each paper is decomposed into four stages — Research Goal, Background, Research Method, and Experimental Design (excluding concrete numerical results) — via a map-reduce procedure using Qwen3-235B-A22B.

This decomposition is what makes the benchmark low-leakage: the question is synthesized from the Research Goal + Background, and the reference answer from the Research Method + Experimental Design — disjoint sections, so contamination is prevented at its source.

Preliminary analysis of PaperGym-20k: domain category breakdown, scorer self-consistency, and pairwise inter-model agreement.

Preliminary analysis. Left: category breakdown of PaperGym-20k across domains. Middle: mean score vs. across-round range for four scoring models over five independent runs. Right: pairwise inter-model agreement on binary rubric verdicts.

Held-out test sets. PaperGym-Innov scores along the innovation dimension (question from Research Goal + Background, reference from Research Method); PaperGym-Design scores along the design dimension (question adds Research Method, reference from Experimental Design). Both sample 400 papers per domain (cs, econ, physics) released after January 2026, two instances each. Rubric criteria split 63.8% Method / 36.2% Experiment; four scoring models are highly self-consistent across runs, with the small Qwen3-8B reaching ~80% agreement with Kimi K2.6.

Data Leakage

An LLM judge checks whether each rubric criterion is directly inferable from the question alone. The four-stage decomposition keeps leakage at only 3.7% — roughly a 3–9× reduction over existing benchmarks.

Dataset	Leakage Rate
PaperGym-20k	3.73%
PaperGym-Innov	4.71%
PaperGym-Design	4.97%
HealthBench	11.90%
RubricHub Science	17.39%
ResearchQA	19.22%
ResearchPlanGen-ML	31.29%
ResearchPlanGen-ArXiv	34.10%

Table 2. Data-leakage comparison. PaperGym maintains the lowest leakage rate among open-source alternatives.

Main Results

OPSD + GRPO achieves the best score on every test set at every scale — improving the average by +5.6 / +5.0 / +4.8 over the base models on Qwen3-1.7B / 4B / 8B — and the fine-tuned Qwen3-8B surpasses Kimi K2.6 on ResearchQA (73.48 vs. 73.19).

Table 1. Main results across model scales and training strategies on RubricHub Science, ResearchPlanGen-ML, ResearchQA, and the in-domain PaperGym-Innov / PaperGym-Design test sets. Best result per column within each section is highlighted; shaded rows are the full OPSD + GRPO pipeline. All Qwen3 base models are evaluated in thinking mode.

Win-Rate Evaluation

On ResearchPlanGen-ML — independent of all training data — a judge compares three models sharing the same training regimen: trained on PaperGym-20k, trained on RubricHub Science, and the untrained base. The PaperGym-trained model wins 58.1% of Overall Score comparisons (vs. 28.2% for RubricHub-trained and 13.7% for the base model).

Overall Score. Overall scientific quality and practical usefulness as a research blueprint.

Goal Alignment. Understanding and addressing the core research goal, including identifying key challenges and formulating relevant hypotheses.

Novel Insight. Originality and research insight, including novel hypotheses, perspectives, or methodologies connected to the research goal.

Scientific Soundness. Scientific rigor and coherence, from hypothesis to methodology to evidence, with consideration of baselines, controls, and validation.

Execution Quality. Clarity, feasibility, and implementation structure, balancing ambition with practicality.

Expected Impact. Likelihood of generating meaningful and interpretable research outcomes that advance understanding or provide methodological contributions.

PaperGym-20k-trained RubricHub-trained Untrained base

BibTeX

If you find PaperGym useful in your research, please consider citing:

@misc{wang2026papergymrubriccenteredevolutionresearchplan,
      title={PaperGym: Rubric-Centered Evolution for Research-Plan Generation},
      author={Yuhan Wang and Zhengxi Lu and Yuchen Yan and Kaitao Song and Wenqi Zhang and Weiming Lu and Jun Xiao and Yueting Zhuang and Yongliang Shen},
      year={2026},
      eprint={2608.31119},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2608.31119},
}