Back Original

The Hallway Track No. 034: PaperRaven, LLM peer review framing, AlphaGenome 2

A common design question appears across several items this week: at which point in the research pipeline should AI output be checked by an independent process? PaperRaven addresses it at the literature stage by refusing any citation the model proposes but cannot locate in OpenAlex or Crossref. The DIME team's code-review agent addresses it at the analysis stage, prompted by an audit finding coding errors in 26 of 110 published economics papers despite most being computationally reproducible. A NeurIPS benchmark addresses it at the review stage and finds that model selection alone produces a 50-percentage-point gap in error detection rates. The LLM framing experiment shows what is at stake when that check is absent: a single prompt change raised aggregate acceptance rates more than fourfold on the same set of published papers.

Literature and Data Infrastructure

Code Quality and Review

Peer Review and Verification