Pangram separated AI editing from AI-written peer reviews in a 25-report experiment
The authors propose different rules for papers and reviewers: investigate flagged research for errors, but reject reviews that outsource human judgment.
Loading page…
The authors propose different rules for papers and reviewers: investigate flagged research for errors, but reject reviews that outsource human judgment.
Listen to this story
In a November 16, 2026 paper, economists Dario Caldara and Jesús Fernández-Villaverde argue that journals should screen research papers and referee reports but treat detector flags differently: a paper merits closer review, while AI-delegated peer judgment may warrant rejection. Their test found Pangram separated light edits from fully rewritten or generated reviews, but it involved only 25 published reports, two language models and one detector. The authors frame screening as a way to verify research, not as proof that flagged work is unsound.
Pangram classified proofreading and language-edited versions as almost entirely human, while rewritten and paper-generated reviews were almost entirely AI-generated.
Every AI-generated review recommended major revision, even though each source paper had already been published.
For research papers, the authors propose closer scrutiny after a flag and rejection at the first substantive error, not blanket exclusion.
An AI detector distinguished light editing from machine-written peer reviews in an experiment built around 25 published reports. In a paper dated November 16, 2026, economists Dario Caldara and Jesús Fernández-Villaverde use that finding to argue for journal screening—with an important distinction: an AI flag on a research paper should prompt scrutiny, not automatic rejection.
The researchers started with 25 referee reports published by Nature Communications. These are the assessments reviewers write when evaluating a manuscript. They asked two large language models to perform four tasks: proofread each report, edit its language, rewrite it, and produce a new review using only the published paper. They then scored every version with Pangram, a detector of AI-generated text.
The distinction matters to their proposed rules. They want researchers to retain help with grammar and expression while remaining responsible for the work behind the words. Their experiment examines whether a detector can distinguish those uses from more extensive generation, rather than treating every encounter with an AI tool as equivalent.
The generated reviews also shared a recommendation: every one called for major revision of a paper that had already been published. The experiment covered one detector, two models and 25 source reports. The authors acknowledge a further limit of detection: text classified as human may still be AI-generated, including when someone deliberately tries to evade screening.
The paper’s central argument concerns verification, not writing style. Before AI, the authors contend, producing carefully written text took enough effort to offer an imperfect signal that someone had thought through the analysis. Cheap generation weakens that signal. A diligently checked paper and an unchecked one can look alike, leaving readers to spend more time determining which deserves their trust.
Their concern extends beyond individual bad submissions. Time spent checking careless work takes attention away from thoughtful research, they argue. Their economic model makes a consequential assumption: authors who skip engaging with the writing are also more likely to skip checking the analysis. That is the proposed link between a detector flag and closer scrutiny—not a finding that every AI-written paper contains errors.
The authors’ proposed journal policies distinguish assistance from delegated responsibility, and research papers from the reports used to judge them. They recommend screening both, alongside detailed rules explaining acceptable AI use and disclosure.
The stricter treatment reflects their view of what a reviewer contributes: personal judgment that an editor cannot obtain simply by running a prompt. The authors also disclose using AI themselves to challenge their arguments, suggest developments and improve grammatical flow. They report no financial interest in the companies cited.
For authors, their proposed boundary is not a ban on AI doing research tasks. AI may help execute data work, code or proofs, they argue, provided the author verifies the output. Responsibility for the result stays with the researcher.
Loading discussion...
Join the conversation
Explain whether useful feedback is enough, or personal judgment is part of the obligation.
Be the first to share a perspective or an experience.
Reader comments
Newest comments first. Replies stay oldest first.