A custom AI reviewer for overclaiming

PerfectPaper’s overclaim agent reads a manuscript backwards: it takes each claim in the abstract and discussion, traces it to the specific result meant to support it, and reports every claim whose wording outruns that support. For each one it writes a replacement sentence the evidence does support. Copy the brief below into a custom agent slot.

The agent proposes wording rather than advising the author to be more cautious, because advice to be careful hands the problem back unsolved. The pattern it targets is near-universal and rarely deliberate. Methods and results are written carefully by the person who did the work; the abstract and the final discussion paragraph are written last, often by someone else, and written to be interesting.

What overclaiming is, and how often it appears in published work

Overclaiming is wording whose strength exceeds the evidence the same paper reports — a claim that is broader, more causal, more mechanistic, larger, more clinical or more novel than the result behind it. The methodological literature calls the published form spin. Chiu, Grundy and Bero, in their systematic review in PLoS Biology (2017;15(9):e2002173), quote the definition Boutron and colleagues gave it: “specific reporting strategies, whatever their motive, to highlight that the experimental treatment is beneficial, despite a statistically non-significant difference of the primary outcome, or to distract the reader from statistically non-significant results.” Their own review works from a broader version — reporting practices that distort the interpretation of results so they are viewed more favourably — because the narrow wording covers only trials.

The prevalence is measured, not asserted. Boutron, Dutton, Ravaud and Altman (JAMA 2010;303(20):2058–2064) examined 72 randomised trials whose primary outcome was statistically non-significant and coded them for spin. Spin appeared in the abstract conclusions of 58.3% (95% CI 46.1–69.8%), in the abstract results sections of 37.5% (95% CI 26.4–49.7%), and in the titles of 18.0% (95% CI 10.0–28.9%). Chiu and colleagues’ systematic review of 35 reports found the nature of spin varies by design, with the highest and most variable prevalence in trials, and named two of the commonest practices as detracting from statistically non-significant results and inappropriately using causal language.

The vocabulary itself has drifted. Vinkers, Tijdink and Otte (BMJ 2015;351:h6467) counted preselected words across all PubMed abstracts from 1974 to 2014 and found positive words rose from 2.0% (1974–80) to 17.5% (2014), a relative increase of 880%, with individual terms including “robust”, “novel” and “innovative” rising up to 15,000%. Negative words rose from 1.3% to 3.2% over the same period, a relative increase of 257%. An author writing in the register everyone else now uses is not being reckless; the register moved.

The governing standard is explicit and short. The ICMJE Recommendations, under Preparing for Submission, instruct authors in the Discussion guidance to “Link the conclusions with the goals of the study but avoid unqualified statements and conclusions not adequately supported by the data”, and to “distinguish between clinical and statistical significance”. On abstracts the same document states that “authors need to ensure that they accurately reflect the content of the article”. Those three clauses are the whole of what this agent enforces.

Common questions about overclaiming

What counts as overclaiming in a research paper?

Wording whose strength exceeds the evidence: generalising beyond the system tested, asserting cause from association, asserting mechanism from phenotype, implying magnitude the effect size does not support, or claiming clinical relevance from preclinical work. Claiming priority is a sixth form, and it is the one a manuscript cannot verify about itself.

The six forms of overclaiming this agent separates

PerfectPaper’s overclaim agent separates six forms — generality, causality, mechanism, magnitude, clinical relevance and novelty — because each has a different repair, and a finding that reports them together as “tone” tells an author nothing they can act on. A generality overclaim is fixed by adding a scope clause; a magnitude overclaim is fixed by inserting a number; a novelty overclaim usually cannot be fixed inside the manuscript at all.

Generality

Generality overclaiming extends a claim beyond the system tested, the population studied or the conditions used. One cell line, one mouse strain, one cohort, one dose or one assay does not license a general statement about the biology or the disease.

The tell is a subject term that changed between the results and the abstract: results say “in HCT116 cells”, the abstract says “in colorectal cancer”. Reviewers read the swap as an assertion about every colorectal tumour and respond accordingly. The repair is a scope clause in the same sentence as the claim, not in a limitations paragraph 4,000 words later — “in HCT116 and SW480 cells” adds six words and removes the objection. CONSORT 2025 deleted its standalone generalisability item and folded that content into the limitations item at 30, which means a trial report now has one place where scope is expected and no excuse for leaving it implicit. The fix scales with the independence of the evidence, not with how many times a result was counted: a claim broadened across two cohorts recruited at different sites is defensible where the same claim built on two technical replicates of a single cohort is not, because the second case has not actually sampled a wider population — it has measured the same one twice. Where the generality question is specifically about whether a mouse experiment speaks to human disease, use model relevance instead of this agent.

Causality

Causal overclaiming asserts cause where the design supports association. The agent flags the verbs — causes, reduces, increases, improves, prevents, leads to, results in, protects against — applied to a cross-sectional, cohort, registry or correlative result, and quotes both the claim and the result it points at.

Two nuances matter. A randomised trial is not the only design entitled to causal language: an observational study built around a specific causal-inference method — instrumental variables, regression discontinuity, difference-in-differences, or an explicitly framed target-trial emulation — earns the same verbs when it states its identification assumptions, and flagging one of these as association-only is a false positive that trains authors to ignore the agent. A plain cohort or cross-sectional comparison without one of these designs still supports association language only. And a recommendation sentence is a causal claim wearing different clothes: “clinicians should consider screening for X” asserts that acting on X changes outcomes, which no observational association establishes. Where the whole paper turns on this, causal language discipline is the more specific agent, and correlative, not causal covers the reviewer comment itself.

Mechanism

Mechanism overclaiming asserts a mechanism where a correlation or a phenotype was observed. Attributing a phenotype to a target requires evidence that the intervention engaged that target — target engagement, rescue, a separation-of-function mutant, or an orthogonal perturbation with a different modality.

The characteristic sentence is “X regulates Y through the Z pathway”, supported by a knockdown that changed Y and a western blot showing Z components moved. That is consistent with the mechanism and does not demonstrate it, because a knockdown perturbs everything downstream of X. The replacement preserves the finding: “X is required for Y, and loss of X is accompanied by reduced Z signalling.” The specific artifact differs by technique — an siRNA silences unintended transcripts that pair with only its seed region rather than matching the intended target along its length, and a CRISPR knockout clone can differ from its parent for reasons unrelated to the edit, because single-cell cloning fixes whatever variation the parental population already carried — so “we used a knockout instead of a knockdown” is not by itself the fix a reviewer is asking for, and one clone is not an experiment. Reviewers phrase the objection as mechanism not shown, and the usual remedy they ask for is orthogonal validation: a second perturbation of a different modality, such as a chemical inhibitor alongside the genetic knockout, because two methods sharing the same failure mode do not rule it out.

Magnitude

Magnitude overclaiming implies an effect larger than the one reported. The agent flags “substantial”, “marked”, “dramatic”, “profound”, “striking” and “significantly” used as an intensifier, wherever the reported effect size does not support the adjective, and it flags a statistically significant difference described in language that a reader will convert into a clinical impression.

The repair is arithmetic, not adjectival: replace the adjective with the number and its interval. “Substantially improved survival” becomes “improved median overall survival from 11.2 to 13.8 months (HR 0.81, 95% CI 0.69–0.95)” — longer, and immune to the objection. Where a clinically anchored threshold exists, the honest sentence compares the estimate to it; see effect size for how to choose the estimate that carries the comparison. There is a second, quieter source of magnitude overclaiming that has nothing to do with word choice: when a study has low power, the estimates that do clear significance are systematically larger than the true effect. Gelman and Carlin (Perspectives on Psychological Science 2014;9(6):641–651) call this a type M (magnitude) error, and quantify it as an exaggeration ratio that grows as power falls; the same phenomenon is called the winner’s curse elsewhere. An intensifying adjective placed on an already-inflated point estimate compounds the problem twice, which is one more reason to report the interval alongside the number rather than the number alone. The reviewer version of this finding is the effect is not meaningful, and it is much harder to answer after acceptance than before submission.

Clinical relevance

Clinical-relevance overclaiming asserts benefit to patients from evidence obtained in cells, animals or a surrogate endpoint. The agent reports every instance, because this form recurs throughout translational work and is the one that survives longest — it usually sits in the final sentence of the abstract, where it is also the sentence most likely to be quoted by a press office.

Three constructions carry it: “these findings will inform treatment”, “represents a promising therapeutic target”, and any claim of benefit built on a marker rather than an outcome. A change in a surrogate endpoint is a change in a marker until a trial links the two — correlation between the marker and the real outcome is not sufficient on its own, since a marker can track a disease process closely and still fail to capture a treatment’s full effect on how a patient actually fares. Regulators draw exactly this line: a marker with a strong mechanistic or epidemiological rationale is only reasonably likely to predict benefit, while a validated surrogate is one where clinical evidence has actually shown that an effect on the marker predicts the clinical benefit. The replacement converts an assertion into a testable proposition — “these findings identify a hypothesis for clinical testing” — which is both accurate and, for a grant reviewer, stronger. Dose is the other half of this question: an in vivo effect at an exposure no patient will receive is not translational evidence, which translational dose relevance addresses directly.

Novelty

Novelty overclaiming asserts being first, unprecedented or previously unreported. The agent reports these as unverifiable within the manuscript and notes the risk, because it is reading one paper and cannot search the literature to prove a negative.

The risk is asymmetric and worth stating plainly to authors: a first-claim that survives review gains the author very little, and a first-claim that a reviewer refutes with one citation damages the credibility of every other claim in the paper. Editors read a contradicted priority claim as carelessness about the literature. The safe construction states the contribution rather than the ranking — “we show X in Y, which has not been reported in Z” is checkable and specific, while “the first demonstration of X” is a bet. Preprint servers have made that bet harder to win quietly: a preprint posted ahead of submission creates a public, checkable timestamp, so a priority claim now has to hold up against the preprint record of the field, not only against the peer-reviewed literature the author has read. When a reviewer pushes back on positioning rather than on priority, that is a different objection with a different answer; see novelty objections.

Questions about specific forms of overclaiming

Can I say a treatment “causes” improvement if I only ran a correlational study?

Only if the design earns it. A randomised trial, or an observational study built around a specific causal-inference method — instrumental variables, regression discontinuity, difference-in-differences, or an explicitly stated target-trial emulation — can support causal wording if it states its identification assumptions. A plain cohort or cross-sectional comparison without one of these supports association language only: “was associated with,” not “caused.”

Is it overclaiming to call a mouse result “a promising treatment”?

Usually, yes, unless the sentence names what still stands between the result and a patient. “Promising” asserts a clinical relevance a xenograft or knockout model cannot establish alone. The defensible version states the gap directly — “a candidate for evaluation in a model with greater predictive validity” — or names the next step: dose translation, a second species, or a link to a validated outcome.

I only tested one cell line — can I still talk about the disease in general?

Not in the same sentence. Keep the line name in the sentence that draws the conclusion — “in HCT116 cells” — rather than letting it become “in colorectal cancer” by the time the reader reaches the abstract. A second, independent line showing the same result is what licenses the broader noun; one line supporting a general noun is the generality form of overclaiming on its own.

Is “significantly improved” the same thing as a big effect?

No. “Significant” in the statistical sense means the difference is unlikely to be chance at that sample size, not that it is large or noticeable to a patient. A result can be statistically significant and clinically trivial, or clinically important yet underpowered. State the effect size and its interval, and compare it against a clinically anchored threshold where one exists, rather than letting the p-value carry the adjective.

When to use this agent

• Your abstract was written after the results, by a different author
• The discussion argues for clinical or translational relevance
• A reviewer has previously asked you to temper your conclusions
• The paper reports one experiment and the discussion generalises from it
• Your final paragraph contains the phrase these findings suggest

The agent brief

Paste this into a custom agent. Suggested settings: work type analysis, skill level expert, no tools required.

Name: Overclaim check

You are reviewing the distance between what this study demonstrated and what
it says it demonstrated. Work backwards from the claims to the evidence, not
forwards from the data.

Method:

1. Extract every substantive claim from the title, abstract, the opening of
   the discussion, and the final paragraph of the discussion. These four
   locations carry most overclaiming; examine them exhaustively.

2. For each claim, identify the specific result meant to support it: the
   figure, table or statistic. Report any claim you cannot trace to a specific
   result, since an untraceable claim is the strongest form of this finding.

3. For each traceable claim, compare the strength of the wording against the
   strength of the evidence, considering:

   - Generality. Does the claim extend beyond the system tested, the
     population studied, or the conditions used? A result from one cell line,
     one cohort, or one model does not license a general statement.
   - Causality. Does the wording assert cause where the design shows
     association?
   - Mechanism. Does the claim assert a mechanism where a correlation or a
     phenotype was observed? Attributing a phenotype to a target requires
     evidence of engagement with that target.
   - Magnitude. Does the wording imply an effect larger than reported? Report
     statistically significant differences described as substantial,
     marked or dramatic where the effect size does not support it.
   - Clinical relevance. Does the claim assert benefit to patients from
     evidence obtained in cells, animals, or a surrogate endpoint? Report
     every instance; this is the most common form in translational work.
   - Novelty. Does the claim assert being first or unprecedented? Report
     these as unverifiable within the manuscript and note the risk.

4. For every overclaim, quote the sentence, name which of the above applies,
   and write a replacement sentence that the evidence does support. The
   replacement must preserve the finding. Do not simply recommend caution and
   do not propose deleting the claim.

5. Separately, report any place where the paper undersells a result the data
   support, which is less common and worth telling the author about.

Do not evaluate whether the study should have been designed differently. Take
the evidence as given and assess only whether the words match it.

How to adapt the brief to your paper

The brief is written for a paper with an abstract, a discussion and figures, and three edits adapt it to most other shapes. Each edit is an addition to step 3, so the backwards-reading method and the replacement-sentence requirement stay intact.

For a registered trial, add a determination: compare every claim in the abstract conclusion against the pre-specified primary outcome, and report any conclusion that rests on a secondary or exploratory endpoint without saying so. A conclusion drawn from a secondary endpoint after the primary missed is one of the spin strategies Boutron and colleagues coded, and it is the one an editor is most likely to send back.

For a methods or resource paper, replace the clinical-relevance check with a performance-generality check: does the claimed performance extend beyond the datasets used to develop it, and was any reported accuracy measured on data held out before development began? Claims of generalisation from in-sample performance are the overfitting form of this same defect.

For a paper that explicitly adopts a causal framework, add the exemption verbatim so the agent does not flag correct work: where the manuscript states its identification assumptions, causal language is appropriate and should be reported as appropriate rather than flagged.

Two limits are worth keeping as written. The brief forbids re-designing the study, so an agent that starts recommending more experiments has been given too broad a scope — see how to write a custom reviewer agent for why the negative instructions are the load-bearing ones. And the brief covers original research articles; a narrative review or a meta-analysis has a different claim structure and needs a different brief.

What this agent catches

Worked example: tracing an abstract sentence back to a figure

Consider a constructed but ordinary case: a manuscript whose abstract ends “inhibition of kinase K restores chemosensitivity in resistant tumours and represents a promising therapeutic strategy.” The agent’s job is to find the result that sentence points at and compare the two.

Working backwards, the supporting evidence is Figure 4: a xenograft experiment in one resistant cell line, eight mice per arm, in which combination treatment reduced mean tumour volume relative to chemotherapy alone at day 21. That single result carries four separate claims. “Resistant tumours” is a generality overclaim — the evidence covers one line in one host strain. “Restores chemosensitivity” is a magnitude overclaim if the combination arm merely slowed growth rather than returning it to the sensitive-line baseline, which is what the word restores asserts. “Therapeutic strategy” is a clinical-relevance overclaim resting on a mouse experiment. “Promising” carries no information and is the word the press release will keep.

A replacement that preserves every part of the finding: “In a chemoresistant xenograft model, combining kinase K inhibition with chemotherapy reduced tumour growth relative to chemotherapy alone, identifying kinase K as a candidate for further preclinical evaluation.” Nothing was deleted and no result was weakened; the sentence now describes what Figure 4 shows. The agent would also check whether the mice are the unit of analysis actually used, because a claim built on a comparison of replicate tumours rather than animals is a pseudoreplication problem underneath the wording problem, and the reporting that would settle it belongs in the methods rather than in a rewritten abstract.

How to check for overclaiming yourself

Overclaiming is found by reading the paper backwards, in one sitting, with the figures open beside the text. The procedure below takes well under an hour on a full-length article and surfaces the claims a reviewer is most likely to challenge.

Build a claim ledger. List every substantive claim from the title, the abstract’s final two sentences, the first paragraph of the discussion and the last, one line each. Against each, write the figure or table number that supports it. Any line whose second column stays empty is the finding — an untraceable claim is the strongest form of overclaim, and it is usually the sentence the author is proudest of.

Compare subject terms across sections. Put the results heading and the abstract sentence side by side and look for the noun that grew: cells to cancer, cohort to population, model to disease, marker to outcome. The generality overclaim is almost always visible as a single changed word.

Search the manuscript for the verbs and intensifiers. Causes, reduces, prevents, protects, leads to, results in, demonstrates, establishes, confirms, proves; substantially, markedly, dramatically, profoundly, strikingly, robustly, notably. Each hit needs the specific result beside it or a rewrite.

Check the abstract against the pre-specified analysis. For a trial or a pre-registered study, the abstract conclusion should rest on the primary endpoint. A conclusion built on a secondary endpoint is not disqualified; it is disqualified only if it does not say so.

Read the last sentence of the abstract as a journalist would. That sentence is the one the press office and the reader will carry away, and it is where Boutron’s 58.3% concentrates. Guidance on writing it is in how to write an abstract.

Questions about catching overclaiming before submission

How do I check my own paper for overclaiming before submission?

List every claim in the title, abstract and first and last discussion paragraphs, and write the figure or table number beside each one. Any claim with no number beside it is the finding. Then compare the subject nouns between the results and the abstract, and search for the causal verbs and the intensifiers — substantially, markedly, dramatically — that assert magnitude no effect size supports.

Reviewer 2 said my conclusions aren’t supported by the data — what do I do?

Do not soften the verb; trace the sentence back to the figure or table and check whether the noun, verb or population changed between the result and the conclusion. Write the exact scope, number or comparison the evidence supports in that same sentence. Reviewers writing this comment want a precise correction, not a more careful-sounding restatement — vague hedging reads as evasion, not as fixed.

My PI wants me to say we’re the first to show this — is that safe to write?

State it as a bounded, checkable claim rather than an unbounded one: “to our knowledge, this has not been reported in [population or system]” rather than “the first demonstration of X.” An unqualified priority claim that a reviewer refutes with a single citation damages trust in every other claim in the paper, while a scoped version keeps the credit and survives a literature check you cannot run yourself.

What reviewers say when a claim outruns its evidence

Reviewers rarely write “you have overclaimed”. They write the specific version, and these comments decide papers rather than merely annoying authors.

“The conclusions are not supported by the data presented.” “The authors extrapolate from a single cell line to colorectal cancer generally.” “The title asserts a causal relationship that the study design cannot establish.” “The abstract states that the treatment improves outcomes; the primary endpoint was not met and this is not stated in the abstract.” “The claim of a novel mechanism rests on correlative evidence; target engagement is not demonstrated.” “The word ‘restores’ is not supported by Figure 4, which shows partial and non-significant reduction.” “The authors claim the first demonstration of this phenomenon; see [reference], which reported it several years ago.” “The clinical implications section is disproportionate to the preclinical evidence and should be removed or heavily qualified.”

The last two are the expensive ones. A refuted priority claim invites a reviewer to distrust the rest of the paper, and an editor asked to adjudicate a “first” claim will usually side with the reviewer holding the citation. Answering these after review is a rewriting exercise conducted under time pressure, and it is far quicker to do this work before submission than to defend it afterwards.

When the obvious remedy is unavailable

The obvious remedy — add the scope clause, add the number, downgrade the verb — is unavailable more often than review guidance admits, and each case has a specific substitute.

The abstract word limit is already exceeded. A 250-word structured abstract cannot absorb three scope clauses. Spend the words where the risk is: one qualifier on the conclusion sentence buys more than three scattered through the results. Deleting a secondary finding from the abstract to make room for an accurate conclusion is a good trade, because the secondary finding survives in the paper and the inaccurate conclusion would not survive review.

A co-author or a funder wants the translational framing. Convert the claim to a hypothesis rather than dropping it. “These findings support a clinical trial of X in Y” is a strong, defensible sentence that keeps the translational argument and asserts nothing about patient benefit. Reviewers tend to accept a proposed next step more readily than an asserted implication, and a grant panel is reading for exactly that next step; the framing difference is the subject of the study section lens.

The result really is first, and you need the credit. State the contribution as a checkable proposition with its boundary — “to our knowledge, X has not been reported in Y” — rather than an unbounded priority claim. The hedge that most authors reach for instead, “may represent the first”, claims priority and disclaims responsibility for it in the same sentence, and reviewers read that as evasion rather than caution.

You cannot run the experiment that would license the claim. Move the claim into the limitations section as a stated gap with a named next experiment. A limitations paragraph that says which experiment would test the mechanism reads as command of the problem; one that says results “should be interpreted with caution” reads as an admission with no analysis. See how to write a limitations section.

What is not overclaiming

Not every strong sentence is an overclaim, and an agent that flags all of them is worse than no agent, because authors stop reading its findings. Four categories are correct as written.

Causal language in a paper that adopts a causal framework. A manuscript stating its identification assumptions, its estimand and its sensitivity analyses has earned the verb. The brief exempts it explicitly.

Labelled speculation. A discussion paragraph that says “we speculate that” and then reasons about a mechanism is doing what a discussion is for. The failure mode is speculation that migrates into the abstract without its label.

A confident statement of a well-supported result. Precision is not hedging. “Treatment reduced 12-month mortality from 18.4% to 11.9%” is a strong claim and an accurate one, and softening it would be a different defect.

Underselling. The agent reports underselling separately for this reason: a cautious author who buries a result the data support has produced a paper that is harder to review and easier to reject as incremental. That finding is rarer than overclaiming and worth acting on when it appears.

How this differs from the built-in review

PerfectPaper’s standing review team already includes a conclusion-validity specialist that compares conclusions against evidence. The overclaim agent differs in method rather than subject: it works backwards from claims to evidence and requires a written replacement sentence for each finding, which changes what the author receives. A finding that says “this claim exceeds the evidence” prompts a decision; a finding that supplies the sentence to use instead prompts an edit.

The agent also reports underselling, which no other agent looks for. Overlaps deliberately with model relevance and causal language — where those apply to your paper, prefer the specific one.

Review my manuscript

Related agents

The agents below share a review with this one; you can run three at a time. See Custom AI reviewer agents for the full library, organised by research area.

• How to write your own — use it when you are writing your own brief rather than pasting one.
• Ethics, consent and data sharing — use it for approval statements and consent scope for deposition.
• Study section lens — use it when you need the grant reviewer’s question, not the referee’s.

Frequently asked questions

Does the agent tell me to delete my conclusions?

No, and the brief explicitly forbids it. For every overclaim the agent must supply a replacement sentence that preserves the finding rather than deleting or softening it into vagueness, because the underlying result is normally sound and only the wording exceeds it — the fix is a rewrite, not a retraction.

Can an AI tool actually catch spin in my abstract before I submit?

The overclaim agent can trace claims to results reliably, which is the mechanical part of the check: does this sentence’s noun, verb and population match the figure or table it points at. It cannot verify a novelty claim against the literature or judge whether a co-author’s framing choice is deliberate, which is why the brief reports those as flags for a human decision rather than automatic fixes.

Is adding “may” enough to fix an overclaim?

No. A modal verb attached to an untraceable claim leaves the claim untraceable, and reviewers read “may represent the first demonstration” as a priority claim with the responsibility removed. The repair is a scope clause, a number or a named next experiment in the same sentence as the claim, not a softer verb in front of it.

Can the agent tell me where I undersold a result?

Yes — underselling is reported separately, as its own finding, not folded into the same tone bucket as overclaiming. Underselling is rarer, but worth acting on: a cautious author who buries a result the data support more strongly than the text admits has produced a paper that reads as weaker and more incremental than the evidence justifies.