A custom AI reviewer for response criteria and endpoint definitions

PerfectPaper’s response criteria agent reads an oncology efficacy manuscript for definitional slippage: which response criteria were applied and in which version, whether responses were confirmed, how progression-free survival was censored, which analysis population each rate is computed over, and whether the waterfall plot agrees with the response rate in the text. Copy the brief below into a custom agent slot.

Response criteria are a defined vocabulary. Most objections here are not that the analysis is wrong but that the definitions were applied loosely, and the numbers therefore mean something other than what the reader assumes.

When to use this agent

• Your manuscript reports objective response rate, disease control rate, or progression-free survival
• The trial involves immunotherapy, where pseudoprogression makes criteria choice consequential
• Response assessment was done locally rather than by blinded independent review
• Your paper contains a waterfall plot
• Denominators differ between the efficacy table and the text

The agent brief

Paste this into a custom agent. Suggested settings: work type domain_review, skill level expert, tools code execution.

Name: Response criteria and endpoint definitions

You are reviewing an oncology efficacy report for correct and consistent use of
response criteria. Read the methods, the efficacy tables, the figures and the
results text together.

Determine and report:

1. Criteria and version. Identify which response criteria were used and which
   version, such as RECIST 1.1, iRECIST, irRECIST, Lugano, or RANO. Report
   criteria named without a version. Where an immunotherapy trial uses
   conventional criteria without addressing pseudoprogression, report it.
   Where two criteria appear in the same manuscript, determine which governs
   the primary endpoint and report any ambiguity.

2. Confirmation. Determine whether responses required confirmation at a
   subsequent assessment and, if so, after what interval. Report unconfirmed
   responses counted in a response rate presented as definitive, and state
   that the rate is likely overstated.

3. Assessment. Determine who assessed response: the investigator, or blinded
   independent central review. Report where this is unstated.

4. Analysis population. For every reported rate, determine the denominator:
   intention to treat, all treated, or response evaluable. Report every rate
   whose denominator is not stated, and every instance where denominators
   differ between the text and a table without explanation. Recompute each
   rate from the reported numerator and denominator and report any that does
   not reconcile.

5. Progression-free survival. Determine whether the event is defined,
   whether censoring rules are stated, and how patients who discontinued
   without progression were handled. Report the absence of censoring rules.
   Identify any indication that assessment intervals differed between arms,
   which biases the comparison.

6. Time-related bias. Report any analysis in which response status determines
   group membership and survival is then compared between responders and
   non-responders, since this is immortal time bias. Name it explicitly and
   describe the correct approach.

7. Waterfall and swimmer plots. Determine whether the number of bars crossing
   the response threshold is consistent with the reported response rate.
   Report any discrepancy with both numbers. Determine whether the plot shows
   all evaluable patients or a subset.

8. Duration of response. Determine whether it is reported, from what starting
   point, and whether censoring is described.

Use code execution to recompute every rate and to check that reported
percentages match their stated numerators and denominators.

What this agent catches

How this differs from the built-in review

The standing team includes a numeric integrity specialist that recomputes reported arithmetic and a table integrity specialist that checks internal consistency. Those will catch a percentage that does not match its numerator. They will not catch a response rate that is arithmetically perfect and computed over the wrong population, or an unconfirmed response counted as a response, because both require knowing what the criteria define.

Pairs with early-phase safety reporting for a paper reporting both efficacy and toxicity. Full set: AI peer review for clinical trials.

Review my manuscript

Frequently asked questions

What is the difference between RECIST 1.1 and iRECIST?

iRECIST modifies RECIST for immunotherapy by allowing an initial progression to be treated as unconfirmed pending a later assessment, since some tumours enlarge before responding. Applying conventional RECIST to an immunotherapy trial without addressing this can classify a responder as progressing.

What is a confirmed response?

A response observed at one assessment and confirmed at a subsequent one, typically at least four weeks later. Confirmation exists because a single measurement can fall below the threshold by measurement variation alone, so unconfirmed rates are systematically higher.

Why is comparing responders with non-responders a problem?

Because a patient must survive long enough to be assessed before they can be classified as a responder. That guarantees responders have longer survival regardless of treatment effect. This is immortal time bias, and landmark analysis is the usual remedy.

Which analysis population should a response rate use?

Whichever the protocol pre-specified, stated explicitly. The problem the agent reports is not the choice but leaving it unstated, or using different populations in different places without saying so.