A custom AI reviewer for in vivo experimental rigour

PerfectPaper’s in vivo rigour agent reads an animal study for the design elements that determine whether its causal claim holds: how animals were allocated to groups, who was blinded and at which step, whether the group size was justified before the experiment, which animals were excluded and why, and whether the outcome measure is defined precisely enough to reproduce.

Copy the brief below into a custom agent slot. The agent goes further than checklist compliance. A checklist asks whether randomisation is mentioned; this agent asks whether what is described is randomisation, and whether the conclusion survives if it is not.

The gap the agent exists to close is measured. Macleod and colleagues, in “Risk of bias in reports of in vivo research: a focus for improvement” (PLOS Biology 2015;13(10):e1002273), assessed a random sample of 146 in vivo publications and found randomisation reported in 20% of the 134 studies in which it would have been appropriate, blinded assessment of outcome in 3%, and a sample size calculation in none. In a further set of 1,173 publications from leading UK institutions, a sample size calculation appeared in 1.4% and inclusion or exclusion criteria in 10.4%. Those four items are exactly the four the agent is built to interrogate.

When to use this agent

• Your manuscript reports a tumour growth, survival or treatment experiment in animals
• Allocation to groups is described as random without saying how
• Measurements were taken by someone who knew the group assignment
• Some animals were removed from analysis
• You are submitting to a journal that requires an ARRIVE checklist
• Group sizes are round numbers (n = 5, n = 6, n = 10) with no stated basis
• The n in a figure legend does not match the n in the methods
• A reviewer has already written that the experiment was not replicated or that the sample size is too small

What in vivo experimental rigour means

In vivo experimental rigour means the set of design and reporting features that let a reader decide whether a difference between animal groups was caused by the intervention rather than by allocation, measurement or analysis. Rigour is not a synonym for effort, replicate count or statistical significance; it is a property of the design, fixed before the first animal is dosed.

Two documents define the substance. Landis and colleagues, reporting an NINDS workshop in Nature (2012;490(7419):187–191), proposed a core set of four minimum standards: randomisation with the method reported, blinding including allocation concealment and blinded outcome assessment, sample-size estimation computed at design time, and data handling covering pre-specified stopping rules, inclusion and exclusion criteria, outlier handling and attrition. The ARRIVE guidelines 2.0 (Percie du Sert et al., PLOS Biology 2020;18(7):e3000410) turn the same ground into the Essential 10: study design, sample size, inclusion and exclusion criteria, randomisation, blinding or masking, outcome measures, statistical methods, experimental animals, experimental procedures, and results. The ARRIVE Recommended Set carries the ethical statement, competing interests, protocol registration and data access.

The agent reads for the substance behind those items rather than their presence. A sentence reading “animals were randomised into treatment and control groups” satisfies a checklist and answers nothing, because it names no mechanism, no time point relative to tumour implantation, and no person.

Allocation: what counts as randomisation

Randomisation in an animal experiment is a described mechanism that assigns each animal to a group by chance, applied at a stated point in the timeline, with the resulting assignment concealed from the person who dosed the animal. A statement that animals were randomised, with no mechanism, is unverifiable and the agent reports it as such.

Named methods a manuscript can state, in ascending order of what they control:

Simple randomisation. Each animal is assigned by a random number generator or a published random sequence. Names the tool. Leaves group sizes and baseline tumour volumes unequal by chance, which matters at n = 6.

Block randomisation. Assignment proceeds in blocks (say, blocks of four across two arms) so group sizes stay balanced through the experiment. Report the block size, and note that a predictable block size can unblind the last animal in each block.

Stratified randomisation. Animals are ranked on a baseline variable — most often tumour volume on the day of first dosing — divided into strata, and randomised within stratum. This is the standard design for a treatment study started at a target tumour volume, and it should be reported as stratified randomisation rather than as randomisation.

Minimisation. Each animal is allocated to the arm that minimises imbalance across several baseline covariates at once. Appropriate for small n; requires a stated algorithm.

The allocations that are not randomisation, and which the agent names as systematic or convenience allocation, are: assigning by cage, assigning in arrival or handling order, taking animals from one end of a rack, and hand-balancing tumour volumes to match group means without a chance element. Cage-based allocation is the most consequential, because cage is a source of shared microbiome, hierarchy, food access and handling, and it converts the cage into the effective experimental unit — the unit of analysis problem, and a first cousin of a batch effect.

What a reviewer writes: “The method of randomisation is not stated. Please describe how the allocation sequence was generated, when allocation occurred relative to tumour implantation, and whether the person administering treatment had access to the sequence.” What to write instead: “On day 7 after implantation, when mean tumour volume reached 100 mm³, 40 mice were ranked by tumour volume, divided into 10 strata of 4, and randomised within stratum to four arms using the NC3Rs Experimental Design Assistant; cage cards were coded by an investigator who took no further part in the experiment.”

A specialist qualification worth making explicit: stratified randomisation on baseline tumour volume controls for imbalance at the moment of allocation, but it does not control for measurement error in the baseline reading itself. Where the stratifying measurement is noisy — a caliper reading on an irregular subcutaneous tumour, for instance — the more defensible analysis is not finer stratification but an analysis of covariance that adjusts the endpoint for each animal’s actual recorded baseline value, rather than for the stratum it happened to be assigned to. The methodological case is Vickers and Altman (BMJ 2001;323(7321):1123–1124): analysis of covariance gives the same answer whether or not the groups differ at baseline, and generally has greater power to detect a treatment effect than comparing follow-up values or change scores. Reviewers who ask for “baseline-adjusted analysis” are asking for this, not for a repeat of the randomisation statement.

Common questions about randomisation and allocation

My reviewer wrote “the method of randomisation is not stated” — what do I actually add?

Add the mechanism, the timing, and who held the key: state how the allocation sequence was generated (a random-number generator, a published random sequence, or software such as the NC3Rs Experimental Design Assistant), the day relative to implantation or dosing that allocation happened, and who was blind to the sequence when treatment started. A named tool and a day number resolve this comment; the word “randomised” alone does not.

Is assigning mice by cage, or in arrival order, the same as randomising them?

No. Assigning by cage, by arrival order, or by position on a rack is systematic or convenience allocation, not randomisation, because it contains no chance element and it confounds treatment group with cage — a shared source of microbiome, hierarchy and handling effects. Report it as systematic allocation rather than randomisation, and expect a reviewer to ask whether cage, not treatment, explains the result.

Can I still call it randomised if I balanced tumour volumes by hand before assigning groups?

No — hand-balancing tumour volumes to equalise group means is a deliberate, non-random assignment, even when the intention is fair grouping. The design that is randomisation and achieves the same balance is stratified randomisation: rank animals by baseline tumour volume, divide into strata, then randomise within each stratum. Report whichever you actually did; “matched groups” and “stratified and randomised” describe different procedures.

Blinding: three separate steps, reported separately

Blinding in an animal experiment is three decisions, not one, and a manuscript can honestly claim one while failing the others. The agent reports the administrator, the outcome assessor and the analyst separately, because a single sentence saying “the study was blinded” hides which of the three actually was.

Allocation concealment means the person assigning treatment on day one could not foresee the next assignment. It is broken by an unconcealed sequence and by small predictable blocks.

Blinded conduct means the person handling and dosing the animal does not know the arm. It is frequently impossible — vehicle and drug can differ in colour, viscosity or route — and the honest response is to say which step failed and why, not to omit the sentence.

Blinded outcome assessment is the step that most changes a number, and caliper measurement is its worst case: manual, repeated over weeks, on a tumour whose edge the assessor defines, usually by the person who knows the arm. Macleod’s random sample of 146 publications reported blinded outcome assessment in 3%.

Practical mechanisms a manuscript can name: cage cards and data sheets coded by a third party who holds the key until the database is locked; treatment syringes prepared and labelled by pharmacy or a colleague outside the experiment; an independent assessor who measures tumours and never sees the treatment log; automated readouts, such as bioluminescence quantified from images with arm labels stripped from filenames; histology and image scoring performed on filename-randomised files, which is also the substance of image quantification integrity.

When blinding of the operator is genuinely unavailable, three fallbacks remain, and the agent looks for them: blind the outcome assessor even if the dosing operator cannot be blinded; blind the analyst by delivering group labels as A/B/C/D until the primary analysis is complete; and pre-specify the analysis so that expectation cannot select the test after the curves are seen, which is the animal-study form of p-hacking.

Welfare and safety are the one accepted reason for leaving a role unblinded, and the guidance draws that exemption narrowly. The ARRIVE 2.0 explanation and elaboration (Percie du Sert et al., PLOS Biology 2020;18(7):e3000411) states that welfare or safety reasons may prevent blinding of animal care staff, but that in most cases blinding is possible, and that it should always be possible to conduct at least some of the stages blind. So a manuscript waiving blinding on welfare grounds owes the reader the specific reason and the stages that were blinded anyway. Where an unblinded role genuinely is needed — someone judging a humane endpoint on a high-dose arm — the defensible design names that role and keeps outcome measurement with a separate, blinded person, reported as two roles rather than resolved into one blanket blinding sentence.

Questions about blinding in animal experiments

Do I have to blind the person measuring tumours if the drug visibly changes the mouse’s fur or behaviour?

Blind what you can, and name what you cannot. If a side effect makes the arm identifiable to whoever measures the tumour, state that directly — “outcome assessment could not be fully blinded because [effect] was visible” — rather than omit the sentence. Where possible, move measurement to a colleague who does not administer treatment, or substitute an automated, arm-blind readout for caliper judgement.

Is it acceptable if only the data analyst was blinded, not the person dosing or measuring?

It’s reportable, but it is the weakest of the three blinding steps, and a reviewer will note which two were not addressed. Analyst blinding — delivering results as group A/B/C/D until the primary analysis is locked — guards against selecting a favourable test after seeing the data; it does not protect the measurement itself from the assessor’s expectation, the step Macleod’s audit found blinded in only 3% of studies.

Why does a reviewer care specifically who held the calipers?

Because caliper measurement is manual, repeated over weeks, and the assessor defines the tumour’s edge by eye — exactly the conditions under which expectation most easily shifts a reading. An unblinded caliper measurement is the single design element most likely to inflate an apparent treatment effect without anyone intending it, which is why it draws comment even when every other design element is sound.

Sample size: justification before the experiment, not power after it

A sample size justification is a calculation made before the experiment that states six things: the effect size the study was designed to detect, the standard deviation or variance assumed and its source, the significance level, the target power, the statistical test the calculation assumed, and the allowance made for attrition. A group size with none of those is an unjustified group size, whatever the n.

Sample size calculation appeared in 0 of 146 publications in Macleod’s random sample and in 1.4% of 1,173 publications from leading UK institutions. The consequence is not only that underpowered experiments miss real effects; an underpowered experiment that reaches significance overstates the effect it found, because only the larger sample estimates cross the threshold. That is why “we found a significant difference, so we were adequately powered” is not an argument, and why a post-hoc power calculation computed from the observed effect answers no question a reviewer asked. See statistical power for the underlying quantity.

The agent also reports the design tells that indicate a sample size fixed by convention or by results rather than in advance: identical n across every experiment in a paper regardless of expected effect size; an n that differs between figure panels of the same experiment with no explanation; a cohort described as “repeated” with the repeats pooled after the fact; and any statement that the experiment was continued, extended or repeated until a difference emerged. Where the design is genuinely exploratory, the agent’s expected response is to say so and to label the result as hypothesis-generating rather than to reverse-engineer a justification.

Where the variance estimate comes from a pilot, the pilot’s own n belongs in the manuscript. A standard deviation taken from three animals is an unstable input, and a power calculation built on it produces a number with false precision.

The guidance goes further than that, and manuscripts rarely reflect it. The ARRIVE 2.0 explanation and elaboration (Percie du Sert et al., PLOS Biology 2020;18(7):e3000411) states that pilot studies alone are unlikely to provide adequate data on variability for a power calculation for a future experiment, and that systematic reviews and previous studies are the more appropriate sources. A pilot supplies an unstable variance estimate and an effect size estimate that tends to run optimistic, and a calculation built on an optimistic input under-recruits for the true, smaller effect. Where a pilot is the only source available, name it as the source and give its n and its estimates rather than presenting the resulting group size as exact.

Exclusions and the numbers that have to reconcile

Exclusions are reported by count, by group, by reason and against a pre-specified criterion, and the agent reconciles every number an animal study reports about how many animals it had. The check is arithmetic and it is the fastest defect in a preclinical manuscript to find.

The numbers that must agree: animals ordered or implanted in the methods; animals per group in the design description; n in each figure legend; the number of points visible in a scatter or growth plot; the number at risk at time zero on a Kaplan-Meier curve; and the degrees of freedom implied by any reported test statistic. A paper stating 10 mice per group, plotting 8 points, and reporting a t-test on 16 degrees of freedom has told the reader three different things about the same experiment.

Legitimate exclusions exist and are reportable: tumours that never established, animals that died of an anaesthetic or dosing accident before the first measurement, and animals that reached a humane endpoint for a cause unrelated to the study. Each needs a criterion set in advance, a count, and the arm it came from. Exclusions that appeared after the data were seen — outliers removed on a statistical rule chosen post hoc, animals dropped for “poor tumour take” defined at analysis, or a whole cage lost from one arm — change the analysed population from the randomised one, and the agent states that consequence rather than noting the omission.

Animals lost to a competing cause are not missing data; they are events of a different type, and treating them as censored assumes the loss was unrelated to treatment. Where an arm loses animals to toxicity while the other does not, the survival comparison needs a competing risks framing, not a footnote.

Where a pre-specified exclusion removes a substantial share of a group — as a rough guide, more than roughly one in ten animals — reporting the result twice, with and without the excluded animals, is more defensible than reporting only the analysed set, because it lets the reader judge whether the exclusion itself, rather than the treatment, is doing the work. ARRIVE 2.0 asks for the criteria and the count; it does not by itself require this two-way check, which is why a reviewer still asks for it separately when the excluded fraction is large.

Common questions about sample size and exclusions

Is n = 5 or n = 6 mice per group enough for a tumour study?

It depends on the effect size and variance the study was designed to detect, not the number itself — five per group can detect a large, low-variance effect and is hopelessly underpowered for a small, noisy one. Report the calculation, not the count: assumed effect size, variance and its source, the test, the power target. A bare “n = 5” answers nothing.

Reviewer 2 says my sample size wasn’t justified, but my result was significant — what do I say?

Significance does not retroactively justify a sample size; an underpowered study that reaches significance anyway tends to overstate the true effect, since at low power only inflated estimates cross the threshold. The credible response is the calculation that should have run beforehand — effect size, variance, test, power — not the p-value obtained after. If none exists, say so and call the finding preliminary.

Why does my figure legend n need to match the number of dots on my graph?

Because every number an animal study reports is a claim about the same population, and a mismatch means one of them is wrong. The agent reconciles animals implanted in the methods, n in the design, n in each legend, and points visible in the plot; a paper stating 10 mice per group while plotting 8 points has told the reader three different things about one experiment.

Animal characteristics, and sex as a biological variable

Animal characteristics are strain, substrain, sex, age and weight at the start of the procedure, supplier, housing and cage density, diet, light cycle, and health or microbiological status. Each is reportable in one clause, and each has changed a published result somewhere: substrain drift between vendors, microbiome differences between facilities, and cage density effects on stress and tumour growth are all documented sources of between-laboratory disagreement.

Sex is the characteristic under explicit policy. NIH notice NOT-OD-15-102, “Consideration of Sex as a Biological Variable in NIH-funded Research”, applies to applications submitted for the January 25, 2016 deadline and afterwards, and expects sex to be incorporated into study design, analysis and reporting in vertebrate animal research, with strong justification from the literature or preliminary data where a single sex is proposed. A single-sex design remains entirely acceptable — it is the unexplained single-sex design that draws comment.

The agent reports whether a reason is given and whether the conclusions are limited to match. The failure mode it names most often is a discussion and abstract written about a disease, from an experiment run in male mice of one strain in one facility, with no sentence anywhere restricting the claim. That is a scope problem as much as a rigour problem, and it is checked again by the overclaim check and by model-to-human relevance.

A qualification the policy itself does not resolve: including both sexes is not the same as showing that the sexes responded differently. That second claim rests on a statistical test of the sex-by-treatment interaction, and it is routinely skipped. Garcia-Sifuentes and Maney (eLife 2021;10:e70817) examined 147 articles across nine biological disciplines and found that where a sex-specific effect of a manipulation was claimed, the authors usually had not tested statistically whether females and males responded differently — so sex-specific effects may be over-reported. They found the mirrored failure too: pooling the sexes without first testing whether a difference exists. An interaction test also asks more of the group sizes than the pooled main effect it sits inside, so a paper that adds a second sex should say which of the two questions its n was chosen to answer.

Outcome definition: tumour volume and survival endpoints

An outcome measure is defined when a second laboratory could produce the same number from the same animal. For tumour studies that requires the volume formula, the measurement instrument, the measurement frequency, and the endpoint at which measurement stops.

The volume formula matters because the common ones do not agree. The modified ellipsoid formula, volume = length × width² / 2, and the ellipsoid formula, volume = π/6 × length × width × height, return different numbers from the same tumour, so a volume reported without its formula cannot be compared across papers. State which was used, whether calipers or imaging produced the dimensions, and how often measurements were taken; measurement interval sets how much of a growth curve is interpolation.

One further point specialists raise and manuscripts rarely state: tumour growth is treated as approximately exponential, so the analysis that matches it works on the log scale and uses every measurement rather than one endpoint day. Hather and colleagues (Cancer Informatics 2014;13(Suppl 4):65–72) fit each animal’s growth by log-transforming volume and regressing it on time, and report that the resulting rate-based comparison uses all of the data where the conventional T/C ratio uses a single measurement, and needs fewer animals for the same power. Name the choice in the methods, because a difference in growth rate and a difference in mean volume on day 21 are different quantities and a reader cannot tell which one a bare “tumour volume was compared” refers to.

The endpoint needs the same treatment. Guidelines for the welfare and use of animals in cancer research (Workman et al., British Journal of Cancer 2010;102(11):1555–1577) state that “for an animal carrying a single tumour, the mean diameter should not normally exceed 1.2 cm in mice or 2.5 cm in rats, or 1.5 and 2.8 cm, respectively, for therapeutic studies”. A manuscript that reports tumour volumes implying diameters well beyond its stated welfare limit has a reporting inconsistency the agent flags, and a reviewer will ask about it.

Two analysis defects follow directly from endpoint handling. First, mean tumour volume plotted past the point where animals in the control arm have been removed for tumour burden is a mean over survivors, and the curves flatten because the largest tumours have left the dataset; the honest presentations are per-animal spaghetti plots, area under the curve to a common day, or a mixed-effects model of the full longitudinal data. Second, a survival analysis needs the event defined — death, or a humane endpoint, and which criteria triggered it — and censoring described, otherwise the Kaplan-Meier curve is a picture of a decision rule as much as of biology. Where every time point receives its own test, the multiple comparisons objection follows.

The agent brief

Paste this into a custom agent. Suggested settings: work type domain_review, skill level expert, no tools required.

Name: In vivo experimental rigour

You are reviewing an animal experiment for the design elements that determine
whether its causal claim is supportable. Read the methods, figure legends and
results together, and judge the design rather than only checking for mentions.

Determine and report:

1. Allocation. Determine how animals were assigned to groups. A statement that
   animals were randomised without a described mechanism should be reported as
   unverifiable. Where animals were allocated by cage, by arrival order, or to
   balance tumour size at treatment start, say which and note that this is
   stratification or systematic allocation rather than simple randomisation.

2. Blinding. Determine separately whether the person administering treatment,
   the person measuring the outcome, and the person analysing the data were
   blinded. Report each unaddressed. Blinding of outcome assessment matters
   most for subjective measures such as scoring, and for caliper measurement,
   where expectation influences the reading.

3. Sample size. Determine whether the group size was justified before the
   experiment, by a power calculation or a stated prior. Report group sizes
   presented without justification, and report any indication that the
   experiment was continued or repeated until a difference appeared.

4. Exclusions. Determine whether any animals were excluded, how many, from
   which groups, and on what criteria. Report exclusions described after the
   fact, exclusion criteria that are not stated, and any mismatch between the
   number of animals started and the number analysed. Reconcile the numbers
   across methods, figures and legends and report every discrepancy.

5. Animal characteristics. Report whether sex, age, strain and supplier are
   stated. Where only one sex was used, report whether a reason is given and
   whether the conclusions are appropriately limited.

6. Outcome definition. For tumour studies, determine whether the volume
   formula is given, whether measurement frequency is stated, and whether the
   endpoint is defined. For survival studies, determine whether the event is
   death or a humane endpoint, and whether censoring is described.

7. Regulatory statement. Report the absence of an ethical approval statement
   naming the approving body and protocol.

For each gap, state what it prevents a reader from concluding, not merely that
it is missing. Where a missing element undermines the central claim, say so
directly.

Editing the brief is expected. Adding a numbered item is the usual way to extend it — a dose justification item for translational work, or a housing and enrichment item for behavioural endpoints. How to write a custom reviewer agent covers the brief structure this one follows.

The regulatory statement

An ethical approval statement names the approving body, the protocol or licence identifier, and the legal framework under which the work was authorised. A sentence reading “all procedures were approved by the institutional animal care committee” names none of the three.

The identifiers that belong in the statement, by jurisdiction: in the United States, the Institutional Animal Care and Use Committee (IACUC) and its protocol number; in the United Kingdom, the Home Office project licence granted under the Animals (Scientific Procedures) Act 1986; in the European Union, the competent authority authorisation under Directive 2010/63/EU. The ARRIVE 2.0 Recommended Set treats the ethical statement as reportable in its own right, alongside protocol registration and data access. Planning-stage guidance is separate: PREPARE (Smith et al., Laboratory Animals 2018;52(2):135–141) covers what should be settled before an experiment begins, where ARRIVE covers what must be reported after it ends.

What this agent catches

How this differs from the built-in review

This in vivo rigour agent differs from PerfectPaper’s standing review team by naming the conclusion a gap breaks, not just the gap itself. The standing team already includes a reporting-guideline router that identifies which checklist applies and checks compliance against it, plus methodology and study design specialists; rather than recording that blinding is unreported, this agent states which specific conclusion becomes unsupportable — the form the objection actually takes in a real review.

PerfectPaper reconciles the animal counts across methods, figure legends and reported degrees of freedom, and reports the specific conclusion each unstated design element makes unsupportable, rather than recording that a checklist item is missing. That distinction decides whether a rigour comment is actionable: “randomisation not reported” produces a one-line rebuttal, while “allocation was made at treatment start by tumour size without a chance element, so baseline group differences cannot be excluded as an explanation for the day-21 separation” produces either an analysis or an honest limitations paragraph.

Pairs with pseudoreplication — that agent needs the design details this one checks are present. Full set: AI peer review for preclinical cancer research.

Review my manuscript

Related agents

The preclinical agents below share a review with this one; you can run three at a time. See AI peer review for preclinical cancer research for the full set of five preclinical agents.

• Unit of analysis and pseudoreplication — use it when a figure legend counts cells, tumours or wells rather than animals.
• Flow cytometry reporting — use it when a percentage has no named parent population.
• Image quantification integrity — use it when a representative image is standing in for a quantification.
• Model-to-human relevance — use it when an immune or microenvironment claim may exceed the host used.

Frequently asked questions

Do I need to use both male and female mice, or can I justify running just one sex?

A single-sex design is acceptable under NIH notice NOT-OD-15-102, which expects sex to be considered as a biological variable in vertebrate animal research from the January 25, 2016 due date onward — it requires justification, not both sexes. The unexplained single-sex design is what draws comment, especially when the abstract’s disease claim is written as if it applied beyond the one sex studied.

My journal wants an ARRIVE checklist — what does a reviewer actually check beyond the boxes being ticked?

A reviewer checks whether what the checklist item names is actually described, not whether the word appears. ARRIVE 2.0 (Percie du Sert et al., PLOS Biology 2020;18(7):e3000410) organises reporting into an Essential 10 plus a Recommended Set, and this agent judges the same way: “animals were randomised” satisfies a box; naming the mechanism and the day it happened is what the ARRIVE checklist is actually asking for.

How is an AI review of my animal study different from just running it through a checklist?

A checklist records whether an item is mentioned; this agent states what each gap prevents a reader from concluding, which decides whether a comment is actionable. Macleod and colleagues (PLOS Biology 2015;13(10):e1002273) found blinded outcome assessment in 3% and a sample size calculation in none of 146 in vivo publications sampled — the items are usually absent outright, which is why the consequence matters more than the checklist restated.