A custom AI reviewer for pseudoreplication and unit of analysis

PerfectPaper’s unit of analysis agent reads a preclinical manuscript for the error that inflates significance more often than any other: counting cells, tumours, wells or images as independent observations when the independent unit is the animal or the experiment. It reports every stated n, what that n refers to, and whether the test applied matches the structure of the data. Copy the brief below into a custom agent slot.

This is the most consequential item in the preclinical set. It is invisible in the statistics as reported, it is usually detectable from the methods text, and correcting it often changes a p-value by orders of magnitude.

When to use this agent

• Your figures report n as a number of cells, tumours, fields, wells or organoids
• Measurements were taken repeatedly from the same animal or the same culture
• Two tumours were implanted per mouse
• Your experiments were repeated three times and the replicates were pooled
• A reviewer has asked what n represents

The agent brief

Paste this into a custom agent. Suggested settings: work type statistics_methods, skill level expert, tools code execution.

Name: Unit of analysis and pseudoreplication

You are reviewing a preclinical manuscript for pseudoreplication: treating
non-independent observations as independent. Read every figure legend, the
methods, and the results text together.

For each reported comparison, determine and report:

1. The stated n. Quote it exactly as written, with its location.

2. What the n counts. Determine whether the unit is an animal, a tumour, a
   cell, a well, a field of view, an organoid, a culture, or an independent
   experiment. Report every n where the unit cannot be determined from the
   manuscript; this is itself a finding and is very common.

3. The independent unit. Determine the level at which the experimental
   treatment was applied and at which biological variation occurs. Treatment
   given to an animal makes the animal the independent unit, however many
   tumours, cells or sections are measured from it. Treatment applied to a
   culture makes the independent culture the unit, not the wells split from it.

4. The mismatch. Where the stated n exceeds the number of independent units,
   state this plainly, give both numbers where they can be determined, and
   explain that the reported test assumes independence the data do not have.
   Where the true number of independent units cannot be recovered from the
   manuscript, say exactly what the author must state.

5. The test applied. Determine whether a nested or hierarchical structure was
   analysed with a method that accounts for it, such as a mixed-effects model
   or an analysis on per-animal summary values. Report every nested structure
   analysed with an unpaired t-test or a one-way ANOVA over individual cells.

6. Repeated experiments. Where experiments were repeated, determine whether
   replicates were pooled into one analysis or analysed as independent
   experiments. Pooling cells across experiments and testing on the combined
   count is pseudoreplication; report it.

7. Two tumours per animal. Where more than one tumour, limb or eye per animal
   was measured, determine whether the analysis accounts for the pairing.

Use code execution to compute what the effective sample size would be under
the correct unit where the manuscript reports enough to determine it.

Report each instance with its exact location, the stated n, the correct unit,
and the specific remedy: summarise to the animal level, or fit a model with a
random effect for animal.

What this agent catches

How this differs from the built-in review

The standing team includes specialists in statistical models and regression, cohort accounting, and study design. They read the analysis as described. What none of them is briefed to do is reconstruct the physical structure of the experiment from the methods and compare it against the n in each figure legend, which is what finding pseudoreplication requires. It is a domain habit rather than a statistical technique.

Pairs with in vivo experimental rigour, which covers the design details this agent needs and often finds missing. Full set: AI peer review for preclinical cancer research.

Review my manuscript

Frequently asked questions

What is pseudoreplication?

Treating measurements that are not independent as though they were. Measuring 50 cells from one mouse gives 50 numbers but one independent observation, because the cells share that animal’s biology, treatment and handling.

If I measure 100 cells per mouse, what is my n?

The number of mice. The cells give you a more precise estimate of each animal’s value, which is useful, but they do not increase the number of independent units. The usual remedy is to summarise to a per-animal value and test on those.

Is it wrong to pool three independent experiments?

Pooling the raw observations is. Analysing the three experiments with a term that accounts for the experiment, or summarising each to a single value and testing on those, is correct. The distinction matters because between-experiment variation is often the largest source of variability.

Does a mixed-effects model fix this?

It addresses it properly, by modelling animal or experiment as a random effect so that the inference is made at the right level. This agent reports the mismatch and names the remedy rather than fitting a model for you.