A custom AI reviewer for batch effects and replicate structure

PerfectPaper’s batch effects agent reads a sequencing or array manuscript for the design failures that cannot be fixed after the fact: batches that align with the experimental condition, batch never entered into the model, technical replicates counted as biological, and replicate numbers too small to support the claim being made. Copy the brief below into a custom agent slot.

A confounded batch is the one problem in this list that no reanalysis can rescue. It is worth finding before a reviewer does, because the honest remedy is usually a limitation paragraph rather than a new figure.

When to use this agent

Add this agent whenever the experiment crosses more than one technical circumstance — a run, a day, a site, a reagent lot — because that is the condition under which a batch effect becomes possible in the first place. Reach for it especially when:

• Samples were collected, prepared or sequenced across more than one run, day or site
• Cases and controls were processed at different times
• Your replicates are cell culture passages, technical repeats, or repeated measures
• Your differential result rests on three or fewer samples per group
• A reviewer has previously asked what n refers to
• Your manuscript reports a comparison between two sample sets that arrived from different collaborators, biobanks or cohorts
• Sequencing was topped up, repeated or added to after an interim analysis

What counts as a batch in a sequencing experiment

A batch is any grouping of samples that shared a technical circumstance capable of shifting measured signal, and in a sequencing workflow there are usually six or more such groupings stacked on top of each other. Leek and colleagues, in “Tackling the widespread and critical impact of batch effects in high-throughput data” (Nature Reviews Genetics 2010;11:733–739), examined nine public datasets and found that between 32.1% and 99.5% of measured features showed a statistically significant association with processing date alone. Processing date is one batch variable among many, and it is rarely the one authors report.

The batch variables the agent enumerates are the ones that leave a signature in high-throughput data:

The last row matters more than most authors expect. A dataset reprocessed with a newer reference build partway through acquisition carries a batch variable created entirely in software, and it is invisible in every wet-lab record. See what is a batch effect for the general mechanism, and confounding for why a technical variable that tracks the condition behaves exactly like a biological one.

Common questions about spotting a batch effect

Can an AI reviewer actually detect a batch effect in my RNA-seq manuscript?

An AI reviewer detects the design conditions that produce batch effects — sequencing dates that align with sample groups, batch variables absent from the model, replicate counts whose type is unstated — by reading the methods, the sample table and the supplementary design description together. It cannot detect a batch effect in raw counts it was never given; the diagnostic plot stays the author’s job.

Can an AI reviewer check my design for batch confounding before I submit?

Yes: cross-tabulating condition against each batch variable — collection date, extraction day, plate, run, operator — takes minutes and answers the question directly, since any empty cell in that table means full confounding. An AI reviewer reads the sample table for the same cross-tabulation, and separately checks whether a principal-component plot broken out by batch appears anywhere in the manuscript.

What does an AI reviewer report when the methods do not describe batches at all?

An AI reviewer reports that the design cannot be assessed and names the exact missing detail, rather than assuming the design was sound. An unstated allocation across runs, plates or sites is itself a finding: no reviewer, human or AI, can evaluate confounding from a methods section that never says how samples were grouped in the first place.

Confounded, partially confounded, and balanced designs

Batch confounding is a property of a cross-tabulation, not a property of the data, and the cross-tabulation tells you at a glance which of three situations you are in. Build a contingency table of condition against batch. What you see decides what the manuscript can claim.

Partial confounding is a matter of degree, not a separate category from balance: the more the batch-by-condition cross-tabulation departs from even cells, the more collinear the two terms are in the model, and the wider the standard error on the condition estimate grows. An 8-versus-2 split in each direction is recoverable but costly; a 6-versus-4 split barely differs from a balanced design. The agent does not score this on a single scale, because the same cross-tabulation can be read two ways: as reduced power in a well-specified model, or — where a correction method estimates unwanted variation from the data rather than from a stated design term — as a design where the correction can absorb part of the true condition effect along with the batch effect it was meant to remove.

Full confounding is a rank deficiency, not a statistical inconvenience. When batch has the same levels as condition, the design matrix has two identical columns, and the software drops one of them or refuses to fit — a the model matrix is not full rank error in DESeq2 is the design telling you the truth in the only language it has. Authors who see it and respond by removing batch from the model have not solved the problem; they have deleted the evidence of it.

Auer and Doerge set out the remedy at the design stage in “Statistical design and analysis of RNA sequencing data” (Genetics 2010;185:405–416): bar-code every sample, then split equal portions of each sample across lanes so that “balanced blocks” are formed, which eliminates confounding by lane or batch rather than modelling it away afterwards. The design is decided before any sequencing happens, which is why the agent reads for it rather than for a correction step.

Questions about confounded designs

Is there any way to fix a fully confounded batch design after the fact?

No computational method separates a condition effect from a batch effect once the two share exactly the same grouping — the design matrix has two identical columns, and no amount of modelling recovers information that was never collected. The two remedies the agent points to are a narrowed claim that names the confound in the same sentence, and repeating part of the experiment with mixed batches.

My design is only partially confounded — batch 1 has more cases, batch 2 has more controls. Is that going to sink the paper?

Not on its own; a moderate imbalance costs precision, not validity. The condition effect is still estimable, the standard error is wider than a balanced design would give, and the risk grows only as the split approaches an empty cell. Report the cross-tabulation in the methods so a reviewer can see how far from balanced the design actually is, rather than leaving them to infer it.

The agent brief

Paste this into a custom agent. Suggested settings: work type statistics_methods, skill level expert, no tools required.

Name: Batch effects and replicate structure

You are reviewing a high-throughput experiment for confounding by batch and
for the validity of its replicate structure. Read the methods, the sample
table and any supplementary design description together.

Determine and report:

1. Batch structure. Identify every source of batch in the design: collection
   date, library preparation day, sequencing run, flow cell, plate, reagent
   lot, operator, site. If the manuscript does not describe how samples were
   grouped across these, report that the design cannot be assessed and say
   exactly which detail is missing.

2. Confounding. Determine whether any batch variable aligns with the
   experimental condition. If all cases were processed in one batch and all
   controls in another, the condition effect and the batch effect cannot be
   separated. State this plainly when it is true, and state that no
   computational correction resolves it.

3. Modelling. Determine whether batch was included as a covariate, used in an
   explicit correction step, or ignored. If a correction method is named,
   check whether the same variable is also the variable of interest, which
   would remove the effect being measured.

4. Replicate type. For every reported n, determine whether the units are
   biological replicates, technical replicates, or repeated measures of the
   same unit. Cell culture passages from one parental line, multiple libraries
   from one RNA extraction, and multiple lanes of one library are not
   biological replicates. Report every n where the type cannot be determined
   from the text.

5. Replicate number. Note where a differential or comparative claim rests on
   fewer than three biological replicates per group, and describe what the
   claim can and cannot support at that n.

6. Diagnostics. Determine whether the manuscript shows any assessment of batch
   structure, such as a clustering or dimension-reduction plot annotated by
   batch. Report its absence as a gap rather than as an error.

Do not recommend a specific correction method. Report the design as it stands,
what it can support, and what the author should state explicitly.

The brief deliberately ends by forbidding a method recommendation. A reviewer who names ComBat, sva or RUVSeq has moved from reviewing the design to redesigning the analysis, and the author is left with an instruction rather than a finding. How to write a custom reviewer agent covers why that boundary belongs in the brief itself.

What this agent catches

Seven design failures make up the agent’s checklist, and each one is paired below with why it survives every downstream fix once the samples are already collected.

Worked example: a confound the figures do not reveal

The published case is the Mouse ENCODE comparative expression reanalysis, and it is worth reading because the original conclusion was large and the defect was invisible in the figures. Gilad and Mizrahi-Man, in “A reanalysis of mouse ENCODE comparative gene expression data” (F1000Research 2015;4:121), showed that the assignment of samples to sequencing flowcells and lanes was nearly completely confounded with the species annotations. After accounting for the batch effect with ComBat, the data no longer clustered by species and instead showed a clear tendency to cluster by tissue — the opposite of the headline finding, from the same reads.

Now the ordinary version. Consider a study of 12 tumour and 12 adjacent-normal samples in which the tumours were accrued first and sequenced in March, and the normals were accrued from a second consenting wave and sequenced in June. The volcano plot is clean. The dispersion estimates are unremarkable. The principal component plot separates the two groups sharply along PC1, which the authors read as strong biology. Nothing in any of those displays distinguishes a March-versus-June effect from a tumour-versus-normal effect, because in this dataset they are the same variable.

The tell is not in the analysis at all. It is in a sentence in the methods that gives two sequencing dates, and in a supplementary table that lists run identifiers the results never mention. The agent reads those two artefacts against each other. A reader who finds the confound afterwards will discount every differential gene in the paper, not just the marginal ones, which is why this belongs in a pre-submission pass alongside multiple testing correction.

Biological, technical and repeated measures: what n refers to

An n is only a biological replicate when the units were independently subject to the condition, and manuscripts get this wrong more often than they get it right. Lazic, Clarke-Williams and Munafò surveyed 200 published animal studies from 2011–2016 in which the treatment was applied to parent animals and the outcome measured in their offspring — the design where the experimental unit is easiest to misidentify — and reported in “What exactly is ‘N’ in cell culture and animal experiments?” (PLOS Biology 2018;16(4):e2005282) that 22% (95% CI 17%–29%) used the correct experimental unit, 46% (95% CI 38%–53%) treated pseudoreplicates as genuine replicates, and 32% (95% CI 26%–39%) gave too little information to tell.

The distinctions the agent enforces are concrete:

• Biological replicate. A separate animal, donor, subject or independently derived culture that received the condition on its own.
• Technical replicate. Two libraries from one RNA extraction, two lanes of one library, two wells of one lysate, two injections of one sample. These estimate measurement noise, not biological variability.
• Repeated measures. Several observations of the same unit over time or across fields of view. These are not independent, and averaging them to one value per unit is often the correct handling.
• Passages of one line. Three passages of a single parental line are three measurements of one genome. The claim generalises to that line, not to the cell type.

The distinction decides the denominator of every test in the paper. Where a manuscript nests cells inside dishes inside experiments, the correct analysis is a mixed model with the experiment as a random effect, not a t-test across wells. See what is pseudoreplication, the dedicated pseudoreplication agent, and the reviewer comment it produces at my n is not independent.

How many biological replicates a claim needs

Six per condition is the published minimum for RNA-seq, and twelve is the number for a comprehensive differential expression claim. Schurch and colleagues generated 48 biological replicates per condition in Saccharomyces cerevisiae, comparing wild type against a Δsnf2 mutant, and reported in “How many biological replicates are needed in an RNA-seq experiment and which differential expression tool should you use?” (RNA 2016;22:839–851) that at least six replicates per condition should be used for all experiments, and at least twelve per condition where identifying the majority of differentially expressed genes matters.

That result comes from a single-celled organism under controlled conditions, so it is a floor rather than a ceiling for mammalian tissue. The agent does not declare a number acceptable or unacceptable. It reports the n, its type, and what a claim at that n supports: n=3 supports a statement that a large effect was observed in three independent units, and does not support a statement about small effects, interactions, subgroup differences, or generalisation across genetic backgrounds.

There is a mechanical reason the n=3 ceiling is lower than intuition suggests. DESeq2 and edgeR do not estimate each gene’s dispersion from its own three data points alone — three points cannot support a stable variance estimate — so both shrink the gene-wise estimate toward a trend fitted across all genes at a similar expression level, and that shrinkage is strongest exactly when replicates are fewest, weakening as sample size grows. At n=3 the trend, rather than the gene’s own replicate spread, does most of the work of calling significance.

That has an asymmetry worth knowing. DESeq2 guards one direction: a gene whose own dispersion sits far above the trend is treated as a dispersion outlier and keeps its unshrunk estimate, so a genuinely noisy-looking gene is not quietly made significant. Nothing guards the other direction. A gene that is genuinely variable but whose three replicates happen to agree is pulled up only as far as the trend, and that gene — not the visibly erratic one — is the characteristic low-replicate false positive.

The framing matters because the objection that lands in review is rarely “your n is too small” in the abstract. It is “your conclusion is broader than your design”, which is a claim problem rather than a sample-size problem. Reviewer says the sample size is too small covers the response, which almost never turns on a post-hoc power calculation.

Common questions about replicate numbers

Is n=3 enough for my RNA-seq experiment?

Three biological replicates per condition support a statement that a large effect was observed in three independent units; they do not support claims about small effects, interactions, subgroup differences, or generalisation across genetic backgrounds. Schurch and colleagues (RNA 2016;22:839–851) recommend at least six biological replicates per condition, and at least twelve where identifying the majority of differentially expressed genes matters.

I have three passages of the same cell line — can I call that n=3?

No — three passages of one parental line are three measurements of one genome, not three biological replicates, because none of them was independently subject to the condition. The claim generalises to that specific cell line, not to the cell type or to biology in general, however the passages were labelled in the methods.

Can an AI reviewer tell whether my replicates are biological or technical?

Wherever the methods state what each unit was, yes. A biological replicate is an independent unit subject to the condition — a separate animal, a separate donor, an independently derived culture. Multiple libraries from one RNA sample, or multiple lanes of one library, are technical replicates, and the agent reports every n whose type the text leaves undetermined rather than guessing.

Reviewer 2 says my sample size is too small — what do I say back?

Answer the actual objection, which is usually “your conclusion is broader than your design” rather than a bare power complaint. State plainly what the n supports — a large, reproducible effect in that many independent units — and narrow any claim about small effects, subgroups or generalisation to match. Reviewer says the sample size is too small covers the full response; it rarely turns on a post-hoc power calculation.

What batch correction can and cannot do

Batch correction adjusts for batch when the design is balanced, and manufactures confidence when it is not. Nygaard, Rødland and Hovig demonstrated the mechanism in “Methods that remove batch effects while retaining group differences may lead to exaggerated confidence in downstream analyses” (Biostatistics 2016;17:29–39): when batch-adjusted values are carried into a second analysis that no longer accounts for batch, those values are no longer independent, variance is systematically underestimated, and the inflation depends on the imbalance rather than on the sample size. In their reanalysis of one published dataset, ComBat-adjusted values yielded 2,011 significant genes where blocking on batch in the model yielded 11.

The practical rules the agent reads for:

• Include batch as a covariate in the model. ~ batch + condition in DESeq2, edgeR or limma keeps the batch adjustment and the significance test in one estimation, which is what preserves the standard errors.
• Do not test corrected values. The limma documentation for removeBatchEffect states it directly: “This function is not intended to be used prior to linear modelling. For linear modelling, it is better to include the batch factors in the linear model.” Corrected matrices are for plots and heatmaps.
• Check what a surrogate-variable method absorbed. sva and RUVSeq estimate unwanted variation from the data. When the unwanted variation tracks the condition, they will remove the effect being measured, and the paper reports a null that the design created.
• Never correct a fully confounded design. ComBat applied where batch equals condition returns a matrix in which the biological difference has been defined as noise or preserved by fiat, depending on how the model was specified.

ComBat’s own assumption is worth stating rather than leaving implicit: its empirical Bayes step pools information across genes on the premise that batch shifts location and scale in a way that is similar across genes within a batch. Small batches are not what that premise struggles with — small batches are the case the method was built for. Johnson, Li and Rabinovic introduced it in “Adjusting batch effects in microarray expression data using empirical Bayes methods” (Biostatistics 2007;8:118–127) precisely because the methods available then required batch sizes above 25, which most real studies never reach.

What breaks ComBat is composition, not size. A batch holding only one level of the condition offers no internal contrast to separate its batch shift from the condition effect, so whatever biology those samples carry is absorbed into the batch estimate. That is the defect in the reanalysis above rather than an incidental feature of it: Nygaard and colleagues describe the design they re-examined as “highly unbalanced, with several batches having only one of the main treatments of interest.” Count the condition levels inside each batch before trusting any correction, and treat a batch that contains only cases or only controls as the same problem as a fully confounded design, scaled down.

Questions about batch correction

Should I add batch as a covariate, or run ComBat first and use the corrected values?

Add batch as a covariate in the same model — ~ batch + condition in DESeq2, edgeR or limma — rather than correcting the values first and testing them afterward. The limma documentation for removeBatchEffect says this directly: the function is not intended for use before linear modelling, because testing already-corrected values understates the variance and inflates the apparent significance.

Why did I get thousands of significant genes after running ComBat?

Check first whether the corrected values were then tested as if they were raw. That is a known statistical failure mode rather than a coding error: batch-adjusted values are no longer independent, testing them understates the variance, and the significant-gene count inflates accordingly. In the reanalysis by Nygaard and colleagues, testing ComBat-adjusted values yielded 2,011 differentially expressed genes at 5% FDR where blocking on batch inside the model yielded 11 — same data, same comparison, only the estimation step differed.

Why did my differential expression results completely flip after I ran ComBat?

A flip after correction usually means condition and batch were confounded going in, rather than that the correction misfired. In the Mouse ENCODE reanalysis, sample assignment to sequencing lanes was nearly perfectly confounded with species: before correction the data clustered by species, and after ComBat it clustered by tissue instead — the same reads, revealing the comparison the confound had been hiding rather than producing a new artefact.

How to check your own design before submitting

Checking for batch confounding takes one contingency table and two plots, and all three can be produced before a manuscript is written. Run them in this order.

Cross-tabulate condition against every batch variable. A single table(condition, batch) per variable answers the question completely. Any zero cell in a two-by-two is full confounding. Do this for collection date, extraction day, plate, run and operator separately; partial balance on one does not imply balance on another.

Plot the first principal components and colour by batch, not by condition. Colouring by condition shows you what you hope to see. Colouring by batch shows you what a reviewer will see. Where PC1 separates by run date, report that in the manuscript rather than waiting for it to be discovered.

Regress each principal component on each batch variable. The proportion of variance in PC1 and PC2 explained by run, plate and date is a number you can state in a methods sentence, and a stated number pre-empts the objection that no diagnostic was shown.

Reconcile sample counts across every table. The number of samples in the abstract, the sample table, the figure legends and the model input frequently differ. Samples dropped after quality control form a batch of their own if the reason for dropping relates to the condition.

Check whether the batch variable exists in your records at all. Where plate maps, run identifiers and extraction dates were never recorded, say so, because an unrecorded batch variable cannot be modelled and the limitation is real. Where the replicates themselves point in different directions, my replicates disagree is the diagnostic to run first.

Single-cell and spatial data: the same structure, less visible

Single-cell experiments carry the same batch structure as bulk experiments and hide it better, because the cell count creates an appearance of large n that the design does not support. Hicks, Townes, Teng and Irizarry examined 15 publicly available scRNA-seq datasets in “Missing data and technical variability in single-cell RNA-sequencing experiments” (Biostatistics 2018;19:562–578) and found the proportion of genes detected per cell ranged from under 1% to 65%. Across the eight studies that compared predefined biological groups, the standardised Pearson contingency coefficient between sequencing instrument run and biological condition ranged from 82.1% to 100%, meaning perfect confounding in at least one. Their conclusion is the one that matters for review: at that level of confounding, it is difficult to impossible to separate technical from biological variation.

The specific traps in single-cell work are the capture day, the 10x chip channel, the hashing or multiplexing scheme, and the sample-to-donor mapping. Two conditions loaded on separate chips on separate days constitute the same defect as two sequencing runs in a bulk experiment, and integration methods absorb it silently by design.

That silent absorption has a documented failure mode. Tran and colleagues benchmarked 14 correction methods in “A benchmark of batch-effect correction methods for single-cell RNA sequencing data” (Genome Biology 2020;21:12) and note that most “require at least one identical cell type to be shared between any pair of data batches, to guide the data alignment”; where sub-populations are not shared across batches, incorrect alignment can follow, “especially if there are cell types that are highly similar.” In their non-identical-cell-type scenario, several widely used methods merged distinct populations into a single well-mixed cluster. That is the exposure whenever the biological condition changes what is present in the sample — a tumour biopsy against adjacent normal tissue, where the honest difference is which cell types exist at all, not a shift within cell types that are shared. The question worth asking before trusting an integrated embedding is not whether the batches mixed well on the plot, but whether the cell type you care about was present in comparable numbers in every batch before integration ran.

The replicate question compounds it. Thousands of cells from one donor is n=1 for any claim about donors, and a p-value computed across cells will shrink toward zero as more cells are captured regardless of whether the biological effect exists — the mechanism set out in my p-value shrinks with more cells. Pseudobulk aggregation to one profile per donor restores the correct unit.

What reviewers write when batch is mishandled

Reviewers rarely use the phrase “batch effect” in their first sentence. They write the specific version, and these are the comments that decide the paper.

“The methods report two sequencing dates but the analysis does not include run as a covariate; please state how samples were allocated across runs.” “All tumour samples appear to have been processed in one library preparation batch and all normals in another; the reported differential expression cannot be attributed to disease status.” “Please provide a principal component plot annotated by processing batch.” “n=3 refers to three passages of a single cell line; the conclusions are stated at the level of the cell type.” “The number of biological replicates is not recoverable from the methods; please state what the experimental unit was for each comparison.” “Batch-corrected expression values appear to have been used as input to the differential test; please refit including batch as a covariate.” “Sixteen samples were excluded after quality control without stated reasons; please report the exclusions by group.” “The authors state that ComBat was applied but do not state whether the model retained the group variable.”

The last one is the comment authors find hardest to answer after the fact, because the answer depends on a parameter choice made months earlier in a script no longer in front of them. Drafting the reply is covered in how to write a response to reviewers; the answer that works is the script and the model formula, not a paragraph of reassurance.

When the obvious remedy is unavailable

Re-running a confounded experiment with mixed batches is frequently impossible, and the honest alternative is a narrowed claim with a stated direction, not a correction step. When the samples are exhausted, the cohort is closed, the consent has lapsed, or the collaborator who ran the first batch has left, the manuscript still has four moves available.

Narrow the claim to what the design supports. A confounded comparison can be reported as an observation with the confound named in the same sentence, and that is a publishable form. It is not a differential expression result.

Report the cross-tabulation. Publishing the condition-by-batch table converts a hidden defect into a disclosed limitation. Reviewers respond very differently to a design they were shown than to one they discovered.

Validate on an orthogonal platform with independent samples. Confirming a subset of results by qPCR, targeted proteomics, immunostaining or a second cohort processed in one batch breaks the confound for those specific findings, which is often the whole substantive claim. See orthogonal validation and, for animal work, in vivo rigour.

Write the limitation with a direction. State which way the batch effect would push the estimate and why, rather than writing that results should be interpreted with caution. A limitation that names the mechanism and the likely sign reads as an analysis; a generic acknowledgement reads as an admission. How to write a limitations section sets out the form, and reviewer says the experiment was not replicated covers the adjacent objection.

How to report the design so a reviewer can check it

Report the design as a sample-level table with one row per sample and one column per batch variable, deposited with the data. That single artefact answers most of the questions above without a reviewer having to ask, and it is the format the agent expects to find.

The columns that matter are sample identifier, condition, collection date, extraction batch, library preparation plate and well, index sequence, flow cell and lane, instrument run, operator, and the biological unit the sample belongs to. The last column is the one that resolves the replicate question outright: two rows sharing a biological unit identifier are technical replicates, however they were sequenced.

MINSEQE, the minimum information standard for high-throughput sequencing experiments, asks for the experimental design including sample relationships alongside the raw reads and processed data. Repository submissions accept this metadata, and the accession belongs in the manuscript rather than in a data-availability sentence with no identifier — the gap covered by the data availability and accessions agent.

In the manuscript itself, one methods sentence carries most of the value: state how samples were allocated across batches, state whether the allocation was randomised or blocked, and state which batch variables entered the model. Three clauses close the most common objection in genomics review.

How this differs from the built-in review

The batch effects agent covers ground the standing review team does not brief for by default. The team’s cohort accounting specialist reconciles sample numbers across the manuscript, and the statistical models specialist reads the analysis, but neither is briefed on what constitutes a batch in a sequencing workflow or on the distinction between a passage and a biological replicate — distinctions that are obvious to a working molecular biologist and invisible to a general statistical reader.

Pairs naturally with multiple testing and enrichment, since an uncorrected batch effect and an uncorrected p-value inflate the same result. Full set: AI peer review for genomics. For animal experiments where the batch is a cage, a litter or a housing room rather than a flow cell, run it alongside AI peer review for preclinical work.

Review my manuscript

Related agents

Four sibling agents share a review with this one; you can run three at a time. See AI peer review for genomics for the full set of five omics agents.

• Multiple testing and enrichment — use it when a gene list is called significant on nominal p, or enrichment runs against the wrong background.
• Data availability and accessions — use it before submission, since a missing accession is what gets a paper returned without review.
• Metabolomics and flux reporting — use it for identification confidence, normalisation, and natural-abundance correction.
• Variant and gene nomenclature — use it when the paper reports specific variants that will be curated downstream.