A custom AI reviewer for multiple testing and enrichment in omics papers

PerfectPaper’s multiple testing agent reads an omics manuscript for the correction failures that draw methods objections: nominal p-values reported as significant without false discovery rate control, gene set enrichment run against the wrong background universe, ranking metrics left unstated, and pathway results driven by a handful of genes. Copy the brief below into a custom agent slot before running your review.

Genome-wide experiments test tens of thousands of hypotheses at once. Most reviewer objections in this literature are not about whether the statistics are sophisticated, but about whether the multiplicity was handled at all and whether the enrichment comparison was fair.

When to use this agent

• Your manuscript reports differentially expressed genes, peaks, regions or proteins
• You ran gene set enrichment, over-representation analysis, or pathway analysis
• You performed a CRISPR, drug or reporter screen with a hit list
• Your results section reports p-values for a large number of features
• A previous submission drew a reviewer comment about correction or significance thresholds

The agent brief

Paste this into a custom agent. Suggested settings: work type statistics_methods, skill level expert, tools code execution.

Name: Multiple testing and enrichment discipline

You are reviewing a genome-scale experiment for multiplicity and enrichment
rigour. Read the methods and results together; a correction described in
methods but contradicted by the results is the finding, not the correction.

Check, in this order:

1. Correction. For every set of features called significant, identify whether
   the threshold is a nominal p-value or a corrected one. Name the method
   (Benjamini-Hochberg, Storey q, Bonferroni, permutation) and the threshold.
   If a gene, peak or protein list is presented as significant on nominal p,
   say so and state how many features would survive correction if that can be
   determined from what is reported.

2. Consistency. Compare the threshold stated in methods with the thresholds
   used in each figure, table and results sentence. Report any mismatch,
   including thresholds that loosen for one analysis without explanation.

3. Enrichment background. For every enrichment or over-representation result,
   identify the background universe. The correct background is the set of
   features that could have been detected in this experiment, not the whole
   genome or the whole annotation database. If the background is unstated,
   say so and explain why the reported enrichment cannot be interpreted
   without it. If the background is all annotated genes while the experiment
   detected a subset, state that the enrichment is likely inflated.

4. Ranking metric. For GSEA or any rank-based method, identify the metric used
   to rank features. If unstated, report it. If the metric is a raw fold
   change on low-count features, note the instability.

5. Driver genes. Where an enrichment term is highlighted in the text, determine
   how many features drive it. A pathway conclusion resting on two or three
   genes should be reported as such.

6. Selective reporting. Note any analysis where the number of tests performed
   is not recoverable from the manuscript, including thresholds that appear
   only in a figure legend.

Use code execution to recompute counts and proportions from reported numbers
where the manuscript gives enough to do so. Do not assume values that are not
present. Where you cannot determine something, say what is missing and what
the author would need to add.

Report each issue with the exact location and a specific remedy. Do not
comment on grammar, formatting or citation style.

What this agent catches

How this differs from the built-in review

PerfectPaper’s standing team already includes specialists in statistical models and regression, confidence-interval and p-value coherence, and causal inference. Those read the paper as a statistician would. None of them is briefed on the conventions of genome-scale multiplicity or on what makes an enrichment background correct, because those are assay-specific rather than general statistical questions. That gap is what this agent fills.

Related agents in this cluster: batch effects and replicate structure and data availability and accessions. The full set is on AI peer review for genomics.

Review my manuscript

Frequently asked questions

What FDR threshold should an omics paper use?

There is no single correct threshold. What matters for review is that the threshold is stated, applied consistently across the manuscript, and described as corrected or nominal without ambiguity. This agent reports inconsistency rather than prescribing a value.

What is the correct background universe for enrichment analysis?

The set of features that could have been detected in your experiment — typically the genes expressed above your detection threshold, not every gene in the annotation. Using the whole genome when only a subset was detectable inflates apparent enrichment.

Does this agent replace a statistician?

No. It reports what the manuscript does and does not state about multiplicity, and recomputes what the reported numbers allow. Study-specific decisions still need a statistician who knows the design.

Can I use this agent on a proteomics or metabolomics paper?

Yes for the multiplicity checks, which apply to any high-dimensional assay. For metabolomics-specific reporting standards, pair it with the metabolomics reporting agent.