False discovery rate or family-wise error rate?

The false discovery rate and the family-wise error rate control different quantities, so the choice follows from what you will claim. Family-wise error rate bounds the probability of even one false positive across the family. False discovery rate bounds the expected proportion of false positives among the hypotheses you reject. Individual claims need FWER; set-level claims need FDR.

That rule is easy to state and routinely misapplied, because most manuscripts pick a procedure — Bonferroni, Benjamini–Hochberg — before deciding what the reported findings are for. This page covers the two formal definitions, the claim-level test that decides between them, a worked example at 20,000 tests, the procedures available under each, what dependence does to both, why the word “family” is contested, the conventions in genomics and in confirmatory trials, and what reviewers write when the choice is wrong.

What each error rate actually controls

Family-wise error rate is the probability of making at least one Type I error across a specified family of hypotheses. Controlling the family-wise error rate at 0.05 means that if every null hypothesis in the family were true, the chance of rejecting any of them is at most 5%. The quantity is a probability, and it does not scale with the number of rejections: one false positive counts the same as forty.

False discovery rate is the expected value of the proportion of rejected hypotheses that are true nulls. Benjamini and Hochberg defined it in 1995 (Journal of the Royal Statistical Society B) as E[V/R], with the ratio set to zero when R = 0, where V is the number of false rejections and R the total number of rejections. Controlling the false discovery rate at 0.05 means that averaged over repetitions of the experiment, no more than 5% of the findings you report should be null.

Two consequences follow that are easy to miss. First, the false discovery rate is an expectation over hypothetical repetitions, not a guarantee about the list in front of you: the realised false discovery proportion in a single study can exceed the nominal level substantially, particularly when R is small. Second, when every null in the family is true, V/R is either 0 or 1, so E[V/R] equals P(V ≥ 1), and false discovery rate control coincides exactly with family-wise error rate control. Benjamini–Hochberg is not a permissive procedure under the complete null: there it carries the same family-wise guarantee as Bonferroni, bounding the probability of any false positive at 5%. It still rejects whenever Bonferroni would, and sometimes more.

Common questions about FDR and FWER definitions

What is the difference between the false discovery rate and the family-wise error rate?

Family-wise error rate is the probability of at least one false positive anywhere in the family. False discovery rate is the expected proportion of false positives among the hypotheses actually rejected. One is a probability of any error occurring; the other is an average contamination level of the findings list. Under the complete null they coincide.

What does a q-value of 0.05 actually mean?

A q-value of 0.05 for a given hypothesis means that among all hypotheses in the list with a q-value at or below that gene’s, roughly 5% are expected to be false positives on average across repeated experiments. It is not the probability that this particular hypothesis is null — that number is the local false discovery rate, a different quantity entirely.

The question that decides it: what will you claim?

The choice between the false discovery rate and the family-wise error rate is settled by the grammatical subject of your conclusions, not by the number of tests. Ask what sentence the reader will carry away from the abstract.

If the sentence names one hypothesis — “the drug reduced the primary endpoint”, “this variant is associated with the trait” — the claim is per-hypothesis, and it needs family-wise error rate control. Each named finding has to stand on its own, and an error rate that tolerates 5% of a list being wrong tells the reader nothing about whether this particular one is among the wrong ones.

If the sentence names a set — “these 312 transcripts were differentially expressed and were carried into pathway enrichment” — the claim is set-level, and the false discovery rate is the right currency. The reader is told, correctly, that roughly 5% of the 312 are expected to be null, and no argument in the paper depends on any individual member being real.

The most frequent error in applied work is mixing the two: controlling the false discovery rate across a genome-wide screen, then building the discussion, the headline figure and the follow-up experiment on the single gene at the top of the ranked list. A q-value of 0.03 is a property of the list, not a posterior probability for that gene. If one member will be named and defended, that member needs protection of its own — a within-family family-wise correction, an orthogonal assay, or independent replication. Reviewers reading multiple-comparisons objections raise this specific mismatch more often than they raise the choice of procedure.

Common questions about choosing FDR or FWER

Should I use FDR or Bonferroni correction?

Use Bonferroni, or preferably Holm, when each finding will be named and defended individually, as with a trial endpoint or a genome-wide association locus. Use Benjamini–Hochberg when the output is a ranked list interpreted as a set, as with differential expression. The unit of your claim decides, not the number of tests.

When is the false discovery rate more appropriate than the family-wise error rate?

The false discovery rate is more appropriate when many hypotheses are tested, a moderate contamination of the results list is tolerable, and no single member of the list carries the conclusion. Screens, differential expression, and any analysis whose output feeds a further experiment fit that description. Confirmatory analyses supporting individual claims do not.

I have a list of significant genes from my RNA-seq screen — can I say any single one of them, on its own, is a real finding?

Not from the false discovery rate alone. A 5% false discovery rate tells you about 5% of the whole list is expected to be null, but it says nothing about which member that is — including the top-ranked gene. To defend one gene individually, protect it separately: an orthogonal assay, independent replication, or a within-family Bonferroni or Holm correction applied just to that follow-up claim.

A worked example at 20,000 tests

Take 20,000 gene-level tests, of which 1,000 are genuinely differential and 19,000 are null. The false-positive arithmetic below is exact given those assumptions; the true-positive counts are illustrative, because they depend on effect sizes and sample size.

With no adjustment at α = 0.05, the 19,000 null genes alone produce an expected 950 false positives. If 600 of the 1,000 real signals are also detected, roughly 61% of the reported list is noise.

Bonferroni at a family-wise error rate of 0.05 sets the per-test threshold at 0.05 / 20,000 = 2.5 × 10⁻⁶. The expected number of false positives across the whole screen is at most 0.05. Only large, well-measured effects survive.

Benjamini–Hochberg at q = 0.05 finds the largest rank k whose ranked p-value satisfies p(k) ≤ (k/m)q, and rejects every hypothesis of rank k or below. Suppose the procedure rejects 500 genes. About 25 of the 500 are expected to be null.

The instructive detail is the threshold Benjamini–Hochberg actually applied. With 500 rejections out of 20,000 at q = 0.05, the largest rejected p-value can be as large as (500/20,000) × 0.05 = 1.25 × 10⁻³ — five hundred times looser than the Bonferroni threshold on the same data. That is where the power difference comes from, and it carries an uncomfortable corollary: under Benjamini–Hochberg the fate of a gene with a fixed p-value depends on what else was on the array. Add 20,000 uninformative probes and the same gene can drop out; add a second strong signal and it can come back in.

Procedures that control the family-wise error rate

Bonferroni rejects when p ≤ α/m. It is valid under any dependence structure, assumes nothing, and yields a fixed numeric threshold that can be written down before any data exist — which is precisely why genome-wide association studies use it. Its weakness is conservatism when tests are correlated.

Šidák uses 1 − (1 − α)^(1/m) and is valid under independence. At m = 20,000 and α = 0.05, Šidák’s threshold is 2.565 × 10⁻⁶ against Bonferroni’s 2.500 × 10⁻⁶, about 2.6% less stringent. The gain rarely justifies the added independence assumption.

Holm (1979, Scandinavian Journal of Statistics) is a step-down procedure: order the p-values ascending and compare p(i) with α/(m − i + 1), stopping at the first comparison that fails. Holm controls the family-wise error rate under arbitrary dependence and is uniformly at least as powerful as Bonferroni, so Bonferroni is statistically dominated except where a fixed pre-specified threshold is itself the requirement.

Hochberg (1988, Biometrika) is a step-up procedure using the same constants, uniformly more powerful than Holm where its assumptions hold, but valid under independence or non-negative dependence rather than arbitrary dependence. The gap between the two is real without being a cliff. Hochberg’s shortcut rests on the Simes inequality (1986), a bound on the ordered p-values that holds under independence and was later proved to hold under non-negative association (Sarkar, 1998); outside that class it is not guaranteed, and the Simes global null test is known to run anti-conservative under negative dependence. Theoretical work has since bounded how far that drift can go and found it much smaller than arbitrary dependence would permit, so Hochberg on negatively dependent tests is a quantified risk rather than a collapse. Holm’s step-down closes the question by assuming nothing at all, which is why it remains the safe default whenever the correlation structure is unknown.

Westfall–Young permutation builds the null distribution of the smallest p-value by resampling, so the correlation among tests is estimated from the data rather than assumed away. It recovers substantial power when tests are strongly correlated — linkage disequilibrium blocks, cis-eQTL windows, repeated measures on the same subjects — at the expense of computation and of a permutation scheme that must preserve the dependence you care about. There is a hard floor on the resolution this buys: a permutation p-value can never be smaller than 1/(B+1) for B permutations, so certifying significance at a Bonferroni-scale threshold of 2.5 × 10⁻⁶ needs on the order of hundreds of thousands of permutations, or an extrapolated tail model, not the few thousand resamples common in exploratory use.

Fixed-sequence (hierarchical) testing orders the hypotheses in advance and tests each at full α until one fails. Nothing is spent on multiplicity at all, which makes it the most powerful option and the most brittle: every hypothesis below the first failure goes untested. Graphical approaches generalise this: each hypothesis is allocated a share of α, the shares summing to the total, each is tested at its own share, and the α of a rejected hypothesis is propagated along weighted edges to the hypotheses still under test. The FDA’s October 2022 guidance Multiple Endpoints in Clinical Trials discusses graphical and gatekeeping approaches alongside Bonferroni, Holm and Hochberg, which is the working menu for confirmatory trial analyses.

Procedures that control the false discovery rate

Benjamini–Hochberg (1995) controls the false discovery rate under independence and under positive regression dependency by ranking all p-values ascending and rejecting every hypothesis at or below the largest rank k that satisfies p(k) ≤ (k/m)q. Its adjusted p-values are enforced monotone by taking a cumulative minimum from the largest raw p-value downward and capping at 1, which is why a set of Benjamini–Hochberg q-values can never exceed 1 and can never invert the raw ordering.

Benjamini–Yekutieli (2001, Annals of Statistics) controls the false discovery rate under arbitrary dependence by dividing the level by the harmonic sum Σ(1/i) over i = 1 to m. At m = 20,000 that sum is about 10.5, so Benjamini–Yekutieli at q = 0.05 behaves roughly like Benjamini–Hochberg at q = 0.0048. Applying it reflexively, in settings where the positive-dependence condition already holds, tightens the effective level by an order of magnitude for no gain in validity.

Storey’s q-value (2002) exploits the fact that Benjamini–Hochberg controls the false discovery rate at π₀ × q, where π₀ is the proportion of true nulls. Storey estimates π₀ from the flat right-hand region of the p-value histogram and recovers that conservatism. Where π₀ = 0.95 the gain is negligible; where π₀ = 0.5, as in a well-powered cis-eQTL scan, plain Benjamini–Hochberg holds the actual false discovery rate near q/2 rather than q, and estimating π₀ recovers that conservatism at no cost to validity. How many additional discoveries that recovery yields depends on the p-value distribution of the alternatives, so it is worth reporting rather than assuming.

Local false discovery rate, developed by Efron, reports for each individual test the posterior probability that it is null given its own statistic, rather than a tail-area quantity computed over the list. This is the object most researchers already believe a q-value to be, and it is the defensible way to make a per-feature probabilistic statement inside a false discovery rate framework.

Questions about FDR and FWER procedures

Is Benjamini–Hochberg or Bonferroni better for my analysis?

Neither is better in general. Benjamini–Hochberg is far more powerful at large numbers of tests, because its effective threshold rises with the number of rejections. Bonferroni gives a fixed threshold, valid under any dependence, that can be prespecified before data exist. Choose by whether your conclusions are about a list or about individual hypotheses.

Which multiple testing correction should I use?

Choose the error rate first, then the procedure. For family-wise control, Holm dominates Bonferroni under any dependence, Hochberg is more powerful under positive dependence, and permutation recovers power when tests are strongly correlated. For false discovery control, Benjamini–Hochberg is the default, Benjamini–Yekutieli for arbitrary dependence, and Storey q-values when the null proportion is well below one.

Why did my gene’s result change when I reran the same test with more genes on the array?

Because Benjamini–Hochberg’s threshold is not fixed in advance — it depends on the ranked p-values of every other hypothesis tested alongside it. Add more null genes and the same p-value can drop below the cutoff and lose significance; add another strong true signal and it can cross back over. A Bonferroni or Holm threshold does not move this way; each is fixed by family size alone.

What dependence does to each

Dependence between tests changes which procedures remain valid and how variable the realised error is. Bonferroni and Holm hold under arbitrary dependence. Benjamini–Hochberg requires positive regression dependency on a subset, the condition Benjamini and Yekutieli proved sufficient in 2001; it is satisfied by multivariate normal statistics with non-negative correlations, by multivariate t, and by comparisons of many treatments against a single control.

Most genomic data satisfy that condition approximately: linkage disequilibrium between nearby variants is positive, and co-regulated transcripts are positively correlated. The awkward case is negative dependence induced by compositionality — microbiome relative abundances, deconvolved cell-type proportions, any vector constrained to sum to one — where positive regression dependency is not assured and Benjamini–Yekutieli or permutation is the honest answer rather than a reflex.

Dependence also inflates the variance of the false discovery proportion even where the false discovery rate is controlled correctly. The average across hypothetical repetitions is right; the single realised list can be far from it. Strong correlation makes the discoveries arrive in blocks, so a study either catches a correlated cluster of true signals or catches a correlated cluster of noise. That variance is an argument for reporting effect sizes and replication alongside q-values, and it is invisible to any procedure. Batch structure produces exactly this pattern, which is why batch-effect scrutiny and multiple-testing scrutiny belong in the same paragraph of a methods section.

What counts as a family, and why that is contested

Neither error rate is defined without a family, and no statistical principle fixes the family’s boundary. Rothman argued in Epidemiology (1990;1:43–46), under the title “No adjustments are needed for multiple comparisons”, that declining to adjust leads to fewer errors of interpretation when the data are observations on nature rather than random numbers. Perneger made a narrower version of the case in the BMJ (1998;316:1236), “What’s wrong with Bonferroni adjustments”: the universal null hypothesis that Bonferroni protects is rarely the hypothesis anyone cares about, and the severity of the adjustment depends on how many other tests the analyst happened to run.

The opposing position is that the family is determined by the decision the analysis supports, and that decision is knowable in advance. The FDA’s October 2022 guidance takes that position: Type I error control is expected to span the primary and secondary endpoint families together, and the boundary is drawn on the other side just as plainly, with exploratory endpoints left outside the adjustment because they are not used to support the conclusions the label rests on.

That is the workable resolution. A family is the set of tests any one of which, if significant, would be used to make the same claim. Its boundary is fixed by the prespecified protocol, not by the argument list of a software call. Where the boundary is genuinely a judgement — a secondary analysis that might or might not enter the abstract — state the judgement and report what happens under the alternative, which is the discipline registered endpoint reporting is designed to enforce.

A weaker standard is sometimes substituted for either error rate without anyone naming the swap: the per-comparison error rate, the chance of a false positive on one specific test considered in isolation, with no correction across the rest of the family at all. Reporting an uncorrected p < 0.05 for each of several planned contrasts, then calling the analysis “conservative” because each individual test used a standard threshold, is per-comparison control dressed up as family-wise control — the two are not the same guarantee, and a manuscript that never names its family is usually running the weaker one by default.

Frequently asked questions about defending your choice

Reviewer 2 says my correction for multiple comparisons is wrong — what do I say?

State the error rate you controlled, its level, and the exact set of tests it spanned — for example, “we controlled the false discovery rate at 5% across all 18,412 genes passing the expression filter.” Most of this objection resolves once the family is named explicitly; if the concern is a single top-ranked hit defended on a set-level guarantee, add independent protection for that one claim.

My reviewer wants me to explain what the “family” of tests is for my Bonferroni correction, but I ran dozens of exploratory tests before settling on four — what should I tell them?

Tell them which four tests support the claims in your abstract, and correct across exactly those — the family is defined by the decision the analysis supports, not by every comparison you looked at along the way. Report the exploratory tests separately as exploratory, uncorrected, and not used to justify any stated conclusion. Retrofitting a family after seeing the results is the pattern a statistical reviewer is trained to catch.

Does controlling FDR instead of FWER mean my analysis is less rigorous?

No. Controlling the false discovery rate is a different guarantee, not a weaker version of the same one, and under the complete null the two are identical. Rigour fails when the guarantee and the claim do not match — for example reporting a set-level false discovery rate and then defending one named feature as though it were individually protected.

How do I decide between q-values and adjusted p-values?

Decide by what the number must support. Family-wise adjusted p-values license a statement about an individual hypothesis. Tail-area q-values license a statement about the contamination of a list. If you need a per-feature probability of being null, neither is correct: report a local false discovery rate, which is the posterior probability for that feature alone.

Field conventions and where they came from

Genome-wide association studies control the family-wise error rate, at p < 5 × 10⁻⁸. That number is 0.05 Bonferroni-corrected for approximately one million effectively independent common variants, an effective count estimated by different methods, and by different groups, with broadly similar answers — Pe’er et al. (2008) and Dudbridge and Gusnanto (2008) are the two most often cited. The field chose family-wise control deliberately, because its unit of currency is the individually named, individually replicated locus, and because a threshold set before genotyping cannot be loosened by whatever else ended up on the array. It is worth being clear about what that number is: a convention, calibrated to the common-variant genotyping era, rather than a quantity derived from first principles for any particular study. Sample sizes have since grown by orders of magnitude, sequencing has pushed the effective number of independent tests up, and the appropriate threshold has been revisited repeatedly in the statistical-genetics literature — which is why genomics reviewers increasingly expect the threshold to be justified for the design at hand rather than cited as a fixed constant.

Exome-wide rare-variant burden testing generally uses about 2.5 × 10⁻⁶, which is 0.05 divided by the roughly 20,000 protein-coding genes, on the same family-wise logic. That distinction matters because the 5 × 10⁻⁸ genome-wide threshold and the exome-wide burden threshold solve two different multiplicity problems, not one at two scales: the first corrects across individually genotyped common variants, the second across genes into which many rare variants are collapsed before a single test is run per gene. Neither threshold absorbs a separate problem that reaches both designs — population stratification, which inflates test statistics genome-wide rather than at scattered loci. It is a property of the sample rather than of the assay, so a sequencing-based burden test is no more exempt from it than an array is. It is conventionally handled by genomic control, which rescales the statistics by their observed overdispersion, or by principal-component adjustment of the association model, and either correction is applied to the statistics before any multiplicity threshold is imposed rather than in place of one.

Differential expression and eQTL mapping control the false discovery rate, conventionally at 5% via Benjamini–Hochberg. Well-constructed eQTL pipelines use both rates in sequence: permutation within each gene’s cis-window produces one family-wise-corrected p-value per gene, then Benjamini–Hochberg is applied across genes. That two-stage structure matches the claim structure exactly — the per-gene claim is protected individually, the gene list is protected as a set — and it is the pattern to imitate whenever a paper names some features and enumerates others.

Confirmatory clinical trials control the family-wise error rate, typically at 2.5% one-sided. Regulatory multiplicity guidance is framed in family-wise terms throughout, and that is not an oversight: a regulator licenses individual claims that appear on an individual label, so an error rate defined over a list has nothing to attach to. The exposure being controlled is easy to quantify. Ten independent endpoints, each tested one-sided at 2.5%, carry a probability of about 22% that at least one is declared significant when every null is true — 1 − 0.975¹⁰ — which is why an unadjusted secondary-endpoint claim is treated as unsupported rather than as marginal.

How the choice is detected in a manuscript

Reading a manuscript for multiple-testing discipline means reconciling three numbers that are usually stated in three different places, and looking for eight specific tells.

First, the number of tests corrected for does not match the number of tests the methods describe. Count the comparisons in the design, not the ones in the results table.

Second, adjusted p-values are reported that exceed 1, or that are not monotone in the raw p-values. Benjamini–Hochberg adjusted values are enforced monotone and capped at 1, so either pattern means the arithmetic was done by hand or the wrong column was pasted.

Third, q-values are labelled as p-values, and a q of 0.03 is glossed as “a 3% chance that this gene is a false positive”. That reading is a local false discovery rate statement, and the tail-area q-value does not supply it.

Fourth, the false discovery rate is applied separately within each of several subgroups and the union is then reported as one findings list. The family was silently redefined between the analysis and the abstract.

Fifth, Bonferroni is applied to the comparisons that were reported rather than the comparisons that were run. This is visible whenever the results section contains fewer contrasts than the methods promised.

Sixth, the p-value histogram is not approximately flat over [0.5, 1]. A histogram that slopes or humps near 1 indicates conservative or dependent test statistics, and every downstream error rate — both of them — is then wrong regardless of which procedure was applied.

Seventh, the paper’s central conclusion rests on the top-ranked item of a false-discovery-controlled list, with no separate protection for that item.

Eighth, and most common, a procedure is named without its family: “we corrected for multiple comparisons using Benjamini–Hochberg”, with no statement of the set the correction spanned. That sentence is not reproducible, and it is the one that most often survives to publication. Automated screening for this class, including the omics multiple-testing check, works by extracting the test count from the methods and comparing it with the correction the results claim.

What reviewers say when it is mishandled

Reviewer comments on this class are usually short and specific, and the phrasings recur across fields.

“The authors control the false discovery rate but then base the central claim on a single gene; FDR does not license a per-gene statement.”

“It is not stated over which set of tests the Benjamini–Hochberg correction was applied.”

“Secondary endpoints are described as significant, but no prespecified Type I error control strategy covering primary and secondary endpoints is presented.”

“Bonferroni appears to have been applied to the comparisons reported rather than the comparisons performed.”

“Given the correlation among the outcome measures, a permutation-based correction would be more appropriate than Bonferroni, which is badly conservative here.”

“The choice of q = 0.10 is not justified. Please state the proportion of false findings the analysis is intended to tolerate, and why.”

The comment that most often decides the outcome is the second one. Where the family is not prespecified, the correction is unfalsifiable: any threshold at all can be obtained by choosing what to count, and a reviewer who notices that will treat every number downstream as provisional. Statistical reviewers escalate this quickly, and it appears frequently among statistical objections that end in a request for reanalysis rather than revision.

How to report the choice

State the error rate, the level and the family in a single sentence, before any result: “We controlled the family-wise error rate at 5% using Holm’s procedure across the four prespecified contrasts”, or “We controlled the false discovery rate at 5% using Benjamini–Hochberg across all 18,412 genes passing the expression filter.”

Report the number of tests, and give both raw and adjusted values so a reader can recompute. Justify the level by the asymmetry of consequences in your setting: what a false positive does to the field here, against what a missed signal does. A screen feeding a further experiment tolerates q = 0.10 comfortably; a claim entering a clinical guideline does not.

If you control the false discovery rate, say in the text that the guarantee is set-level, and say what protects any individual member you name. If you control the family-wise error rate, name the procedure and its dependence assumption — Hochberg and Šidák require independence or positive dependence, Bonferroni and Holm do not.

Prespecify. A correction chosen after the p-values were seen controls nothing at all, and a reviewer asking about overclaiming will read a post hoc correction as a selection effect, not a safeguard.

Limitations of both error rates

Both the false discovery rate and the family-wise error rate are guarantees about Type I error under a correctly specified model, and neither addresses several failure modes that damage papers more often.

Neither rate protects against a wrong test. Correcting 40 p-values that each came from an inappropriate model produces 40 correctly adjusted wrong numbers, which is the situation behind most wrong statistical test objections.

Neither addresses non-independence of observations. Where the unit of analysis is wrong — cells treated as replicates, repeated measures pooled — the raw p-values are already invalid, and multiplicity correction operates downstream of the damage. That is a pseudoreplication problem, and no adjustment reaches it.

Neither says anything about the direction or magnitude of an estimate. Effects that only just clear a stringent threshold are, on average, overestimated, because clearing the threshold requires a favourable draw of sampling error. A stringent family-wise threshold therefore produces fewer false claims and more exaggerated true ones, which is why replication effect sizes in genome-wide association studies are systematically smaller than discovery estimates, and why post hoc power computed from an observed effect is uninformative.

Neither touches selective reporting. If analyses that produced nothing were never written down, the family the correction spans is not the family that was tested, and the guarantee is void in a way no reader can detect from the manuscript.

And the family-wise error rate has one further honest limitation: it says nothing about the composition of a findings list. A study that rejects 200 of 20,000 hypotheses under Bonferroni has not shown that at most 5% of the 200 are wrong; it has shown that, whatever the true state of the nulls, the chance the analysis produced even one false rejection was at most 5%. Those are different assurances, and papers frequently report the first while having established the second.

Neither error rate, once satisfied, licenses a plain confidence interval for the selected hypotheses either. An interval computed only for the features that already cleared a significance threshold is narrower than it should be, because the same data drove both the selection and the estimate — the same winner’s-curse mechanism that inflates the effect sizes of threshold-clearing findings, here applied to interval width rather than to the point estimate. Reporting an unadjusted 95% interval beside an FDR- or FWER-selected finding overstates precision in exactly the way post hoc power overstates it after the fact.

Related

Multiple comparisons objections · Statistics objections · Pseudoreplication · Independence of n · Table 2 fallacy

Checked before submission by the omics multiple-testing reviewer, which reconciles the number of tests described in the methods with the correction claimed in the results and flags per-feature claims made from a set-level guarantee.

Review my manuscript