What is a confidence interval?

A confidence interval is a range of values, computed from data by a procedure whose coverage is guaranteed in the long run: a 95% confidence interval comes from a method that captures the true parameter in 95% of repeated studies. The interval shows which effect sizes are compatible with your data, and how precisely the effect was estimated.

That definition places the 95% on the procedure rather than on the particular interval in front of you, and almost every reporting error in the literature comes from moving it. This page covers what the coverage statement actually promises, how an interval is built, the five misinterpretations named in the methodological literature, when the standard formula returns the wrong interval, the arithmetic checks that expose a mishandled interval in a manuscript, and what reviewers write when they find one.

What the 95% refers to

The 95% in a 95% confidence interval is a property of the method, assessed over hypothetical repetitions of the study, not a probability attached to the single interval you computed. Run the same design and the same interval formula 100 times on data generated by the same underlying truth, and roughly 95 of the intervals produced will contain that truth. Which 95 is unknowable, and the one you have is either right or wrong.

This is the frequentist coverage guarantee, and it is deliberately modest. It says nothing about the parameter being random; the parameter is fixed and the interval is what varies from study to study. A Bayesian credible interval makes the statement researchers usually want — a 95% posterior probability that the parameter lies in the stated range — but it requires a prior distribution, and it is a different object with different assumptions. Reporting a frequentist interval and interpreting it as a credible interval is the most common conceptual slip in applied work, and it is the first of the misinterpretations catalogued below.

Common questions about what a confidence interval means

Does a 95% confidence interval mean there’s a 95% chance the true value is inside it?

No. The 95% describes the method’s long-run performance across repeated studies, not a probability attached to the one interval you calculated. Once the data are collected and the interval is computed, the true parameter is either inside it or it isn’t — there is no remaining randomness under the frequentist definition for a probability statement to attach to.

What’s the difference between a confidence interval and a credible interval?

A confidence interval is frequentist: its coverage rate is a property of the procedure across repeated sampling, and it requires no prior distribution. A credible interval is Bayesian: it states a posterior probability that the parameter lies in the stated range, conditional on a prior and the observed data. The two often report similar numbers but answer different questions built on different assumptions.

Can I say I’m 95% confident this specific interval contains the true value?

Not under the standard definition — “95% confident” describes the procedure, not this one instance. Journals and reviewers that police statistical language object to phrasing such as “we are 95% confident the true difference lies between X and Y” for this reason; the compatibility-interval reformulation described further down this page exists partly to give authors a phrasing that doesn’t invite the mistake.

How a confidence interval is constructed

A confidence interval is constructed by taking a point estimate, attaching a measure of its sampling variability, and extending outward far enough that the resulting range covers the truth at the stated rate. In its most common form — the Wald interval — the recipe is estimate ± (critical value × standard error), with the critical value 1.96 for a 95% interval under a normal approximation, or the corresponding t quantile when the standard error is itself estimated from a small sample.

The 1.96 is specific to a two-sided interval. A one-sided 95% interval — appropriate when only one direction of effect matters, such as demonstrating a value stays above a non-inferiority margin — uses 1.645 and is bounded on only one side. Reporting a one-sided interval as though it were the usual two-sided kind overstates precision on the open side and is a distinct error from the ones catalogued below.

Three ingredients therefore determine the interval: the estimate, the standard error, and the reference distribution. Each can be wrong independently. A correct estimate with an underestimated standard error yields an interval that is too narrow and a coverage rate below 95%. A correct standard error referred to a normal distribution when a t distribution with 8 degrees of freedom was required yields the same failure in a smaller way — 1.96 instead of 2.31, an interval about 15% too narrow.

Not every interval is built this way. Score-based intervals invert a hypothesis test rather than adding a margin to an estimate. Profile-likelihood intervals trace the likelihood function and need not be symmetric. Bootstrap intervals resample the data and read percentiles off the resulting distribution. These alternatives exist because the Wald recipe fails in identifiable situations, described further down this page.

What a confidence interval tells you that a p value does not

A confidence interval carries the magnitude and the units of the effect, and a p value carries neither. “Mortality was lower in the treated group (p = 0.03)” tells a reader that some difference was detected; “absolute risk difference −4.2 percentage points, 95% CI −7.9 to −0.5” tells the reader that the data are compatible with anything from a trivial half-point reduction to a substantial eight-point one, which is a different clinical conclusion at each end.

The two are arithmetically linked for the same test on the same scale: a 95% Wald interval excludes the null value precisely when the corresponding two-sided p value is below 0.05. That correspondence is why an interval is strictly more informative — it contains the test result and adds the effect size and its precision. The International Committee of Medical Journal Editors states the preference directly in its Recommendations, under Statistical methods: “When possible, quantify findings and present them with appropriate indicators of measurement error or uncertainty (such as confidence intervals)”, followed by the instruction to avoid relying solely on statistical hypothesis testing, such as P values, which fails to convey important information about effect size and precision of estimates.

The correspondence is not universal. Fisher’s exact test paired with a Wald interval, a score interval paired with a Wald test, and any other mismatch between the interval method and the test method can put a p value of 0.04 alongside an interval that includes the null. That combination is not automatically an error, but it obliges the methods section to name both procedures rather than only one.

The five misinterpretations named in the literature

Greenland and colleagues catalogued 25 numbered misinterpretations of statistical tests, p values, confidence intervals and power in the European Journal of Epidemiology in 2016 (31:337–350). Five concern confidence intervals directly, and each is stated there as a false proposition:

• 19. “The specific 95% confidence interval presented by a study has a 95% chance of containing the true effect size.”
• 20. “An effect size outside the 95% confidence interval has been refuted (or excluded) by the data.”
• 21. “If two confidence intervals overlap, the difference between two estimates or studies is not significant.”
• 22. “An observed 95% confidence interval predicts that 95% of the estimates from future studies will fall inside the observed interval.”
• 23. “If one 95% confidence interval includes the null value and another excludes that value, the interval excluding the null is the more precise one.”

Number 21 has a usable quantitative correction. Cumming and Finch showed in American Psychologist (2005) that two independent 95% intervals can overlap by roughly half the average margin of error and still correspond to a p value near 0.05 — so “the error bars touch, therefore the groups do not differ” is wrong in the direction that costs findings, not the direction that manufactures them. The overlap rule fails the opposite way, too, whenever the two estimates are not independent: paired or repeated measurements on the same subjects — a before/after comparison, say — can show non-overlapping individual intervals while the paired difference is not significant, because neither individual interval accounts for the within-subject correlation. Number 22 matters most in meta-analysis, where the confidence interval around a pooled estimate answers a different question from the prediction interval for a future study, and the prediction interval is usually far wider.

Width, sample size and precision

The width of a confidence interval is governed by the standard error, which for an ordinary mean shrinks with the square root of the sample size. Quadrupling n halves the interval width; doubling n narrows it by about 29%. This is the arithmetic behind every underpowered study’s wide interval, and it is why a proposal to “add a few more animals” rarely rescues an inconclusive result.

Two rules of thumb are worth carrying. First, the Cochrane reconstruction: for a large-sample 95% Wald interval, the standard error is (upper limit − lower limit) ÷ 3.92, which lets you recover the standard error from any published interval and check the reported p value against it. Second, the rule of three, from Hanley and Lippman-Hand in JAMA (1983): if an adverse event occurred zero times in n patients, the upper limit of the 95% interval for its rate is approximately 3/n. Zero events in 30 patients is compatible with a true rate as high as 10%, which is the honest reading of most small safety cohorts.

A wide interval is a statement about precision, not a failure of the analysis, and it should be reported as such rather than retrofitted with a power calculation after the fact — see what to do when a reviewer asks for post-hoc power and when a reviewer says the sample size is too small.

Questions about confidence interval width and sample size

Why is my confidence interval so wide?

A wide interval means the standard error is large relative to the effect — usually from a small sample, high outcome variability, or a rare event. Width scales with the standard error, which for a simple mean shrinks only with the square root of n, so a wide interval is a precision problem to fix with more data or a better design, not a flaw in the analysis.

Does a wider confidence interval mean the study is worse?

Not necessarily — it means the estimate is less precise, which is an honest reflection of the data available rather than an error. A wide interval from a well-designed small trial is more trustworthy than a narrow one from a badly confounded observational study, because width reflects only sampling variability and says nothing about bias, confounding or measurement error.

How much bigger does my sample need to be to cut my confidence interval in half?

Roughly four times bigger, for a simple mean or proportion, because the standard error shrinks with the square root of the sample size rather than with the sample size itself. Doubling n only narrows the interval by about 29%, which is why “just add a few more subjects” rarely rescues an inconclusive result — the arithmetic works against that shortcut.

When the standard formula returns the wrong interval

The Wald interval is the default in most software and it fails in several well-characterised situations.

Proportions near 0 or 1. Brown, Cai and DasGupta showed in Statistical Science (2001; 16:101–133) that the Wald interval for a binomial proportion has chaotic coverage that does not settle down even at large n, and can fall far below the nominal 95%. They recommend the Wilson score interval or the Jeffreys interval for small samples and Agresti–Coull for larger ones. A published proportion whose interval extends below 0% or above 100% is a Wald interval that should have been a Wilson one.

Ratio measures. Odds ratios, risk ratios and hazard ratios are estimated on the log scale, so their intervals are symmetric in logs and asymmetric when printed. An odds ratio of 1.50 reported as 95% CI 1.20 to 1.80 is arithmetically suspicious, because the log-scale midpoint of 1.20 and 1.80 is √(1.20 × 1.80) = 1.47, not 1.50. A separate caveat applies to odds ratios specifically: they are non-collapsible, meaning the adjusted odds ratio and its interval can shift after adding a covariate even when that covariate is not a confounder — unlike risk differences and, approximately, risk ratios. A moved odds-ratio interval is therefore not by itself proof that the added variable was confounding anything.

Sparse data and separation. In logistic regression with few events per covariate, the Wald standard error inflates as the coefficient grows, and the interval can widen absurdly or cover the null while the likelihood ratio test rejects it. Profile-likelihood intervals, or Firth’s penalised likelihood, behave correctly where the Wald interval does not.

Correlated observations. Any interval computed as though observations are independent when they are not is too narrow. Repeated measurements from the same animal, multiple cells from one culture, and patients clustered within centres all inflate the effective replicate count and shrink the interval accordingly — the mechanism described in pseudoreplication and objected to as n is not independent. Cluster-robust standard errors fix this only when the number of clusters is large; Cameron and Miller’s practitioner’s guide in the Journal of Human Resources (2015) sets out the small-cluster corrections, including the wild cluster bootstrap and referring the statistic to a t distribution with G−1 degrees of freedom.

Non-normal or skewed statistics. Bootstrap intervals help here, but the plain percentile bootstrap is biased when the statistic’s sampling distribution is skewed; the bias-corrected and accelerated (BCa) variant exists for that reason and should be named in the methods when used.

Confidence intervals quantify random error only

A confidence interval quantifies sampling variability and nothing else. It contains no allowance for confounding, selection, measurement error, protocol deviation, or the analytic choices made before the interval was computed. A 95% interval from a badly confounded observational study is a precise statement about the wrong quantity, and increasing the sample size narrows it around the same bias.

This is worth stating explicitly in a manuscript because narrow intervals read as authoritative. A hazard ratio of 1.8 with a 95% CI of 1.6 to 2.0 in a cohort with unmeasured confounding is not stronger evidence of causation than the same estimate with a wider interval; it is a better-estimated association of unknown causal status. Where the analysis conditioned on a variable caused by both exposure and outcome, collider bias shifts the interval bodily and the coverage guarantee says nothing about it. The same caution applies to intervals printed beside covariates in a regression table, which is the Table 2 fallacy.

Bias analysis is the honest complement. An E-value, a simple bias parameter, or a quantitative sensitivity analysis extends the interval to reflect systematic error, and doing so changes many “significant” findings into ranges that include the null.

How mishandled intervals are detected in a manuscript

Confidence-interval errors are detectable by arithmetic on the reported numbers, which is what makes them one of the more tractable statistical review tasks. The checks below run on a results table alone, with no access to the data.

The point estimate must sit inside the interval, and for a log-scale Wald interval it must equal the geometric mean of the limits: OR = √(lower × upper). A discrepancy beyond rounding means the estimate and the interval came from different models, which is usually a copy error between analysis versions.

The interval must agree with the p value. Recover the standard error as (upper − lower) ÷ 3.92, form z = estimate ÷ SE, and compare against the reported p. An interval excluding the null beside p = 0.09, or an interval spanning the null beside p = 0.01, is an inconsistency requiring an explanation. This reconstruction assumes a symmetric Wald interval; run against a Wilson, Jeffreys, exact or bootstrap interval — all deliberately asymmetric by design — it recovers the wrong standard error and manufactures a false-positive inconsistency, so confirm which method produced the interval before flagging a mismatch.

Interval limits must respect the parameter’s range. Proportions below 0 or above 1, correlation limits outside −1 to 1 (which happens when a Fisher z transformation was skipped), and negative lower bounds on inherently positive quantities all indicate the wrong interval method.

Interval widths must scale with group size. A subgroup of 12 patients whose interval is as tight as the interval for the 400-patient full cohort indicates the wrong denominator, an interval computed on the wrong scale, or clustered observations counted as independent.

Within-group intervals must not be used to infer a between-group difference. Reporting that the treated group improved (95% CI excluding zero) while the control group did not (95% CI including zero) is not a comparison; the interval for the difference has to be computed directly, and it is frequently much less impressive.

Baseline-to-follow-up intervals within randomised arms deserve particular scepticism, because a within-arm change interval answers a question randomisation was not designed to answer.

Four of the six are closed-form tests on numbers already present in the table, so an automated check can run them directly; the last two require reading what the text claims the intervals show. The harder judgement is the one no arithmetic reaches: whether the prose describing an interval is more confident than its limits allow, the pattern handled by the overclaim check.

What a peer reviewer says when confidence intervals are mishandled

Reviewers rarely write “your confidence interval is wrong.” The phrasings below are typical of how the objection is worded in review reports, and each one names a specific defect described above.

“Effect estimates should be presented with confidence intervals rather than p values alone.” “The reported confidence interval is inconsistent with the p value given in the same row.” “The authors interpret a non-significant result as evidence of no effect; the confidence interval extends to a clinically important benefit.” “Overlapping error bars are not a valid test of the difference between groups; please report the interval for the difference.” “The intervals appear to assume independence between repeated measurements from the same subject.” “Confidence intervals for proportions near the boundary should use an exact or score-based method.” “The precision implied by these intervals is not credible given the number of events.”

Two of these decide papers. The first is the “absence of evidence” objection — Altman and Bland’s BMJ note of 1995 remains the standard citation — where an interval running from 0.65 to 1.05 is described as showing no effect, when it is equally compatible with a 35% reduction. The second is the independence objection, which cannot be answered by re-analysis in a revision if the design itself lacked independent replicates; see my replicates disagree and when a reviewer says the wrong statistical test was used.

Questions about confidence intervals and peer review

Reviewer 2 says I should report confidence intervals instead of just p-values — what do I actually change?

Add the interval to every effect estimate in your results and abstract, not only the ones that reached significance, and give both limits in the same units and decimal precision as the estimate. This satisfies ICMJE’s stated preference and CONSORT/STROBE’s reporting items directly; it does not require running a different statistical test, only reporting a fuller summary of the test you already ran.

My confidence interval crosses zero — does that mean there’s no effect?

No — it means the data cannot rule out zero at the stated confidence level, which differs from evidence the true effect is zero. An interval of −7.9 to −0.5 crosses nothing and still spans a wide range of effect sizes; one from −5 to +5 crosses zero and stays compatible with a moderate effect either way. Interpret both limits rather than treating the crossing as the whole finding.

Why did the reviewer say overlapping confidence intervals don’t prove there’s no difference between groups?

Because two independent 95% intervals can overlap substantially and still correspond to a statistically significant difference between the groups — the comparison has to be tested directly, on the difference itself, not read off two separate intervals. Overlap is a weaker, more conservative signal than a formal test of the contrast, which is exactly why reviewers ask for the latter instead.

How to report a confidence interval

Report the interval level, both limits, the estimate, and the units, in that sentence. “95% CI” rather than “CI”; both limits separated by “to” rather than a hyphen, because a hyphen is ambiguous when a limit is negative; and the same number of decimal places on the estimate and on the limits.

The reporting guidelines are explicit. CONSORT 2025, the updated 30-item checklist for randomised trials, requires that results for each outcome be given for each group with the estimated effect size and its precision, such as a 95% confidence interval; for binary outcomes it asks for both absolute and relative effect sizes. STROBE item 16a requires unadjusted estimates and, where applicable, confounder-adjusted estimates with their precision, so that readers can see how far and in which direction adjustment moved the estimate.

Interpret both limits in the discussion, not just whether the interval crosses the null. State what the lower limit would mean if it were the truth, and what the upper limit would mean, and whether either would change practice — that is the reading that distinguishes an inconclusive study from a genuinely null one, and it is closely related to whether the effect size is meaningful. Where many intervals are reported, say how many and whether any adjustment was made; twenty independent 95% intervals will exclude the null about once by chance, which is the interval-scale version of the multiple comparisons problem. A Bonferroni-adjusted simultaneous interval — built at confidence level 1 − 0.05/k for k comparisons — restores family-wise coverage at the cost of extra width, and is the interval-scale equivalent of a Bonferroni-corrected p value; reporting k ordinary 95% intervals as though each were independently trustworthy repeats the same error in a form that arithmetic checks on a single row will not catch.

Contested: should they be called compatibility intervals

A significant body of methodological opinion holds that the word “confidence” is itself the problem. Amrhein, Greenland and McShane argued in Nature in March 2019 (567:305–307), in a comment supported by more than 800 signatories, that authors should interpret estimates rather than tests and explicitly discuss the lower and upper limits of what they term compatibility intervals — the interval containing the parameter values most compatible with the data under the model, with no implied confidence in any one of them.

The relabelling is not universally accepted, and no major reporting guideline adopts it: ICMJE, CONSORT 2025 and STROBE all still say “confidence interval”. Its practical value is that it forecloses misinterpretation 19: a reader who sees “values compatible with these data” is less likely to read a probability statement into it than one who sees “we are 95% confident.” Its limitation is that compatibility is still conditional on the entire model — the distributional assumptions, the adjustment set, and the absence of systematic error — so the renamed interval carries exactly the same blind spot regarding bias described above. Using the term without explaining it in a manuscript submitted to a clinical journal will often prompt a query rather than approval, so if you adopt it, define it at first use and report the level as usual.

Frequently asked questions

What is the formula for a 95% confidence interval?

For a simple mean under the normal approximation, it is estimate ± 1.96 × standard error, with 1.96 replaced by 1.645 for a one-sided 95% interval and by the relevant t quantile when the standard error is itself estimated from a small sample. Proportions, ratio measures and non-normal statistics need different formulas, covered further up this page, because the normal-approximation formula fails for each of them.

Can a confidence interval be negative?

Yes, whenever the underlying quantity can itself be negative or zero — a mean difference, a regression coefficient, a risk difference in percentage points. A negative limit is a problem only when the parameter is mathematically bounded below zero, such as a proportion, a variance, or a ratio measure like an odds ratio; there, it signals the wrong method, not a genuine negative effect.

How do you write a confidence interval in a manuscript?

Give the confidence level, both limits, the point estimate and the units in one sentence — for example, “mean difference 4.2 mmHg, 95% CI 1.1 to 7.3.” Use “to” rather than a hyphen between the limits, because a hyphen is ambiguous when a limit is negative, and match the decimal precision of the limits to the estimate itself.

Related

Effect size not meaningful · Sample size too small · Post-hoc power · Multiple comparisons · Pseudoreplication · Confounding

Checked before submission by statistical objections, which recomputes the standard error implied by each reported interval and flags limits that disagree with the accompanying p value or with the prose describing them.

Review my manuscript