Responding to reviewer comments on methods and design

Methods objections divide cleanly into two kinds, and the division decides how much work your revision is. Some are reporting gaps — the experiment was fine, the description was not, and the fix is text. Others are design gaps, where the experiment as performed cannot support the claim, and the fix is either new data or a narrower claim.

Reviewers rarely mark which kind they mean. Sorting them yourself, before you plan the revision, is the most useful hour you will spend on it.

This page covers the six objections that make up most of this family, the question that sorts them, the six design decisions no rewriting recovers, a worked triage of one review, how to detect each kind in your own manuscript, the sentences reviewers write when they are unconvinced, and what to do when the remedy a reviewer named is not available to you.

The objections

• Reviewer wants more controls — which control they mean is usually inferable, and one class is a reporting fix.
• Reviewer wants an orthogonal method — a request about believing the measurement, not the analysis.
• Reviewer says the results are correlative, not causal — one of the most common objections in molecular biology.
• Reviewer says the mechanism is not shown — often answerable by rewriting rather than by experiments.
• Reviewer questions whether the experiment was replicated — nearly always a reporting gap, and an easy one.
• Reviewer says the model system is not appropriate — the objection that most often ends in a narrower claim.

Sorting reporting gaps from design gaps

Ask one question of each comment: if I had written the methods section perfectly, would this objection still stand?

If it would not, it is a reporting gap. Controls you ran but did not describe, replicates you performed but did not state, validation you did but put in a drawer. These are the majority of methods comments, they are answered with text and a supplementary figure, and they take days rather than months.

If it would, it is a design gap, and you have three options: run the experiment, provide equivalent evidence by another route, or narrow the claim to what the design supports. All three are legitimate. Only the first is expensive, and it is chosen more often than it needs to be.

A third kind hides among the other two and is worth separating on the first pass: a scope objection wearing methods clothing. “The mechanism remains unclear” at a journal that publishes mechanistic advances is a statement about fit rather than rigour, and no supplementary figure answers it — reviewer says this is incremental covers that conversation. Filing a scope objection in the design pile is how authors spend six months on an experiment that was never going to change the decision.

Do the sort in writing rather than in your head. Number every comment, write one of three labels beside it — reporting, design, scope — and record the specific sentence in the manuscript each comment lands on. Comments that land on no sentence are usually scope. Comments that land on a sentence in the abstract are load-bearing whatever their label, because those are the sentences the editor will read against the revision.

Why most methods objections are reporting gaps

Most methods objections are reporting gaps because reporting standards ask for sentences authors omit whenever the answer is unremarkable, not because the experiments behind them were badly done. The vehicle control was run and was flat. The three cultures agreed. Nobody was excluded. Each of those is a sentence that never got written, and its absence is indistinguishable on the page from the thing never having happened.

ARRIVE 2.0, the 21-item animal-research reporting checklist published by the NC3Rs in PLOS Biology in 2020, is explicit that silence is non-compliance rather than a neutral absence. Sub-item 3a asks for inclusion and exclusion criteria and adds: “If no criteria were set, state this explicitly.” Sub-item 3b asks you to report animals or data points left out of the analysis and adds: “If there were no exclusions, state so.” Sub-item 3c asks you to “report the exact value of n in each experimental group” for each analysis. Two of those three sub-items require a sentence even when the answer is nothing.

The same structure runs through the other reporting standards. STROBE, the 22-item guideline for observational epidemiology published in October 2007, states plainly that “the checklist is not an instrument to evaluate the quality of observational research” — it governs what appears in the text. CONSORT 2025, the updated trial-reporting statement with 30 items and a participant flow diagram, governs the trial report rather than the trial. A study can satisfy every item and still be weak; a strong study can fail half of them because nobody wrote the numbers down. That asymmetry is exactly why so many methods comments are answered with prose.

Ticking a checklist at submission does not produce those sentences. The IICARus trial — Hair, Macleod and Sena with the IICARus Collaboration, in Research Integrity and Peer Review in 2019 — randomised manuscripts submitted to PLOS ONE between an arm where authors were asked to complete an ARRIVE checklist at submission and standard editorial practice, then scored compliance in the published papers. Full compliance was essentially absent in both arms and the intervention produced no broad improvement. Ticking a box and writing the sentence are different acts, and the trial separated them cleanly. What produces compliant text is reading the manuscript against the items and adding what is missing — which is the same work as pre-empting this whole family of objections. The ARRIVE guidelines sets out the items one at a time.

The six decisions no rewriting recovers

Six ARRIVE 2.0 items cannot be satisfied retrospectively, and they are the cleanest available test for a design gap. They are sub-item 1b, the experimental unit; 2b, sample size justification; 4a, the randomisation method; 4b, control of confounders; 5, the stages at which blinding was applied; and 6b, the primary outcome measure. Every one of them is a decision made before the first animal is dosed or the first plate is seeded, and no amount of careful writing recovers a decision that was never made.

An objection that lands on any of those six is a design gap by construction. A reviewer asking how animals were allocated to groups is not asking for a clearer sentence; they are asking whether allocation was random, and if it was not, the sentence you can honestly write is a limitation rather than a method.

Sub-item 1b carries the most downstream weight. The experimental unit is the smallest division of the experimental material that can be independently allocated to a treatment, and once it is fixed, it determines what n means in items 2, 3c, 7 and 10. Getting it wrong does not produce a vague objection — it produces a specific one about independence, and the arithmetic moves p-values by orders of magnitude. Reviewer says my n is not independent covers both the structure and the response.

The practical consequence for a revision is that arguing about these six is unproductive. Concede the decision, state its effect on what can be concluded, and move the claim. A reviewer who reads “randomisation was not performed; we have therefore described the comparison as exploratory and removed the causal wording from the abstract” is reading an author who understands their own design.

A worked triage of one review

Consider a constructed but ordinary case: a manuscript reporting that a kinase inhibitor reduces tumour growth through an immune-dependent mechanism, tested in a flank xenograft in nude mice, with a western blot showing target engagement. Six comments arrive. Sorting them takes about an hour and changes the revision from a year to roughly six weeks.

Two comments in that set are answered with text. One is answered by narrowing one sentence in the abstract, and one is a scope objection that no methods work touches. Two require bench work, and only one of those two is load-bearing on the central claim. The default response — running all three experiments — buys nothing on comment six, which is the one most likely to decide the paper.

The immune comment is worth dwelling on, because it shows how much a compartment-level reading saves. Nude mice lack mature T cells but retain B cells, natural killer cells and innate immunity; NSG mice lack substantially more. An immune claim in a nude host is not uniformly unsupportable — an innate claim may survive where an adaptive one cannot. Naming the compartment converts an apparently fatal objection into a specific, defensible narrowing.

How to detect each kind before a reviewer does

Methods gaps are detected by reading your own manuscript in an order you never read it in while writing: claims first, then the figures that support them, then the methods that describe those figures.

Read the title and abstract against the figure panels. Write each claim in the abstract on one line and name the figure panel that establishes it. A claim with no panel beside it is the sentence that will draw the objection. A mechanistic verb — drives, promotes, via, through — with no perturbation behind it is the specific case; see reviewer says the results are correlative, not causal.

Check every n for its unit. For each reported n, state what one unit is: one animal, one culture, one field of view, one patient. If two figures report n with different units and neither says so, a reviewer will find it.

Check whether an exclusion sentence exists at all. Absence is the failure mode, not the exclusions themselves. Under ARRIVE sub-items 3a and 3b, a study with no exclusions still owes the reader a sentence saying so.

Check the raw data before assuming a control is missing. Vehicle-only conditions, untreated baselines, secondary-antibody-only panels and no-template controls are routinely run and routinely omitted. Reviewer wants more controls sorts the classes; the one worth doing is almost always the specificity control.

Check whether your second method shares the first one’s failure mode. Two antibody-based assays against the same epitope share the antibody’s specificity problem. RNA-seq validated by qPCR shares reverse transcription and the same RNA preparation. Different is not orthogonal — orthogonal validation sets out the pairings that hold.

Check that a negative result has a positive control. If you reported no effect, a reader needs evidence the assay would have detected one. Without it, “no effect” and “the assay did not work” are indistinguishable, and no statistical argument separates them.

Check the representative image against the quantification. A representative panel standing in for a quantification that was never performed is the microscopy version of the replication gap, and image quantification reports every panel where it has happened.

What reviewers write when the methods do not convince them

Reviewers rarely write “this is a reporting gap” or “this is a design gap”. They write the specific version, and the wording tells you which one you have.

Reporting-gap comments name a missing statement: “It is unclear how many independent experiments were performed, or whether the panel shown is representative or best.” “No information is given on whether any animals or data points were excluded; please state the criteria or confirm there were none.” “The number of biological replicates is not stated for any panel.” “Antibody clone, catalogue number and validation are not reported.” “The statistical unit does not appear to match the experimental unit; sections from the same animal are not independent observations.” Every one of those is answered by writing a sentence and, at most, adding a supplementary panel.

Design-gap comments name a missing capability: “The data establish an association between X and Y but not that X is required for Y.” “A single siRNA at one concentration does not exclude off-target effects; a second reagent or a rescue is needed.” “The finding rests on one antibody-based assay and should be confirmed by an independent approach.” “The conclusions about adaptive immunity cannot be drawn in this host.” “Allocation to groups is not described; if animals were not randomised, this should be stated as a limitation and the causal language removed.”

The tell is the verb. “Please state”, “please clarify”, “it is unclear” point at text. “Cannot be drawn”, “does not establish”, “is needed to exclude” point at the design. A comment mixing both usually contains two objections that need separate answers, and answering only the text half is how a revision earns a second round.

The three routes when the design gap is real

Three routes answer a genuine design gap, and the choice turns on one question: does the objection bear on a sentence in the abstract?

Run the experiment. Correct when the objection could overturn the central claim and the experiment is tractable in your system. Include the rescue if the objection is about specificity, because a reviewer who has accepted a knockdown and still objects is usually asking for one, and supplying it pre-empts the follow-up round.

Supply equivalent evidence by another route. The most under-used of the three. A functional readout often answers a measurement doubt more convincingly than a second measurement does — if the reviewer doubts that a measured increase matters, showing that perturbing it changes the phenotype answers a larger question. Other routes: reanalysing a deposited dataset that contains the comparison, an existing cohort or archived material, a published orthogonal measurement in the same system cited explicitly as previously published, or a design change in the analysis rather than at the bench.

Narrow the claim. Legitimate, frequently correct, and treated as defeat far more often than it deserves. “X is required for Y” is a real and interesting claim that a loss-of-function experiment supports; “X drives Y” is a larger one that it does not. A smaller claim the data clearly support is more publishable than a large one under dispute. When you take this route, say so explicitly in the response letter — reviewers notice a claim quietly softened and read the silence as evasion rather than compliance.

Whichever route you take, the same discipline applies to all of them: the claim must move with the evidence, in the title, the abstract, the discussion and the figure legends, not only in the section the reviewer pointed at.

When the obvious remedy is unavailable

The remedy a reviewer names is sometimes not available, and saying so plainly with a substitute works far better than a defence that leaves the manuscript unchanged.

The reagent, line or person is gone. State the constraint factually — the line failed authentication, the cohort is closed, the person who performed the work has left — then supply what remains: archived material, banked lysates, the deposited dataset, or a narrowing that removes the dependency. Do not describe the constraint as a hardship; describe it as a limit on what the paper claims.

The experiment exceeds what the revision window allows. Say what it would establish and how long it would take, offer a smaller analysis addressing the same concern, and ask the editor to weigh proportionality. Editors rarely require an experiment out of proportion to the claim it supports, and they cannot weigh a request you did not make.

The model does not exist in your system. Name the alternative, say why it is unavailable, and state what the limitation means for the claim rather than for the field. Then check whether the claim can be narrowed until the missing validation is no longer load-bearing — “detected by ChIP-seq” is a smaller and safer claim than established occupancy.

The reviewer’s premise is wrong. Say so once, factually, with a citation or an analysis, and offer something alongside: the alternative analysis in supplementary material, or the data that let a reader judge. Reasoned disagreement is normal and editors expect it; an ignored comment is what causes trouble. How to write a response to reviewers sets out the shape a declined request should take.

The answer is that the paper is at the wrong journal. If the venue requires a mechanistic advance and the work is descriptive, a well-matched journal publishes faster than a year of experiments aimed at a scope requirement. That is a legitimate outcome, not a retreat; what to do after a rejection covers the sequencing.

Sequencing a methods revision

Sequence a methods revision in one order: sort, then fix the claims, then run experiments, then rewrite the methods, then write the letter. Reversing any two of those creates work you throw away.

Sort first, because the sort determines which experiments are worth starting. Fix the claim set second, before any bench work, because narrowing one abstract sentence frequently removes the need for two of the requested experiments — and because the narrowed claim tells you what a new experiment has to show. Run the experiments third, with the reduced list. Rewrite the methods fourth, adding the sentences the reporting gaps require, including the ones whose answer is “none” and “no criteria were set”. Write the response letter last, quoting the revised text under each comment so the editor can see the change without opening the manuscript.

The failure this order prevents is the most common one in the whole family: a revision that adds the requested control while the abstract keeps its original assertion. That draws the same objection again from a reviewer who now believes the authors were not listening.

What this cluster does not cover

The methods-and-design cluster covers objections about experimental design, controls, validation, replication and model systems. Three neighbouring families have their own pages, and answering one with the other’s advice is a recognisable error.

Statistical objections — sample size, the unit of analysis, multiple comparisons, choice of test, effect size — belong to responding to statistical reviewer comments. They overlap this cluster at exactly one point, the experimental unit, which is a design decision with statistical consequences.

Novelty and scope objections — incremental, already known, out of scope — belong to reviewer objections about novelty. No methods work answers them, which is why sorting them out early matters.

Decisions rather than comments — desk rejection, rejection after review, appeals — belong to why papers get rejected. A methods objection inside a rejection letter is a different decision from the same objection inside a major revision, because in the first case you choose the next journal and in the second you are already negotiating with this one.

Before you resubmit

Whichever route you take, check that the claims moved with the evidence. A revision that adds the requested control while the abstract keeps its original assertion draws the same objection again — see the overclaim check.

If you have not submitted yet, in vivo experimental rigour and model-to-human relevance cover most of this territory pre-emptively, and they report which specific conclusion each gap prevents rather than only that something is missing.

PerfectPaper reads the claims in the title, abstract and discussion against the figures and methods that support them, and reports each methods gap as a reporting gap or a design gap — naming the sentence that would need to change in the first case and the conclusion that cannot be drawn in the second.

Review my manuscript

Frequently asked questions

How do I know whether a methods objection needs new experiments?

Ask whether a perfect methods section would have prevented the comment. If yes, it is a reporting gap and the fix is text. If no, the design cannot support the claim, and you choose between new data, equivalent evidence, and a narrower claim.

How should I respond to reviewer comments about my methods?

Sort every comment into reporting, design or scope before planning anything, and note the manuscript sentence each one lands on. Answer reporting gaps with text and a supplementary panel, answer design gaps by running the experiment, supplying equivalent evidence or narrowing the claim, and answer every comment explicitly including the ones you decline.

Which methods objections cannot be fixed by rewriting?

Objections landing on a decision made before data collection. The experimental unit, sample size justification, the randomisation method, control of confounders, the stages at which blinding was applied and the primary outcome — ARRIVE 2.0 sub-items 1b, 2b, 4a, 4b, 5 and 6b — cannot be satisfied retrospectively, so the honest response is a stated limitation and a narrower claim.

Is narrowing the claim an acceptable response to a reviewer?

Yes, and it is often the strongest one. A smaller claim the data clearly support is more publishable than a large one under dispute, and reviewers read a considered narrowing as competence rather than retreat.

Should I run every experiment a reviewer requests?

No. Requests vary in how load-bearing they are. Address each explicitly, do the ones that change what can be concluded, and explain why the others are unnecessary or disproportionate. Silence on any of them is what causes trouble.

What do I write when the experiment the reviewer wants is impossible?

State the constraint factually, say what the experiment would have established, offer the closest available substitute, and narrow the claim until the missing evidence is no longer load-bearing. Ask the editor to weigh proportionality before the deadline rather than after it.

How do I answer a methods objection I think is mistaken?

Answer it once, factually, with the analysis or citation that settles the point, and put something alongside the disagreement — the alternative analysis in supplementary material, or the underlying data. Reasoned disagreement on a methods point is normal and editors expect it; a comment left unanswered is what draws a second round.