Custom AI reviewer agents for research papers

A custom reviewer agent is a set of instructions you write, or paste, that tells PerfectPaper what to look for in your manuscript beyond its standing specialist review. You can add up to three to any review. Each takes a name, instructions up to 6,000 characters, a work type, a skill level, and optional tools: web search, scholar search, code execution and vision.

PerfectPaper’s standing team of sixteen specialist agents covers what every paper needs — structure, language, numeric integrity, figures, tables, citations, prior work, logic, methodology, statistics, causal inference, journal fit. Custom agents exist for what only your subfield needs, and every field has conventions that a generalist reviewer does not know to check.

What a custom agent adds that the standing team does not

A custom agent adds a convention, not a capability. The sixteen standing agents already read the whole manuscript for the failures that cross every field: an effect size that contradicts its own confidence interval, a figure panel that does not show what the legend claims, a reference that does not support the sentence citing it. What they do not carry is your subfield’s private rulebook, and that rulebook is where papers actually stall.

Three examples of the gap, each drawn from a prebuilt brief on this site. A generalist statistics reviewer will check whether a t-test was appropriate for two groups; it will not check whether the n of 120 in your figure legend counts cells drawn from eight mice, which is the pseudoreplication error that inflates significance by treating correlated measurements as independent observations, and which is often visible in the reported degrees of freedom. The generalist check and the subfield check disagree on the fix, too: a generalist flags “n is not independent” and stops there, while a reviewer who knows the convention names the remedy — collapse to one value per animal before testing, or refit with animal (not cell, not well) as the random-effects grouping variable, rather than collecting more cells from the same eight animals. The qualifier matters as much as the remedy, and it is the part a generalist reliably omits: either re-analysis recovers the correct n only where treatment was assigned at the animal level to begin with. A design that treated one animal and measured a hundred of its cells is not rescued by any model fitted afterwards, which is why the check belongs before submission rather than in the response letter. A generalist methods reviewer will note that a genome-wide analysis reported p-values; it will not check whether the enrichment test ran against the right background universe — the same nominally significant term can lose significance once the background is restricted from “every annotated gene” to “every gene the assay could actually detect,” which is a convention with no equivalent outside omics. A generalist reviewer will read your data availability statement as prose; it will not ask whether a GEO, SRA, dbGaP or MetaboLights accession is present, whether the genome build is named, or whether tool versions are stated — the checks that decide whether a paper is returned before review rather than after it.

The pattern in all three is the same. The generalist knows the principle and does not know the local form the principle takes. A custom agent supplies the local form.

Common questions about what a custom agent catches

Why would I need a custom agent if PerfectPaper already reviews everything?

The standing sixteen agents check what applies to every paper: structure, language, figures, citations, statistics, methodology. They do not carry your subfield’s private conventions — what counts as an independent n in a mouse study, which accession database a dataset needs, what background list an enrichment test should use. A custom agent supplies that local knowledge; the standing team supplies everything else.

My last reviewer wrote that my n wasn’t independent — can an agent actually catch that before I submit?

Yes, if you tell it what to look for. A generalist check confirms a t-test suits two groups; it does not know that 120 cells drawn from eight mice is pseudoreplication unless instructed to compare the stated n against the reported degrees of freedom and flag correlated measurements reported as independent — exactly the check in PerfectPaper’s pseudoreplication brief.

Can I get an AI reviewer to check something as specific as whether I named a GEO accession?

Yes — that is precisely the kind of check a custom agent exists for. A generalist reviewer reads a data availability statement as prose and stops there. An agent given the data-availability brief checks for a GEO, SRA, dbGaP or MetaboLights accession, a named genome build and stated tool versions, the specifics that decide whether a submission is returned before review.

Prebuilt agent briefs by field

PerfectPaper publishes twenty-three prebuilt agent briefs: five for each of four research areas, plus three that suit any field. Each is a plain-text brief you copy into an agent slot unchanged, and each states on its own page what the standing team already covers so a slot is never spent twice.

Genomics, epigenetics and metabolism — multiple testing and enrichment, data availability and accessions, batch effects, metabolomics reporting, variant nomenclature. The batch effects brief is the one that catches batch confounded with condition, and technical replicates presented as biological. The distinction it draws matters: a batch effect that is merely correlated with condition can sometimes be adjusted for statistically, but a batch that is fully confounded with condition — every treated sample run on one day, every control on another — cannot be recovered by any correction method after the fact, because the design has made batch and treatment the same variable. The brief is written to tell the two apart rather than to recommend a correction blind to which one occurred.

Preclinical cancer research — pseudoreplication and unit of analysis, in vivo rigour, flow cytometry reporting, image quantification, model relevance. The image quantification brief exists because a representative micrograph and a quantified one are different claims, and a manuscript can present the first as though it were the second without ever saying so.

Clinical trials and experimental therapeutics — response criteria, early-phase safety, registration and endpoints, synergy claims, translational dose. The registration and endpoints brief is a verification task with web search, because it opens the registry record rather than checking the manuscript against itself.

Epidemiology and population science — health equity reporting, competing risks, registry data validity, causal language, qualitative rigour. The causal language brief is the most reusable of the four sets, because the distance between an association estimated and an effect claimed is a defect in every observational literature; it reads the verb in every results sentence and flags the jump from “associated with” to “increases the risk of” or “leads to” wherever the design cannot support it. The competing risks brief checks a narrower and more specific thing: whether a cause-specific analysis (a Cox model censoring on the competing event) has been mistaken for a subdistribution analysis (a Fine-Gray model on cumulative incidence), because the two answer different questions from the same data — the cause-specific hazard is the aetiological quantity, asking whether an exposure influences the biological process, while the subdistribution hazard is the prognostic one, asking what proportion of this population will actually experience the event by a stated time — and reviewers routinely ask for whichever one the manuscript did not run.

Agents that suit any field

Three prebuilt briefs are field-independent, and they are the ones worth reaching for when your paper’s specific risk is not covered above.

Ethics, consent and data sharing — approval statements, consent scope for data sharing, de-identification, data management plan compliance. Consent scope is the item worth checking first: a consent form that permitted analysis does not automatically permit deposition into a controlled-access repository, and the mismatch surfaces at the point of publication rather than at the point of analysis.

Overclaim check — the distance between what was shown and what the discussion says, tuned to translational overreach. The check operates on pairs: a result sentence and the discussion sentence that reports it, read together, with the verb compared. The verbs form a ladder, not a binary: “was associated with” and “correlated with” sit at one end, “was independently associated with” adds a claim about the named covariates, “predicted” quietly switches to a discrimination claim that an explanatory coefficient does not establish, “was linked to” carries no defined statistical meaning at all, and “caused,” “drove” or “led to” require a design — randomisation, an instrumental variable, a validated mediation analysis — that most single-cohort papers do not have. The brief flags a rung higher than the methods earn, not the use of any particular word.

Study section lens — reads the manuscript the way a grant reviewer scoring significance and innovation would, which is a different job from peer review. A referee asks whether the work is sound; a study section asks whether it matters enough to fund, and papers written for the first question routinely answer the second badly — a discussion section that defends every methodological choice in detail but never states, in one sentence, what changes about the field if the hypothesis is confirmed.

Choosing which three to run

Choose the three agents by asking what would return your paper, what would sink it in review, and what your last reviewer already objected to — in that order. The three slots have different jobs, and filling all three with the same kind of check wastes two of them.

The first slot belongs to whatever prevents a return without review, because that failure is cheap to fix and expensive to incur. For an omics submission that is data availability and accessions. For a trial report it is registration and endpoints. For a paper going to a journal with a strict reporting standard it is the checklist the journal names.

The second slot belongs to the design or analysis defect your study type is most prone to. For a preclinical paper with per-animal treatment and per-cell measurement, that is unit of analysis. For a time-to-event analysis where patients can die of something else, it is competing risks. For an analysis built on a claims or registry extract, it is registry data limitations.

The third slot belongs to you. Write it for the convention a referee has already objected to on one of your papers, because that objection will come again, and an agent written against a real review comment outperforms any prebuilt brief on your own work. If a reviewer has previously asked what your n represents — the objection catalogued at reviewer says my n is not independent — that is the agent to write.

Writing your own

The prebuilt briefs are complete and can be pasted unchanged, but the most valuable agent is usually one you write for a convention you have been burned by before. How to write a custom reviewer agent covers the structure that works, the instructions worth including, and the ones that make an agent worse.

Two drafting aids sit next to the instruction field. Write at least 50 characters of rough notes and Expand into a fuller brief develops them into a structured brief, preserving your intent and inventing no requirements; one undo restores your own words exactly, because a button that can silently eat what you wrote is not a drafting aid. If your brief already exists as a file, Supplement instructions appends a plain-text file — .txt only, UTF-8, 8 KB combined with the typed field.

PerfectPaper can also propose an agent for your specific paper from its title and abstract, which you can then edit before running. It proposes one agent rather than a menu, and it is told what the sixteen standing agents already cover, so the proposal is a gap rather than a duplicate. Ask again and it excludes the names it has already offered.

Questions about writing and adding a custom agent

How do I add a custom AI reviewer agent to a review?

Open an agent slot when you set up the review, name the reviewer, and paste or write its instructions. Work type, skill level, tools and review depth all have defaults, so instructions plus a name is enough to run one. A prebuilt brief from this site can go in unchanged, with no editing required.

How many custom AI reviewer agents can I add to one review?

Three per review, and that cap does not flex for a longer or more complex manuscript. They run alongside the sixteen built-in specialists rather than replacing any of them, and each agent keeps its own name, instructions, work type, skill level and tool selection independent of the other two, so one agent’s setup never leaks into another’s.

Is there an actual character limit on the instructions, or is that a soft cap?

It is a hard limit: 6,000 characters typed, with a further optional plain-text file appended — .txt only, UTF-8, 8 KB combined with the typed field. Every prebuilt brief on this site fits comfortably inside the typed limit alone, leaving room to add your own items. The agent’s name field runs to 80 characters.

Can I just paste in my lab’s old reviewer checklist as the instructions?

Yes, and for many labs that is the fastest path to a working agent. A checklist written as flat prose works, but a brief written as discrete, checkable determinations — one per line, each answerable from the manuscript — returns more locatable findings than the same content written as a paragraph of general guidance.

The settings, and what each one actually does

Four settings sit under the instructions, and three of them are suggestions rather than commands.

Work type picks from ten values: general_review, analysis, editing, verification, domain_review, statistics_methods, causal_inference, citation_verification, literature_research, figures_tables. Leaving it on “Let PerfectPaper decide” is a reasonable default; the value you choose is a hint that the routing may override if the brief plainly describes a different job.

Skill level picks from three: generalist, graduate, expert. Choose expert for anything requiring specialist knowledge, which is nearly every brief worth writing.

Tools are four checkboxes, and a selection adds to what PerfectPaper already routes for that work type rather than replacing it. Web search looks up sources and claims on the open web. Scholar search queries the scholarly record for cited works. Code execution recomputes numbers and checks arithmetic — it is what lets an agent recompute a percentage from the manuscript’s own printed numerator and denominator instead of judging arithmetic in prose, though it only re-derives what the manuscript already reports and cannot detect a fabricated raw number that was never printed anywhere for it to recompute against. Vision inspects the paper’s figures as images, which is what a flow cytometry gating check needs when the gating strategy is a supplementary panel, though a gate drawn across a low-contrast or heavily overlapping channel is exactly the case where a specialist’s own eye still outperforms an automated read, so a borderline vision finding is worth a manual look rather than an automatic accept. Give tools only where the task needs them: a checking task that reads only the manuscript is usually more reliable without them, because a tool that is available is a tool that gets used.

Review depth is a real fork rather than a hint. Standard review reads the paper and streams its findings alongside the main review team. Deep research browses the literature in the background and merges in later, and adds anywhere from 10 minutes to 3 hours to the review. Choose deep research only when the finding genuinely depends on evidence outside your manuscript — a reported human pharmacokinetic parameter for a named compound, for a translational dose check — and standard for everything else.

Questions about tools and review depth

Should I just turn on every tool so the agent doesn’t miss anything?

No — an available tool gets used whether the task needs it or not, and an agent that reaches for web search on a question the manuscript already answers introduces an outside source where the internal evidence was enough. Turn on only what the determination requires: code execution for arithmetic, vision for a figure-only check, web or scholar search for a claim only the outside world can settle.

What’s actually different between standard review and deep research?

Standard review reads only your manuscript and reports alongside the main team, within the normal review window. Deep research browses the literature in the background and merges its findings in later, adding roughly 10 minutes to 3 hours. Reserve it for a determination that genuinely needs an outside source — a compound’s reported human dose, say — rather than defaulting to it out of caution.

Ground truths: the field that stops advice you cannot act on

Ground truths is an optional field of up to 2,000 characters holding what the agent must treat as already settled about your study. Examples: the study is complete; no new participants can be recruited; the sample size is fixed; the assay cannot be repeated on the remaining sample.

The field exists because the most common form of useless review advice is a remedy that cannot be executed. “Increase the sample size” is not a revision instruction for a completed cohort study, and “run the orthogonal validation” is not one when the tissue is exhausted. An agent that knows the constraint spends its attention on remedies that fit inside it — a re-analysis, a bounded sensitivity analysis, a narrowed claim, a limitation stated with its direction.

Ground truths are handled as facts rather than as instructions, and are labelled that way where the agent reads them. That distinction matters: an author can constrain what the agent believes about the study without being able to constrain what the agent is willing to report. “The sample size is fixed” stops the agent from recommending a bigger cohort, but it does not stop the agent from noting that the fixed n limits what the study can conclude — a ground truth narrows the remedy an agent proposes, never the finding it is willing to raise. Leave the field blank if nothing is settled yet, which is the right answer while the experiments are still running.

What a custom agent can see

A custom agent reads the same context the standing team reads, and this is the part most authors do not expect. Alongside the manuscript, it receives your answers from the pre-review interview, a profile of your own writing habits synthesised from your prior publications and past PerfectPaper feedback, your target journal, and the study design your paper reports.

The interview answers are authoritative for intended scope and for execution detail no reader could recover from the page — why a control was omitted, what the pilot showed, which analysis was pre-specified. They never override arithmetic, quoted text, figures, tables or verified external evidence. The author profile describes the submitting author alone and is never attributed to co-authors, and a recurring-weakness note in it is prior review feedback rather than an established fact about anyone’s record.

The practical consequence for your brief: do not spend characters restating things the agent already has. A brief that says “the target journal is a general medical journal” is spending its budget on a field the agent was handed. Spend it instead on the determination you want made and the failure mode you want prevented.

How a custom agent’s findings are checked before you see them

Every finding a custom agent produces goes through the same adversarial verification pass as every finding from the standing team. The verifier receives the candidate findings and their grounding, tries to falsify each one, omits the false positives, and calibrates severity and priority once across the survivors.

Where possible, that verification runs on a different model family from the one that produced the finding, and the review records whether the check was independent or same-family. This matters more for custom agents than for built-in ones, because a brief you wrote yesterday has had no calibration against thousands of manuscripts, and the characteristic failure of a review agent is not missing a problem — it is inventing one. A same-family check still catches an unsupported finding, but it is the cross-family check that catches the failure mode both models share by training rather than by task — which is the reason the record distinguishes the two rather than reporting “verified” as a single undifferentiated state. The verification pass is the structural answer to invention, and it is why a brief’s negative instructions (“do not infer values that are not present”; “do not attempt to validate that a specific accession exists in any database”) are worth more than an extra check.

Your instructions also cannot displace the review contract. PerfectPaper supplies the surrounding format and reporting rules, so a brief describes what to look for rather than how findings are reported, and text pasted into the field cannot rewrite the output shape.

Why three

Three slots is a deliberate limit rather than a tier. An agent asked to check everything checks nothing in particular, and the briefs that produce useful findings are narrow ones. Three narrow agents return findings you can act on where one broad agent returns a summary, and the constraint pushes toward the narrow form.

The mechanism is attention, and it is visible in the output. A brief that names one determination — “for each reported comparison, determine whether the stated n counts animals or cells” — returns findings with locations and numbers. A brief that names six areas returns a paragraph of impressions on each, and impressions are what you could have written yourself. Attention is finite, and an unordered list of six topics gives the agent no basis for allocating it.

The cap also protects the review from a subtler failure. Three agents whose briefs overlap produce three versions of the same finding, which reads as corroboration and is not. Narrow, disjoint briefs make each finding independently traceable to the check that produced it.

When a custom agent is the wrong tool

A custom agent is the wrong tool for anything the standing team already covers, and for anything that is not a determination about your manuscript.

Do not spend a slot on grammar, formatting or citation style: four standing agents already read for those, and a specialist agent asked to also check writing quality spends its attention where it adds nothing. Do not spend one on a general request — “check the manuscript for problems” returns the observations you could have made yourself. Do not spend one on a claim about the outside world that no tool can settle; an agent asked to confirm that a specific accession exists in a public database will sometimes report a false absence, which is worse than not checking.

Two adjacent jobs belong elsewhere. If what you want is the review itself rather than a supplement to it, start at AI peer review, which describes what the sixteen standing agents do without any brief from you. If what you want is a general-purpose prompt to run somewhere else, how to write a custom reviewer agent is written to be portable and does not assume PerfectPaper.

Frequently asked questions

Will a custom AI reviewer agent just duplicate the built-in review and tell me the same things twice?

A brief aimed at general quality can, which is why every prebuilt brief on this site states what the standing team already covers. The best use of a slot is a convention specific to your assay, design or field — the sharpest agent is usually the one aimed at an objection a referee has already raised on your own work, not a restatement of what a careful reader would notice.

If I tell the agent to ignore something, will it just stay quiet about it?

Your instructions can narrow what an agent looks for, but they cannot rewrite PerfectPaper’s review contract or suppress a finding another part of the review already surfaced. A brief describes a determination to make, not a gag order on the rest of the report, and a ground truth changes what remedy an agent proposes rather than what it is willing to raise.

Can an AI reviewer actually tell if my data availability statement is missing an accession number, or does it just skim for the word “accession”?

The data-availability brief checks for the specific prefix each repository stamps on its own records — GSE for a GEO series, an SRA or ENA run, a dbGaP phs study ID with its version suffix, a MetaboLights identifier — not the presence of the word “accession” itself, which is why a statement reading “data are available upon reasonable request” still gets flagged even though it mentions availability.

Related

How to write a custom reviewer agent · AI peer review · Manuscripts are not used to train models

PerfectPaper runs up to three author-written reviewer agents alongside its sixteen standing specialists, verifies every finding they produce against an adversarial pass before you see it, and returns each one with its location, its consequence and a specific remedy.

Review my manuscript