A procurement checklist for AI tools that handle unpublished research

A procurement checklist for AI tools comes down to sixteen questions a research office, IT security team, or librarian should ask any service that will receive unpublished manuscripts, grant applications, or other pre-publication research. The questions are vendor-neutral and grouped into five areas: training and retention, subprocessors and data flow, residency and encryption, certification and contracts, and deletion and exit.

A vendor that answers all sixteen in public is easier to evaluate than one that answers them only under an agreement, and reluctance to answer a specific question is itself informative. The sixteen are written so that each has a checkable answer — a number, a named company, a region, a date, a document — rather than a posture. An answer you cannot write into a sentence with a unit attached has not been given.

How to run the checklist

Run the checklist by sending all sixteen questions in one written message, asking for written answers, and requiring each answer to name the document it comes from. Written answers are the point: a verbal assurance in a demonstration call cannot be attached to a file, cannot be re-read by the colleague who inherits the assessment, and cannot be compared against the same vendor’s answer eighteen months later.

Three procedural details decide whether the exercise produces anything usable. First, name the account your institution will actually hold, because several of the answers change with it. Second, ask which of the four surfaces each answer lives on — a marketing page, a public trust or security page, a signed data processing agreement, or a support reply — since only the third carries a counterparty and a remedy. Third, record the date you asked, because a vendor’s answer describes a deployment on a day rather than a permanent property.

Institutions that already run a general questionnaire — the Higher Education Community Vendor Assessment Toolkit maintained by EDUCAUSE, or the Cloud Security Alliance’s Consensus Assessments Initiative Questionnaire — should treat this checklist as the pre-publication overlay rather than a replacement. Those instruments assess a cloud service. These sixteen assess what happens to an unpublished manuscript, which is a document with no priority claim, no publication date and no public record to fall back on.

Common questions about vetting an AI vendor

I’m on our research IT team and we’ve been asked to sign off on an AI proofreading tool — what should I actually send the vendor?

Send all sixteen questions in one written message rather than raising them piecemeal across a demo and a follow-up email. Start with the four that reverse the most provisional approvals: what each subprocessor receives, whether residency covers data already uploaded, whether deletion reaches backups, and the account-tier clause hidden inside the training question. Require every answer to name the document it comes from.

Is a vendor demo call enough to approve them, or do we need something in writing?

A demo call is not enough on its own. A verbal answer cannot be filed, cannot be re-read by the colleague who inherits the assessment next year, and cannot be compared against the same vendor’s answer eighteen months later once the product or its subprocessor list has changed. Get written replies to all sixteen questions before the call, and treat the call as clarification, not as evidence.

Training and retention

1. Is customer content used to train or fine-tune models, and does the answer differ by account tier? Training defaults are frequently set per tier rather than per company, so a company-level answer is not an answer.

Ask for the default that applies to the specific tier your institution will hold, whether that default can be changed, and by whom — an individual account holder or a workspace administrator. A common way to get this wrong is to inherit a colleague’s accurate statement about a different account type; the same vendor routinely applies one default to a consumer login and another to an API or enterprise agreement. Is it safe to upload a manuscript to ChatGPT works through that distinction with the published terms.

A “we do not train” answer is also narrower than it sounds if it excludes a separate feedback or product-improvement pipeline. Some vendors route a user’s thumbs-up or thumbs-down rating, or a flagged output, into a review process distinct from the one the training answer describes — ask whether the no-training promise covers every mechanism that can feed customer content back into model behaviour, not pretraining alone.

Extend the question to derived artefacts in the same sentence, because most vendors answer only about the document. Embeddings computed from your abstract, extracted metadata, the model’s own reasoning trace and the model’s reply are each separable from the manuscript and each can be governed differently.

2. Is content retained after deletion, and for how long? Retention and training are separate commitments. A service may decline to train on content while retaining it for abuse monitoring, support, or legal obligations.

There are three clocks, and a good answer gives all three in days: how long content lives while the account is active, how long a copy survives after the user deletes it, and how long anything survives after the contract ends. Abuse-monitoring copies usually run on a different clock from the user-facing one, and the second frequently outlives the first. Read the qualifying clause as carefully as the number — “unless longer retention is required by law” means the published window is a floor on exposure rather than a ceiling.

A fourth clock is easy to miss: access logs and audit trails. A vendor can truthfully say the manuscript itself is gone while a record of who opened it, when, and from where persists for years under a security-retention policy that the content-deletion answer never mentions — ask about that clock by name, since “content” and “logs about content” are not the same commitment.

3. May staff read customer content, and under what circumstances? Most providers reserve some human review for safety or abuse investigation. The question is what triggers it and who is permitted.

Ask for four specifics: the trigger, the role, whether the access is logged, and whether the customer is told. “Access is limited to authorised personnel” names none of them. A defensible answer looks like: access occurs on an automated classifier hit or a support ticket the customer opened, is limited to a named on-call role, is written to an audit log, and is disclosed on request.

A separate category is easy to miss because it is routine rather than exceptional: human annotators grading model output for quality, unrelated to any abuse investigation. A vendor can answer the abuse-trigger question honestly while leaving a quality-review programme unmentioned — ask whether one exists as its own line item, with its own trigger and role.

4. Is content used to build evaluation or benchmark datasets? This is a separate question from training and is the one most often omitted. Ask it explicitly.

The reason it is omitted is structural rather than evasive. Keeping an interesting manuscript as a regression fixture updates no model weights, so a team that does it can go on saying truthfully that it does not train on customer data, while the document sits in a test corpus indefinitely. Ask whether the evaluation corpus contains customer content, and if it does, what the removal path is. How the no-training guarantee is enforced sets out what an answer to this question looks like when the corpus is synthetic.

A corpus’s current composition also does not describe its history. Ask whether any customer content entered the corpus before a synthetic-only policy took effect, and whether it was subsequently removed or simply stopped growing from that point — “the corpus is synthetic” and “the corpus has always been synthetic” are different claims.

A good answer names the account tier it applies to and states retention in days rather than describing a philosophy.

Questions about AI training and manuscripts

Will an AI writing tool train on my unpublished paper?

Ask the vendor directly, because the honest answer usually depends on account tier rather than on a single company-wide policy. A consumer login and an enterprise or institutional agreement from the same company frequently carry different training defaults, and a change a workspace administrator makes to one does not automatically apply to the other. Get the default in writing for the specific tier your institution will actually hold.

If a company says it doesn’t train on customer data, does that cover everything?

No — training is a narrower promise than it sounds. A “we do not train” answer typically covers model-weight updates only, not keeping a document as a regression-test fixture, not the embeddings computed from it, and not a support agent reading it during a ticket. Ask separately about evaluation corpora, derived artefacts and human review, since each can be governed differently under the same training policy.

What actually happens to my manuscript after I delete it from an AI tool?

Deleting a file from the interface usually removes the row a user sees, not every copy that exists. Ask for three numbers: how long content persists while the account stays active, how long a copy survives after deletion, and how long backups retain it once the contract ends. A backup snapshot commonly ages out on its own fixed rotation, independent of when the primary copy was removed.

Subprocessors and data flow

5. Which third parties receive customer content, named individually? Name the legal entity, not the category. “Our cloud providers” and “leading model vendors” are not a list, and cannot be copied into a record of processing.

6. What does each receive — the full document, or metadata only? This is the single most load-bearing question in the sixteen. A service that receives a title and an abstract to compute an embedding is a different exposure from one that receives every paragraph, table and figure of an unpublished paper, and a flat list of company names renders the two identical.

A subprocessor list also has a horizon problem. The company you can name is the vendor’s direct subprocessor; that company may itself run on infrastructure or models supplied by a fourth party the vendor’s own disclosure never reaches. Ask whether the list is audited one layer down, or only assembled from the vendor’s own direct contracts.

7. Where does each process it? Per recipient, not per vendor. A vendor with one company in the EU and three in the United States has a United States answer for three of your documents’ journeys.

8. How are customers notified when a subprocessor changes? European law already specifies the shape of this answer. Article 28(2) of Regulation (EU) 2016/679 states that “the processor shall not engage another processor without prior specific or general written authorisation of the controller. In the case of general written authorisation, the processor shall inform the controller of any intended changes concerning the addition or replacement of other processors, thereby giving the controller the opportunity to object to such changes.” Ask for the notice period in days, the mechanism — a dated changelog, a mailing list, an in-product notice — and what happens when you object.

Article 28(2) actually describes two regimes, and most contracts default to the weaker one. “Specific” authorisation means the vendor must ask before adding each new subprocessor; “general” authorisation — the model almost every SaaS agreement actually uses — only requires advance notice and a right to object after the fact. Ask which regime governs your agreement, because “we will tell you” and “we will ask you first” are not the same commitment.

A good answer is a published list that distinguishes what each processor receives. A flat list of company names hides the difference between a service that sees a title and one that sees the whole manuscript, and that difference is usually the most important fact in the assessment. Counting subprocessors is the wrong measurement: four services that read a manuscript and eleven that handle payments, email and diagnostics is a fifteen-row table and a four-row exposure.

Residency and encryption

9. Where are documents stored, and where does inference run? These are two questions and vendors frequently answer only the first. A tool can store in the EU while processing elsewhere.

Add a third clause that almost nobody asks: is the inference region read from the running configuration, or typed into the page by an editor? A hand-typed region is accurate until a deployment changes and nobody updates the copy. Where your manuscript is processed sets out a six-stage route in which residency is a separate answer at every stage, which is the shape most vendor pages collapse into one sentence.

A third state sits between the two: data in use, meaning the plaintext a running process holds in memory while it works on the document. Encryption at rest and encryption in transit say nothing about this state, since neither protects data while it is being actively computed on. A small number of providers now offer confidential computing — hardware-isolated memory the cloud operator itself cannot read — but it remains uncommon enough in this market that the honest default answer is that inference runs on decrypted content in ordinary memory, protected by access controls rather than by encryption.

10. Is data encrypted at rest and in transit, and are keys provider-managed or customer-managed? Encryption at rest is the least discriminating phrase in vendor security copy, because it is close to universal among cloud services. Key custody is the discriminating half: a provider-managed key means the storage provider can decrypt, a customer-managed key means your institution holds the revocation lever.

Ask one further question that is quick to answer and occasionally finds something: what does the storage key contain? An object path built from the uploaded filename puts your paper’s working title into any bucket listing, which is a disclosure that survives every encryption control above it.

11. Does a residency setting apply to existing data or only to new data? Residency changes often apply going forward, leaving an older estate where it was.

Ask for a date rather than a policy. “Documents uploaded before 27 August 2026 remain in their original region” is an answer an assessor can act on; “we have migrated to EU storage” is not, because it does not say whether the verb covered the existing objects. Then ask the fallback question: when the requested region is unavailable, does the request fail, or does it succeed somewhere else — and is that recorded? A vendor whose fallback is silent cannot tell you afterwards which documents crossed a border.

Residency and sovereignty are also not the same guarantee, and the two are frequently conflated in vendor copy. Residency states where a server physically sits; sovereignty asks which country’s courts and law-enforcement orders can compel disclosure regardless of the server’s location — a question that turns on where the company is incorporated, not where its data centre is built. A residency claim answers the first question; ask separately whether it was meant to answer the second.

Questions about where manuscript data lives

Does “encrypted at rest” actually mean my paper is safe?

Not on its own. Encryption at rest protects a stolen disk or an unauthorised database query; it does nothing once the application itself decrypts the document to run a review, the state security engineers call data in use. The more discriminating question is who holds the decryption key — a provider-managed key means the storage vendor itself can read the file if compelled or compromised.

If my university requires EU-only storage, does that cover the paper I uploaded last year?

Usually not automatically. A residency commitment typically applies going forward from the date it changed, leaving an existing estate exactly where it was unless the vendor states a migration date. Ask for the cutoff date in writing — “documents uploaded before [date] remain in their original region” is answerable; “we have moved to EU storage” is not, because it does not say which files the verb covers.

What’s the difference between where my file is stored and where it’s actually processed?

Storage and inference are two separate locations, and a vendor that answers only the first has answered half the question. A document can sit on a server in Frankfurt while the model reading it runs somewhere else entirely. Ask both, ask whether the answer is read from live infrastructure configuration or typed by hand into a page, and ask what happens when the requested processing region is unavailable.

Certification and contracts

12. What third-party attestations exist, and can a report be requested? Four follow-ups turn this from a badge question into an assessment: which report, what period it covers, which systems were in scope, and whether it can be shared under a non-disclosure agreement. A report whose scope excludes the production inference path answers a question you did not ask.

A certification and an attestation report are not the same instrument, though vendors use the words interchangeably. ISO/IEC 27001 is a certification: an accredited body audits the vendor’s information security management system against a fixed public standard and issues a certificate with a scope statement and an expiry date. Many other attestation reports are bespoke — the vendor selects which of its own controls to be tested against — so “independently assessed” can describe two very different levels of external scrutiny.

13. Is a data processing agreement available? Article 28(3)(h) of Regulation (EU) 2016/679 requires the processor to make available “all information necessary to demonstrate compliance with the obligations laid down in this Article and allow for and contribute to audits, including inspections, conducted by the controller or another auditor mandated by the controller.” Ask whether that clause is present unmodified, and read what the agreement says about onward transfers and about the no-training commitment, since a promise repeated inside the agreement is enforceable and the same promise on a web page is editable.

For a transfer out of the European Economic Area, ask specifically which mechanism the agreement relies on — the European Commission’s Standard Contractual Clauses, an adequacy decision for the destination country, or a supplementary measure layered on top of either. “We comply with data protection law” does not name one, and a transfer mechanism can be challenged or invalidated independently of the rest of the contract.

14. Is a business associate agreement available if protected health information is in scope? Its absence is a hard stop only when the manuscript itself carries protected health information; a methods paper reporting de-identified findings from a clinical dataset does not necessarily require one, though the underlying dataset the paper describes might.

A vendor that says plainly which certifications it does not hold is giving you more usable information than one that lists frameworks it is aligned with. Alignment is not attestation, and the difference is the whole point of the question. “Aligned with”, “built to the principles of”, “mapped to” and “readiness programme” all mean that no auditor has issued an opinion, and a vendor using one of those phrases has told you the answer if you read it literally.

Deletion and exit

15. Can a user delete content themselves, and does deletion propagate to backups and subprocessors? Deletion has three surfaces and vendors typically answer for one. Ask about the primary store, the backup estate, and every subprocessor that received the document. Ask whether the user-facing action is a soft delete that hides a row or a hard delete that removes it, and ask for the backup expiry window in days, because that window is the real answer to “how long does my paper exist after I delete it”.

16. Can an institution export its data if it stops using the service? Article 28(3)(g) of Regulation (EU) 2016/679 requires the processor, “at the choice of the controller”, to delete or return “all the personal data to the controller after the end of the provision of services relating to processing” and to delete existing copies unless law requires storage. Ask which of the two the vendor performs by default, in what format the return arrives, and whether the export includes the derived material — the reviews, comments and version history — or only the files you originally uploaded.

A good answer includes a self-service path rather than a support ticket. A deletion process that depends on vendor goodwill is not a control.

What a non-answer looks like

Non-answers to this checklist arrive in five recognisable shapes, and naming them shortens the second round of correspondence considerably.

The philosophy answer. “We take the security of your research extremely seriously and treat customer data as confidential.” Reply by restating the question with the unit attached: how many days.

The company-level answer to a per-tier question. “We do not train on customer data” in response to question 1. Reply by naming the tier and asking for the default that applies to it.

The alignment answer. “Aligned with industry-standard frameworks.” Reply by asking which auditor issued which opinion covering which period, and accept “none” as a complete answer.

The deferral. “Happy to cover that on a call with our security team.” Reply that the answers are needed in writing for the assessment file, and that a call is welcome afterwards.

The substituted question. Answering a processing-residency question with a storage-residency fact is a frequent substitution in this subject, and it is often unintentional. Reply by asking the two halves in separate numbered sentences so that one answer cannot cover both.

None of these is proof of a problem. A small vendor with no compliance function frequently gives shape-one answers to questions it could answer well. Treat them as a signal to ask again more narrowly, not as a verdict.

Which answers change an approval decision

Four of the sixteen questions do most of the work in practice, and an assessment under time pressure should start with those four.

Question 6, what each subprocessor receives. The answer changes the exposure by an order of magnitude and is the one a flat vendor list conceals. Nothing else in the sixteen moves the risk estimate as far.

Question 11, existing data versus new data. Residency answers are usually forward-looking, which means an approval granted on a residency claim can leave the institution’s existing corpus exactly where it was. This is the question that most often reverses a provisional yes.

Question 15, whether deletion propagates. A deletion control that stops at the primary store is a user-interface feature rather than a data protection control, and the gap only becomes visible when someone asks about backups and subprocessors in the same sentence.

The account-tier clause inside question 1. A training default that holds for an enterprise agreement and not for a consumer login produces an approval that is correct on paper and wrong in the browser, because the account a researcher signs into is often not the account the assessment described.

Questions 12 and 14 are frequently asked first and are among the least predictive of the sixteen, because a vendor holding no attestation report can still be fully assessable on the properties that actually govern your data.

Frequently asked questions about the approval decision

Which of these sixteen questions actually matters most?

Four do most of the work: what each subprocessor receives, whether a residency setting covers data already uploaded, whether deletion propagates to backups and subprocessors, and the account-tier clause hidden inside the training question. An assessment under time pressure should answer those four before working through the rest, because they are the ones that most often reverse a provisional approval.

Can we get our data back if we stop using the AI tool?

Ask specifically, because the two outcomes are different and a vendor may default to either one: deletion of everything, or return of the data to your institution before deletion. Ask which happens without a special request, in what file format the return arrives, and whether it includes derived material — reviews, comments, version history — or only the documents your institution originally uploaded.

What should a librarian ask before recommending an AI tool to researchers on campus?

Ask which account tier researchers will actually hold, since training and retention defaults commonly differ between a consumer login and an institutional agreement — an approval based on the institutional terms does not describe the account a researcher signs into on their own. Then ask which of four surfaces each answer lives on: a marketing page, a trust page, a signed agreement, or a support reply.

My department wants to start using an AI review tool on unpublished manuscripts, but I don’t know where to begin on the data-privacy side — what should I actually do?

Send the vendor all sixteen questions in one written message rather than asking piecemeal, and require a written reply naming the document each answer comes from. Start with the four that reverse most approvals — subprocessor exposure, whether residency covers existing files, deletion propagation, and the account-tier training default — before spending time on the certification questions that predict less.

What this checklist does not cover

The sixteen questions assess data handling, and four adjacent questions a research office also has to answer are deliberately out of scope here.

Review quality. Nothing in this checklist tells you whether the tool’s output is any good. Evaluating that means giving it a manuscript whose defects you already know and reading what comes back — the approach described in peer review software and what an AI peer review tool does.

Journal and disclosure policy. Whether a researcher may use an AI tool on a manuscript, and what they must declare, is governed by publisher policy rather than by procurement. Journal AI policies for authors and referees separates the author rulebook from the referee rulebook, and reviewers using AI on manuscripts covers the case where the manuscript belongs to someone else — a confidentiality obligation that no vendor answer can discharge.

Accessibility and procurement law. Public-sector accessibility requirements, competitive tendering thresholds and export-control classifications each have their own instruments.

The document’s own sensitivity. No vendor answer makes a document safe to upload if the document should not leave your institution at all.

Scoping the assessment to the document

Scope the assessment to what is actually being uploaded, because a de-identified manuscript and a dataset of individual-level records are not the same procurement question.

A methods and results manuscript containing no participant identifiers, headed for a journal in three months, is a confidentiality and priority problem rather than a personal-data problem. The controls that matter are the training answer, the retention window and the subprocessor list. A manuscript carrying direct or indirect participant identifiers is a personal-data problem as well, and the residency, transfer-mechanism and deletion answers all become load-bearing.

Indirect identifiers deserve their own attention, because “no names in the manuscript” is not the same claim as “not reidentifiable.” A rare-disease cohort of a handful of patients at one named clinic, described by condition, sample size and institution, can be reidentifiable from that combination alone even with every name removed — the same scoping question applies to the manuscript text, not only to any dataset filed alongside it.

Two categories deserve a firmer answer than “assess the vendor”. Individual-level genomic data held under controlled access, and identifiable clinical trial records, belong in a repository under a data access agreement rather than inside a manuscript uploaded to any third-party tool — see genomics review and clinical trial review for what belongs in the paper and what does not. The reporting-level discipline that ethics and data-sharing statements already require of authors is usually the right pre-upload standard too.

How PerfectPaper answers

PerfectPaper never uses customer content to train a model and holds no customer manuscript in its evaluation corpus, which is eight synthetic papers written for the test harness. The answer does not vary by account type, and enforcement is contractual rather than a per-request flag — a distinction stated openly rather than glossed. Manuscripts and reviews remain in your account until you delete them.

Subprocessors are named publicly with what each receives, and the distinction question 6 asks about is answered per row: the review models receive the whole manuscript, while the embedding and scholarly lookup services receive a title, an abstract, a DOI or an author name. Six services hold or read the paper — the application servers, the database, object storage, the two review-model providers and the embedding service; the remainder handle payments, email, edge delivery, analytics and scholarly lookups.

Application servers, database and document storage run in Frankfurt and the eu-central region, encrypted at rest with provider-managed rather than customer-managed keys, and processing regions are reported from live configuration rather than typed into a template — the full route is here, including the components that do not run in Europe and the date before which uploaded files remain in their original region. Where a request’s actual region differs from the region it asked for, the mismatch is recorded as an event rather than absorbed silently. Deletion is self-service, and a privacy rights path handles access, export and erasure.

PerfectPaper holds no third-party security attestation report and offers no business associate agreement. The attestation answer appears on secure AI review for unpublished research rather than being left to a meeting, alongside the rest of what PerfectPaper does not claim.

Related

Secure AI review for unpublished research · Where your manuscript is processed · AI that never trains on your research · Uploading to ChatGPT · Journal AI policies

Review my manuscript