
A reproducible search strategy preserves the exact query and search conditions so another researcher can rerun it and compare results. Saving keywords alone isn't enough: you also need the database, platform, dates, filters, and retrieved records.
Build the audit trail while you search, not before submission. For a systematic review search, start with searchable concepts and record each translation, revision, and result set.
Key Takeaways
Record search conditions and results alongside the exact executable query.
Translate database-specific syntax while preserving the underlying concept structure.
Test both result counts and study retrieval; reporting completeness alone doesn't establish reproducibility.
What Makes a Reproducible Search Strategy?
Reproducibility concerns consistent methods and results
The National Academies' definition of reproducibility involves consistent results using the same input data, computational steps, methods, code, and analytical conditions. Replicability concerns consistent findings across studies using newly collected data.
For a literature search, distinguish quantitative reproducibility, similar counts when rerunning with the original last-search-date limit, from content reproducibility, retrieving almost all originally included studies. Neither measure alone establishes search quality.
Published searches often lack essential details
In Rethlefsen and colleagues' reproducibility study, only 1% of 100 sampled systematic reviews were fully reproducible across all database searches. In 91%, reporting didn't identify the database or platform for every search.
Across 453 database searches, 47.2% named both database and platform, 22.1% fully reported limits and restrictions, and 22.7% supplied exact search dates. Only 4.9% reported all six assessed PRISMA-S items.
The study classified a search as reproducible when it met those six items, ran without editing, and returned results within 10% of the original count. Only 47 searches, 10.4%, met that tolerance; three matched exactly.
1. Turn the Research Question Into Searchable Concepts

Choose a framework that fits the review
PICO separates population, intervention, comparison, and outcome. PICOS adds study design. For other questions, PEO covers population, exposure, and outcome; SPIDER covers sample, phenomenon of interest, design, evaluation, and research type.
These frameworks clarify scope, but every element doesn't need a search block. Comparators and outcomes may be poorly indexed, so requiring them can exclude relevant studies.
For a question about telemedicine versus usual care for blood-pressure control in adults with hypertension, begin with hypertension and telemedicine. Apply age, comparator, and outcome eligibility during screening unless testing supports narrower retrieval.
Build a vocabulary for each concept
Combine controlled vocabulary with free-text keywords. MeSH in PubMed and CINAHL Headings organize indexed concepts, while free text captures recent records, acronyms, spelling variants, and terminology outside the thesaurus.
Exploding a subject heading includes narrower headings beneath it. Examine that hierarchy before using it.
Use recent reviews and known relevant primary studies to identify terminology. Test candidate terms against those studies to balance sensitivity and precision and spot missing terms. A literature review matrix can keep each candidate term connected to its source paper.
2. Combine Terms Into a Complete Search String
Use Boolean operators: OR between synonyms and AND between concepts. Parentheses preserve the intended grouping. Use NOT cautiously because a record mentioning an excluded concept may still contain eligible evidence.
For the hypertension and telemedicine question, this complete PubMed search string combines MeSH subject headings with title/abstract terms:
("Hypertension"[Mesh] OR hypertens*[tiab]) AND ("Telemedicine"[Mesh] OR telemedicin*[tiab] OR telehealth[tiab] OR "remote consultation"[tiab])
Here, [Mesh] searches MeSH headings, which normally include narrower headings in PubMed. Meanwhile, [tiab] targets title/abstract text. The asterisk retrieves words sharing a stem, such as hypertension and hypertensive.
This is a starting strategy, not evidence of comprehensive coverage. Test whether it retrieves known eligible studies, inspect relevant records for missing terminology, and revise before the final search.
Also inspect PubMed's Search Details. Automatic term mapping and query expansion can change how an interface interprets unqualified terms. Preserve the interpreted query when it differs from your input.
3. Translate the Strategy for Each Database

Preserve concepts, not punctuation
Boolean logic transfers across platforms in a database search. Field codes, thesaurus terms, truncation, phrase handling, and proximity operators don't.
PubMed and Ovid MEDLINE illustrate the distinction: both support MeSH, but their interfaces use different syntax and search different underlying collections unless coverage is aligned. CINAHL requires its own subject-heading labels rather than copied labels.
Likewise, Ovid's adj3 and another platform's NEAR/3 shouldn't be treated as interchangeable without understanding their distance rules.
Rebuild and test the translated version
An Ovid MEDLINE translation of the example uses these lines:
Line | Ovid MEDLINE strategy |
|---|---|
1 |
|
2 |
|
3 |
|
4 |
|
5 |
|
The concepts remain consistent, while headings and field notation change. Translation doesn't guarantee identical results because database coverage and indexing differ.
Record the selected Ovid MEDLINE segment, including its displayed coverage, rather than writing only "MEDLINE."
4. Record Each Run Using PRISMA-S
Create search documentation for every execution and revision. For the worked PubMed strategy, save the full search strategy verbatim as version 1. Identify PubMed as the database resource and interface, and record the database platform separately when it differs.
Record the actual search date and timestamp, coverage, applied filters, and pre-deduplication count. Use the values from the search run, never an estimate or manuscript date.
PRISMA-S is a 16-item extension of PRISMA for reporting literature searches. Its items provide this reporting map:
Item | What to document |
|---|---|
1. Database name | Database and search platform |
2. Multi-database searching | Databases searched together |
3. Study registries | Registry names and searches |
4. Online resources and browsing | Resources and browsing methods |
5. Citation searching | Seed records and citation methods |
6. Contacts | People contacted for evidence |
7. Other methods | Additional identification methods |
8. Full search strategies | Exact strategies for each source |
9. Limits and restrictions | Restrictions and their justification |
10. Search filters | Published filters used or adapted |
11. Prior work | Reused or adapted strategies |
12. Updates | Update methods and changes |
13. Dates of searches | Last search date for each strategy |
14. Peer review | Search-review methods |
15. Total records | Records retrieved by source |
16. Deduplication | Process and software used |
Use these items alongside a protocol appropriate to your review type. They aren't a universal search design, and complete reporting doesn't prove that a search reproduces successfully.
5. Preserve Revisions and Review the Strategy
Keep executable files and untouched exports
Save strategies as UTF-8 plain text. Word processors and PDFs are poor master formats because smart quotes, autocorrection, conversion, and copying can corrupt executable syntax.
Preserve dated database exports with abstracts and identifiers such as DOI or PMID. Keep originals read-only, then deduplicate working copies. Record imported totals, removed duplicates, software, and settings.
Each revision needs a version, date, changed expression, reason, and new count. Adding telehealth[tiab], for example, should leave a visible history rather than overwrite the earlier query.
Apply PRESS before the final run
The PRESS guideline (2015) covers six peer-review elements: translation of the research question; Boolean and proximity operators; subject headings; text-word searching; spelling, syntax, and line numbers; and limits and filters.
Ask a librarian or information specialist to review the strategy before translating the final version across databases. Preserve the feedback and your responses.
If AI suggests terminology, retain the prompt and accepted changes, then assess the suggestions against source papers and database vocabulary. Our guide to systematic review tools by task distinguishes discovery assistance from screening and extraction workflows.
6. Rerun the Search and Compare Its Contents
Calculate count differences consistently
Rerun the saved strategy without silently repairing it. Reapply the original limits and constrain dates to the original cutoff, recording the exact date field used.
Calculate percentage difference as:
100 × absolute difference between rerun and original counts ÷ original count
The study's 10% tolerance was an operational threshold, not a universal methodological standard. Use it as a diagnostic, with any threshold defined in your protocol. If the original count was zero, percentage difference is undefined; report both counts directly.
When execution fails, retain the error and create a separate revised strategy. Otherwise, you lose evidence of why reproduction failed.
Compare identifiers, not only totals
Compare search results using stable identifiers, not only the number of results. Match original and rerun exports, then inspect unmatched records. For content reproducibility, calculate how many originally included studies the rerun retrieves, using studies expected within that database and date range as the denominator.
Equal result counts can conceal different records. Identifier overlap reveals replacements that a count comparison misses.
Databases add, remove, and correct records. They also change fields, syntax, controlled vocabulary, retrieval algorithms, automatic mapping, and sometimes cease operating. Therefore, an original publication-date limit can't reconstruct an earlier database snapshot.
Preserve exports alongside queries. For citation chasing, retain seed papers, backward or forward direction, service, dates, settings, and retrieved records.
Frequently Asked Questions
Should every PICO element appear in the search?
No. PICO structures the question, but indexing determines which concepts belong in the query. Requiring poorly reported outcomes or comparators can reduce sensitivity. Test narrower versions against known relevant studies.
Can a well-documented search still miss evidence?
Yes. Documentation makes methods inspectable; it doesn't establish completeness. Test terminology, use relevant databases, and supplement searches with appropriate citation chasing. Keep supplementary methods separate in the search log.
Make the Search Rerunnable, Not Merely Readable
A reproducible search strategy connects concepts, executable syntax, search conditions, and preserved results. Its audit trail lets another researcher distinguish a changed database from a changed method.
Save each strategy and export when you run it. Then compare both counts and study retrieval before treating the search as reproducible.