San Francisco Declaration on Research Assessment

San Francisco Declaration on Research Assessment
DORA · CC BY-SA 4.0 · source
Name	San Francisco Declaration on Research Assessment
Abbreviation	DORA
Presented	2012
Venue	American Society for Cell Biology meeting
Location	San Francisco
Subject	Research evaluation, bibliometrics, academic publishing

San Francisco Declaration on Research Assessment The San Francisco Declaration on Research Assessment is a 2012 statement addressing the use of journal-based metrics in research evaluation, emphasizing assessment reform across universities, funders, publishers, and researchers. It was drafted by leaders attending a meeting hosted by the American Society for Cell Biology with participation from representatives of Wellcome Trust, National Institutes of Health, European Molecular Biology Organization, Howard Hughes Medical Institute, and others, and quickly gained support from a diverse set of institutions and individuals. The declaration advocates for changes in hiring, promotion, and funding practices to reduce reliance on the Journal Impact Factor, and encourages transparent, article-level assessment and responsible metrics.

Background

The declaration originated at a meeting convened by the American Society for Cell Biology in San Francisco where participants included staff from Wellcome Trust, Howard Hughes Medical Institute, National Institutes of Health, European Research Council, and editors from journals such as Nature (journal), Science (journal), and PLOS. Concerns traced to critiques of the Journal Impact Factor produced by Clarivate and the historical influence of Eugene Garfield, whose work on citation indexing shaped practices across Thomson Reuters and later Web of Science. Debates referenced bibliometric research by scholars linked to Leiden University, University of Leiden, University of Leiden's Centre for Science and Technology Studies, and investigators such as Loet Leydesdorff and Derek J. de Solla Price. Prior initiatives that informed the declaration included recommendations from Organisation for Economic Co-operation and Development, discussions at the Royal Society, and policy dialogues involving European Commission research directors.

Principles and Recommendations

The declaration sets out principles urging assessment based on the intrinsic merits of individual research outputs, not the venue of publication, and recommends using a variety of quantitative and qualitative measures. It explicitly challenges reliance on the Journal Impact Factor and promotes article-level metrics as used by PLOS, F1000Research, and platforms like Altmetric. It calls for funders such as Wellcome Trust and National Institutes of Health to align evaluation criteria with diverse outputs including datasets deposited in Dryad (repository), code hosted on GitHub, and preprints on bioRxiv and arXiv. Institutions such as University of California, University of Cambridge, University of Oxford, and disciplinary societies like American Chemical Society and Association for Computing Machinery are encouraged to revise promotion criteria, while publishers including Elsevier, Springer Nature, and Wiley are urged to provide transparent metadata and reduce emphasis on aggregate journal metrics.

Implementation and Signatories

Adoption involved a broad coalition of signatories: universities like Harvard University, Stanford University, Massachusetts Institute of Technology, and University College London; funders including Gatsby Charitable Foundation and European Research Council; publishers such as Public Library of Science and BioMed Central; and professional societies like American Society for Cell Biology and Royal Society of Chemistry. Signatories ranged from individual researchers such as Paul Nurse and Jo Handelsman to administrative leaders in organizations including National Science Foundation, Wellcome Trust, and Canada Research Chairs Program. Implementation efforts were coordinated with groups like Research Councils UK, Australian Research Council, and national agencies in France, Germany, and Netherlands.

Impact and Criticism

The declaration catalyzed policy changes and spawned debates in venues such as Science (journal), Nature (journal), and The Lancet. Positive impacts included revised hiring guidelines at institutions like University of Glasgow, altered grant review practices at funders including Wellcome Trust and European Research Council, and development of alternative metrics by companies like Altmetric and initiatives at COPE and ORCID. Critics argued that the declaration lacked enforcement mechanisms and that alternatives risk replicating biases; commentators from Clarivate, analysts at Leiden University, and editorial voices at Elsevier highlighted challenges in operationalizing qualitative assessment at scale. Scholars including John Ioannidis and Stephen Curry participated in public discussion about unintended consequences and the burdens on peer review systems managed by Publons and journal editors.

Adoption and Policy Changes

Following the declaration, several national and institutional policy shifts occurred: funding bodies such as Wellcome Trust and Research Councils UK updated grant assessment frameworks, the European Commission referenced DORA principles in research integrity and evaluation guidance, and consortia including the Coalition for Advancing Research Assessment promoted practical tools for implementation. Universities such as University of Leiden, University of Amsterdam, University of Melbourne, and University of Toronto revised promotion dossiers to emphasize narrative CVs and responsible metrics. Publishers including PLOS and eLife adopted article-level indicators and transparent peer-review models, while research infrastructure projects like Crossref and Datacite expanded metadata practices to support discoverability and credit.

Related Initiatives and Developments

The declaration is linked to parallel efforts: the Leiden Manifesto for research metrics, the Metric Tide report commissioned by Research England, the formation of the Coalition for Advancing Research Assessment (CoARA), and the work of Committee on Publication Ethics on ethical publishing. Technological and policy developments intersecting with DORA include widespread adoption of ORCID, growth of preprint servers such as bioRxiv and medRxiv, and open science movements championed by Plan S and funders like Horizon Europe. Ongoing dialogues involve stakeholders from European University Association, Association of American Universities, and global funders shaping next-generation research assessment frameworks.

Category:Academic publishing