OpenReview

OpenReview
Name	OpenReview
Type	Platform
Founded	2012
Founders	Stefanie Lindstaedt
Headquarters	San Francisco, California
Products	Peer review platform

OpenReview OpenReview is an online scholarly peer review platform designed to support transparent, accountable evaluation of academic manuscripts and conference submissions. It integrates submission management, public commentary, and editorial workflows to serve research communities across computer science, artificial intelligence, and related fields. The platform has been used by major conferences and organizations, influencing debates around transparency in scholarly communication and peer review reform.

Overview

OpenReview operates as a web-based service connecting authors, reviewers, area chairs, and program committees for venues including NeurIPS, ICLR, AAAI Conference on Artificial Intelligence, ACL, and SIGGRAPH. The platform supports public posting of reviews, meta-reviews, and author responses, linking to discussions among contributors affiliated with institutions such as Massachusetts Institute of Technology, Stanford University, Carnegie Mellon University, University of California, Berkeley, and University of Toronto. It aims to facilitate reproducibility alongside initiatives like arXiv and repositories such as GitHub, promoting interaction with funding agencies like the National Science Foundation and publishers including IEEE and ACM.

History

OpenReview emerged in the early 2010s amid reforms to peer review influenced by controversies at conferences like NeurIPS 2014 and policy discussions in venues such as Science and Nature. Its development involved collaborations with researchers from institutions including University of California, Irvine and technology firms active in machine learning such as Google and OpenAI. Key milestones include adoption by the International Conference on Learning Representations (ICLR) and uptake by workshop organizers from European Conference on Computer Vision and regional meetings supported by societies like the Association for Computational Linguistics and IEEE Computer Society.

Platform and Features

The platform implements features for submission intake, conflict-of-interest management, reviewer bidding, and program committee orchestration used in events such as NeurIPS, ICML, and KDD. It integrates cryptographic and identity mechanisms used by contributors from organizations like ORCID and institutions such as Harvard University and Princeton University. OpenReview provides APIs and integration patterns similar to services from Zenodo and Figshare, enabling linkage with code archives on GitHub and preprints on bioRxiv and arXiv. Administrative workflows accommodate roles familiar to members of the IEEE and ACM, including support staff and external auditors.

Peer Review Model

OpenReview popularized variants of open peer review and interactive public commentary influenced by models trialed in journals like eLife and platforms such as F1000Research. The system permits anonymous, pseudonymous, and signed reviews, enabling participation from reviewers affiliated with Google Research, Microsoft Research, DeepMind, and academic labs at ETH Zurich. It supports rebuttals and meta-reviews akin to editorial practices at Nature Communications and PLOS ONE, and its threaded discussion model echoes community moderation seen on platforms like Stack Overflow and scholarly networks associated with ResearchGate.

Adoption and Impact

Adoption by major machine learning conferences such as ICLR and workshops attached to NeurIPS and ICML popularized OpenReview among researchers at MIT Media Lab, Berkeley AI Research, Facebook AI Research, and Google Brain. Funders including the NSF and philanthropic organizations exploring open science saw OpenReview as complementary to mandates from institutions like Wellcome Trust and European Research Council. The platform influenced policy debates in editorial boards of Nature and Science and catalyzed tools for reproducibility championed by initiatives such as the Reproducibility Project and the Open Science Framework.

Criticisms and Controversies

OpenReview has been criticized in community discussions hosted by organizations like ACM and IEEE for potential reviewer harassment, confidentiality breaches, and conflicts involving industry-affiliated reviewers from Google, Microsoft, and Amazon. Debates in outlets such as Nature and panels at NeurIPS highlighted concerns about anonymity policies, reviewer bias, and workload for program committees at events like ICLR and ACL. Legal and ethical questions raised by stakeholders including university counsel at Stanford University and University of California campuses have led to ongoing policy refinements and experiments with hybrid review models resembling practices at PLOS and eLife.

Category:Academic publishing platforms