PropBank

PropBank
Name	PropBank
Launched	2005
Developer	Columbia University; University of Pennsylvania
Language	English
Discipline	Computational linguistics; Natural language processing
License	Various academic licenses
Website	(academic resource)

PropBank

PropBank is a predicate-argument annotation corpus created to provide a layer of semantic role labels over corpora annotated with syntactic structure. It complements syntactic resources and has been used alongside statistical models, machine learning frameworks, and multilingual projects to support research in semantic parsing, information extraction, and text understanding.

Overview

PropBank was developed to add a consistent set of predicate-argument labels to corpora such as the Wall Street Journal portion of the Penn Treebank and to support comparative work across projects like FrameNet and VerbNet. Early development involved teams at Columbia University and the University of Pennsylvania and drew on methodologies from projects at Institute for Logic, Language and Computation, Massachusetts Institute of Technology, Stanford University, University of Oxford, University of Edinburgh, University of Toronto, University of California, Berkeley, Carnegie Mellon University, University of Washington, Johns Hopkins University, University of Illinois Urbana-Champaign, University of California, San Diego, University of Pennsylvania, University of Maryland, Rutgers University, New York University, Princeton University, Yale University, Harvard University, Cornell University, Brown University, University of Southern California, University of British Columbia, McGill University, University of Michigan, Georgia Institute of Technology, University of Pittsburgh, University of California, Los Angeles, University of Texas at Austin, University of Massachusetts Amherst, University of Chicago, University of North Carolina at Chapel Hill, Dartmouth College, Indiana University Bloomington, Peking University, Tsinghua University, University of Tokyo, Max Planck Institute for Informatics, SRI International, Microsoft Research, Google Research, Facebook AI Research, IBM Research, Amazon AI, Allen Institute for AI, DeepMind, and OpenAI through citations and downstream use.

Annotation Scheme

The annotation scheme assigns numbered argument roles (Arg0, Arg1, ...), adjuncts (ArgM-LOC, ArgM-TMP, etc.), and predicate-specific role descriptions aligned to lexical entries. Scheme design was informed by linguistic work at Linguistic Society of America conferences and draws on role semantics present in descriptions by researchers associated with Noam Chomsky-influenced syntactic theory, distributions noted in corpora processed with parsers like those from Michael Collins and systems evaluated at the Conference on Computational Natural Language Learning and the Association for Computational Linguistics. The role inventory permits mapping between PropBank frames and verb classes cataloged in resources created by authors associated with Beth Levin and datasets coordinated with the SemEval campaigns. Annotation guidelines were developed by contributors connected to projects at National Science Foundation, Defense Advanced Research Projects Agency, European Research Council, and teams publishing at venues including ACL (conference), EMNLP, NAACL, COLING, and LREC.

Construction and Corpus

The PropBank corpus was constructed by annotating predicate-argument structures over existing syntactic trees such as those in the Penn Treebank; annotators included graduate researchers trained via protocols from groups at Columbia University and the University of Pennsylvania. Annotation tools and adjudication workflows referenced software from laboratories at Stanford University and Carnegie Mellon University. The initial release covered the Wall Street Journal corpus and later extensions included conversational text, newswire, and multilingual efforts linked to initiatives at LDC (Linguistic Data Consortium), ELRA (European Language Resources Association), Universal Dependencies, and collaborations with language centers at University of Hong Kong, Seoul National University, Indian Institute of Science, Universidade de São Paulo, University of Melbourne, University of Cape Town, and University of Auckland.

Applications and Use in NLP

PropBank has been widely used to train semantic role labeling (SRL) systems implemented by teams at Google Research, Microsoft Research, Facebook AI Research, DeepMind, OpenAI, and academic groups at Stanford University, NYU, Carnegie Mellon University, Columbia University, University of Pennsylvania, University of Chicago, University of Toronto, University of Washington, Johns Hopkins University, Massachusetts Institute of Technology, Princeton University, Harvard University, Yale University, University of California, Berkeley, University of California, Los Angeles, University of Illinois Urbana-Champaign, and Georgia Institute of Technology. Tasks benefiting from PropBank annotations include semantic parsing evaluated in projects like AllenNLP, question answering systems used by teams at SQuAD-related research groups, information extraction pipelines in corporate labs such as IBM Research and Amazon AI, and downstream tasks in machine translation work at Google Translate and syntax-aware language models developed by OpenAI and DeepMind. PropBank has influenced datasets and benchmarks curated by organizers of SemEval, CoNLL shared tasks, and multilingual SRL efforts linked to the Universal Proposition Bank initiative.

Evaluation and Relations to Other Resources

PropBank is often evaluated alongside FrameNet and VerbNet for coverage, consistency, and utility in downstream systems; comparisons appear in publications by researchers affiliated with Columbia University, Carnegie Mellon University, Stanford University, University of Pennsylvania, Massachusetts Institute of Technology, University of Oxford, University of Edinburgh, University of Toronto, and Johns Hopkins University. Shared task results at CoNLL and SemEval highlight differences in granularity between PropBank's argument-numbering scheme and FrameNet's frame semantics or VerbNet's class-based descriptions. Integration efforts tie PropBank annotations to the Universal Dependencies project and to multilingual lexicons produced by teams at LDC and ELRA.

Limitations and Criticism

Critiques of PropBank include concerns about its predicate-specific role labels versus more theory-driven frames like those in FrameNet and potential inconsistencies noted by evaluators from panels at ACL (conference), EMNLP, and LREC. Corpus coverage limitations have been raised by researchers at University of Pennsylvania, Columbia University, Stanford University, Carnegie Mellon University, and University of California, Berkeley who advocate for broader genres and languages. Additional criticisms concern annotation speed and inter-annotator agreement reported in studies by teams at Johns Hopkins University, University of Maryland, University of Illinois Urbana-Champaign, University of Texas at Austin, University of Massachusetts Amherst, and the effects of mapping PropBank roles into neural architectures developed at Google Research, Facebook AI Research, OpenAI, and DeepMind.

Category:Corpus linguistics