ITU‑T P.910

ITU‑T P.910
Title	ITU‑T P.910
Status	Recommendation
Organization	International Telecommunication Union
Domain	Multimedia quality assessment
First published	2008
Latest revision	2020

ITU‑T P.910

ITU‑T P.910 is an international Recommendation for subjective video quality assessment methods aimed at interactive and conversational multimedia applications. It provides standardized procedures for conducting perceptual experiments, specifying viewing conditions, participant selection, and rating scales to evaluate video and audiovisual quality for services such as videoconferencing, streaming, and telepresence. The Recommendation connects to broader standardization efforts and research in audiovisual quality, human factors, and codec evaluation.

Overview

The Recommendation was developed by the International Telecommunication Union's ITU‑T Study Group 12, informed by experts from European Telecommunications Standards Institute, British Standards Institution, American National Standards Institute, Institute of Electrical and Electronics Engineers, and contributors from Fraunhofer Society, Bell Labs, Nokia, and Huawei. It synthesizes methods used in research at institutions such as Massachusetts Institute of Technology, Carnegie Mellon University, Stanford University, University of Cambridge, and ETH Zurich, while aligning with measurement frameworks from MPEG, 3rd Generation Partnership Project, Alliance for Open Media, and Moving Picture Experts Group. The Recommendation interoperates with codecs and transport technologies developed by ISO/IEC JTC 1/SC 29, IETF, Google, and Apple.

Scope and Objectives

P.910 targets subjective assessment for conversational and interactive video, including one‑to‑one and multi‑party scenarios encountered in deployments by Microsoft, Zoom Video Communications, Cisco Systems, Tencent, and Amazon Web Services. Objectives include enabling comparative evaluation of codecs such as those from AV1, H.264/MPEG-4 AVC, H.265/HEVC, VP9, and experimental codecs from Xiph.Org Foundation and Mozilla Foundation, and to support testing of streaming services like Netflix, YouTube, and Hulu. The Recommendation aims to facilitate reproducibility across laboratories like National Institute of Standards and Technology, European Broadcasting Union, and university research labs, and to provide guidance consistent with practices in Radiocommunication Sector and World Radiocommunication Conference activities.

Test Methods and Procedures

Procedures specify controlled environments modeled on setups used by BBC Research and Development, NHK Science & Technology Research Laboratories, and Fraunhofer IIS. It defines participant recruitment criteria informed by methodologies at University College London, Delft University of Technology, and Tsinghua University, with screening similar to protocols from American Psychological Association ethics guidelines and World Health Organization hearing standards. Test signals and content selection reference motion and texture exemplars from archives like IETF Test Vectors, productions from BBC Archive, sports footage from Olympic Games broadcasts, and conversational clips resembling trials at MIT Media Lab. Equipment and presentation align with recommendations from European Telecommunications Network Operators' Association and measurement practices at National Physical Laboratory and Japan Quality Assurance Organization.

Subjective Rating Scales and Metrics

P.910 standardizes subjective scales including absolute category rating, continuous scales, and paired comparison methods paralleling studies at Stanford Psychophysics Laboratory, Max Planck Institute for Human Cognitive and Brain Sciences, and University of California, Berkeley. It defines computation of mean opinion score and confidence intervals used by ITU‑T Study Group 12 rapporteurs, and statistical analysis approaches consistent with practices at Royal Statistical Society and presented in conferences such as ACM Multimedia, IEEE International Conference on Image Processing, and European Signal Processing Conference. The Recommendation addresses inter‑rater reliability and outlier detection with methods adopted in meta‑analyses from Cochrane Collaboration and standards from International Organization for Standardization committees.

Applications and Use Cases

Deployment scenarios include quality benchmarking in products from Skype, FaceTime, WhatsApp, and enterprise services by Avaya and Polycom. It supports regulatory testing and consumer labeling used by agencies like Federal Communications Commission, European Commission, and national regulators in Japan and South Korea. Research applications include codec development at Xilinx and Intel Labs, perceptual optimization in streaming platforms run by Spotify and Deezer (for video integrations), and end‑to‑end quality assurance in live‑streaming events such as FIFA World Cup and UEFA Champions League broadcasts.

History and Revisions

The Recommendation evolved from earlier ITU efforts including ITU‑T P.800 and dialogues with ITU‑R working groups, reflecting inputs from standardization meetings in Geneva, Geneva Conference Centre, Beijing, and Geneva Academy of International Humanitarian Law and Human Rights forums. Major revisions incorporated findings from interlaboratory studies coordinated with European Telecommunications Standards Institute and results published at IEEE Transactions on Multimedia and Signal Processing: Image Communication. Subsequent updates harmonized P.910 with newer work items from ITU‑T Study Group 16, codec evaluations by Moving Picture Experts Group, and perceptual modeling advances reported at NeurIPS and CVPR.

Category:Telecommunications standards