Project Wycheproof

Project Wycheproof
Name	Project Wycheproof
Developer	Google Brain
First release	2015
Latest release	2017
Repository	GitHub
License	Apache License 2.0

Project Wycheproof is a research initiative by Google Brain to evaluate cryptographic library implementations for known weaknesses and pitfalls using automated test suites. The project produced a collection of test vectors and harnesses that exercise cryptographic primitives across widely used libraries, and its results influenced patching and disclosure across open source and commercial projects. It combined automated analysis with targeted human review and was disseminated via code repositories and academic commentary.

Overview

Project Wycheproof assembled test vectors and harnesses to exercise implementations of RSA, elliptic curve, AES block ciphers, HMAC, and signature schemes across libraries such as OpenSSL, Bouncy Castle, NSS, GnuTLS, LibreSSL, and WolfSSL. The suite identified edge cases including malformed inputs, boundary conditions, and protocol-specific misimplementations, producing reproducible failures and facilitating coordinated disclosure to projects such as Google, Mozilla, Oracle and vendors maintaining Android stacks. The project published artifacts on GitHub, accompanied by issue reports, and was discussed at venues including USENIX Security Symposium, IEEE Symposium on Security and Privacy, and Black Hat.

Background and Motivation

Researchers behind the effort drew on prior work in implementation testing exemplified by Cryptol, CWE, and the National Institute of Standards and Technology test vectors to address recurring interoperability and safety problems found in libraries like OpenSSL after incidents such as the Heartbleed bug and vulnerabilities reported in GnuTLS. Motivation included reducing risks exposed by historical events involving RSA-CRT, Bleichenbacher attack, and malformed X.509 certificate handling that had affected ecosystems ranging from Apache HTTP Server deployments to Android applications. The team aimed to provide concrete, automatable checks to complement theoretical cryptanalysis performed by groups associated with IACR, ACM, and IEEE.

Test Suites and Methodology

Wycheproof developed a corpus of JSON-formatted test vectors and harnesses invoking library APIs and command-line tools across platforms such as Linux, Windows, and macOS. Tests included boundary cases for algorithms standardized by NIST, IETF, and ANSI, covering primitives like AES-GCM, RSAES-OAEP, ECDSA, and HKDF. The methodology combined fuzzing-inspired mutation, hand-crafted malformed inputs inspired by attacks like Bleichenbacher attack and Padding oracle attack, and cross-library comparison similar to techniques used by Google OSS-Fuzz and American Fuzzy Lop. Results were recorded as pass/fail outcomes with metadata linking failures to specific versions and commits tracked on GitHub and coordination channels including Bugzilla and vendor issue trackers.

Findings and Notable Vulnerabilities

The project exposed a range of issues, from incorrect validation of leading zeros in RSA keys to acceptance of weak parameters in ECDSA implementations and signature malleability in DSA. It documented cases where libraries accepted non-canonical encodings for ASN.1 structures, mishandled ciphertext lengths for AES-GCM, and performed inadequate checks for X.509 certificate fields—issues reminiscent of prior advisories affecting OpenSSL, Bouncy Castle, and NSS. Several findings led to CVE assignments and patches issued by vendors including Mozilla, Oracle, and maintainers of OpenJDK, often after coordinated disclosure facilitated via CERT Coordination Center processes.

Impact and Adoption

Wycheproof's test vectors were incorporated into continuous integration pipelines for projects like Bouncy Castle, OpenSSL, and LibreSSL, and influenced quality assurance practices in organizations deploying cryptographic stacks such as Google, Amazon Web Services, and enterprises using Red Hat Enterprise Linux. The corpus was reused in academic evaluations at conferences such as USENIX Security Symposium and NDSS, and referenced by standards bodies including IETF working groups and implementers participating in IETF TLS specifications. The visibility of Wycheproof fostered collaboration between open source maintainers and industry vendors, accelerating remediation of several implementation defects.

Limitations and Criticisms

Critics noted that Wycheproof focused on known classes of failure and did not substitute for formal verification approaches used by projects like Everest or tools such as Coq, Cryptol, and Frama-C. The test-oriented approach produced high-value practical checks but could miss emergent algorithmic weaknesses discovered by cryptanalysts at forums like CRYPTO and EUROCRYPT. Some maintainers cautioned that reliance on the corpus without complementary static analysis could produce false confidence, echoing debates between proponents of fuzzing exemplified by Google OSS-Fuzz and advocates of formal methods in communities around DARPA initiatives and academic labs at MIT and ETH Zurich.

Category:Cryptography