Netcraft

Netcraft is an Internet services company founded in 1995 that provides web server and hosting data, anti-phishing services, and security reputation products. The firm publishes empirical surveys and datasets used by researchers, technology companies, and institutions such as Mozilla Foundation, Google, Microsoft, Amazon (company), and Cloudflare. Known for its long-running web server survey, the company interacts with infrastructures and standards bodies including IETF, ICANN, and various certificate authorities such as Let's Encrypt and DigiCert.

History

Netcraft was established in 1995 during the early commercial expansion of the World Wide Web and the dot-com era, a period contemporaneous with firms like Netscape and Yahoo!. Early work focused on web server identification and measuring deployments of software such as Apache HTTP Server, Nginx, and Microsoft Internet Information Services. Over time the company expanded into anti-fraud and security services alongside partnerships and dialogue with organizations including Royal Mail (for fraud awareness), the Metropolitan Police Service in the context of cybercrime investigations, and academic groups at institutions such as University of Cambridge and MIT conducting Internet measurement research. The firm’s timeline includes adaptation through the Dot-com bubble, the growth of cloud computing by providers like Amazon Web Services and Google Cloud Platform, and the rise of certificate transparency initiatives involving Google and Apple.

Services and Products

Netcraft offers a suite of products aimed at web infrastructure visibility and anti-fraud. Core offerings include the long-standing web server and hosting surveys that identify software such as OpenSSL, BIND (Berkeley Internet Name Domain), and container platforms influenced by Docker. Security products include anti-phishing takedown services used by financial institutions like Barclays and HSBC (bank), SSL/TLS certificate testing aligned with industry practices promoted by IETF working groups, and malware detection that integrates with platforms such as Splunk and CrowdStrike. The company supplies APIs and datafeeds consumed by content delivery networks like Akamai Technologies and monitoring suites from New Relic (company). It also offers browser extensions and plugins interoperable with Mozilla Firefox and Google Chrome for site safety indicators.

Research and Reports

Netcraft publishes regular empirical reports and datasets tracking server market share, hosting provider trends, phishing takedowns, and SSL/TLS configuration statistics. Its web server survey provides longitudinal data on projects such as Apache Tomcat, Lighttpd, and Apache HTTP Server while documenting shifts toward architectures promoted by Kubernetes and NGINX, Inc.. The company’s phishing reports have been cited by regulators and publications, and its takedown metrics intersect with actions by organizations like FBI cyber units and industry groups such as Anti-Phishing Working Group. Research outputs have been referenced in academic work at Stanford University and University of Oxford as well as in analyses by think tanks including RAND Corporation and policy bodies like European Commission digital security initiatives.

Security Incidents and Controversies

Netcraft has both reported on and been entwined with notable security incidents and debates. Its datasets have been used in investigations into large-scale phishing campaigns linked to actors discussed by Europol and INTERPOL, and its takedown operations sometimes involved coordination with hosting providers such as Hetzner Online and OVHcloud. Controversies have arisen around measurement techniques and fingerprinting practices similar to discussions involving Shodan and Censys, prompting dialogue with privacy advocates and oversight by regulators including agencies in United Kingdom and European Union jurisdictions. The company’s scanning activity has at times elicited responses from operators of high-profile platforms like Facebook and Twitter over suspected scanning impacts, mirroring tensions seen historically between scanners and platform defenders during incidents such as the Mirai (malware) botnet investigations.

Business Structure and Partnerships

Operating as a privately held company headquartered in London, Netcraft engages in commercial relationships with technology vendors, financial institutions, and law enforcement. Partnerships and customers have included cloud infrastructure firms like DigitalOcean and Microsoft Azure, certificate authorities such as Let’s Encrypt and DigiCert, and security vendors including McAfee and Symantec (company). The company participates in industry events like RSA Conference and collaborates with standards organizations including IETF working groups and the CA/Browser Forum. Netcraft’s commercial model combines subscription datafeeds, professional services, and incident response work, interfacing with compliance regimes overseen by bodies such as Financial Conduct Authority and cybersecurity frameworks promoted by NIST.

Category:Internet measurement