CAIDA Ark

CAIDA Ark
Name	Ark
Organization	Center for Applied Internet Data Analysis
Launched	2007
Purpose	Active Internet topology measurement, network research
Location	San Diego, California

CAIDA Ark

CAIDA Ark is an active Internet topology measurement platform operated by the Center for Applied Internet Data Analysis (CAIDA). It performs large-scale probing and measurement of the global Internet infrastructure to support research in network topology, routing, security, and performance. The platform coordinates distributed vantage points, standardized probing software, and curated datasets to enable reproducible studies by researchers at institutions such as University of California, San Diego, Los Alamos National Laboratory, INTERNET2, and commercial partners.

Overview

Ark was designed to provide continuous, Internet-wide vantage for mapping Autonomous system graphs, inferring router-level topology, and analyzing routing dynamics in the context of protocols like Border Gateway Protocol and Internet Protocol. The project extended earlier measurement efforts including projects at RIPE NCC, CAIDA predecessors, and research initiatives funded by agencies such as the National Science Foundation and the Defense Advanced Research Projects Agency. Ark leverages a global mesh of monitors hosted at research networks, academic campuses, and exchange points including nodes in regions served by European Organization for Nuclear Research, Korea Advanced Institute of Science and Technology, and regional operators participating in PeeringDB.

Measurement Infrastructure

Ark’s infrastructure comprises distributed measurement hosts called monitors, centralized control services, and data repositories. Monitors are typically colocated on platforms run by organizations like Internet2, ESnet, SURFnet, and commercial entities that participate in measurement collaborations. Each monitor runs specialized probing software configured to perform traceroute-style measurements to large, rotating targets drawn from address registries such as ARIN, RIPE, APNIC, LACNIC, and AFRINIC. The control plane coordinates schedules and aggregates metadata using timekeeping referencing services like Network Time Protocol, while the storage and dissemination of results rely on institutional repositories at University of California, San Diego and mirrors hosted by partner organizations.

Data Collection and Methodology

Ark employs active probing methodologies derived from classic tools including traceroute, Paris traceroute, and newer measurement paradigms. Probes follow carefully designed sampling strategies to reduce bias introduced by load-balancing mechanisms operated by networks like Level 3 Communications and AT&T. The methodology incorporates alias resolution techniques influenced by work from researchers at Carnegie Mellon University and Massachusetts Institute of Technology to map IP addresses to routers, and it uses path inference algorithms related to studies produced at Princeton University and Stanford University. Ark collects timestamps, IP hop lists, ICMP responses, and BGP-informed annotations by correlating with feeds from route collectors such as Route Views and RIPE NCC RIS. Measurement campaigns are versioned and documented to support reproducibility by researchers at institutions including ETH Zurich and University of Tokyo.

Datasets and Tools

Ark publishes curated datasets that have been widely used in topology and security research. Core datasets include longitudinal traceroute archives, inferred router- and AS-level topologies, and alias-resolution outputs produced using tools inspired by projects at University of Massachusetts Amherst and Georgia Institute of Technology. Ark’s toolchain includes implementations of constrained traceroute schedulers, dataset parsers, and visualization utilities that interoperate with software from CAIDA and community tools developed at UC San Diego. Data products are distributed in formats compatible with analysis platforms used by researchers at University of Cambridge and Imperial College London, and have been cited in studies on Internet resilience by teams at Columbia University and University of California, Berkeley.

Research and Applications

Researchers have used Ark data to investigate Internet topology evolution, routing resiliency during events such as outages affecting networks like Level 3 Communications and incidents involving operators in regions overseen by APNIC, to study censorship behavior observed in various national deployments, and to analyze performance impacts of protocol changes including IPv6 adoption. Studies leveraging Ark datasets have appeared from groups at University of Michigan, Duke University, Fudan University, and Tsinghua University, addressing topics from interdomain policy inference to anycast measurement experiments similar to those by teams at Google and Cloudflare. Ark-enabled analyses have informed operational decisions at large content providers and research collaborations with organizations such as IETF working groups and regional registries.

Privacy, Ethics, and Legal Considerations

Ark’s active probing raises privacy and legal issues addressed through operational policies, opt-out mechanisms, and coordination with network operators and oversight bodies like Institutional Review Board (IRB) offices at partner universities. CAIDA implements best practices to minimize intrusiveness, including rate limiting, probe identification strings, and channels for network operators to request measurement suppression as practiced by RIPE NCC and other measurement initiatives. Ethical discussions surrounding active Internet measurement involving Ark have involved stakeholders from Electronic Frontier Foundation and academic ethics committees, and legal considerations have been informed by precedents in cases related to network scanning and research engagement with entities such as Federal Communications Commission and regional data protection authorities.

Category:Internet measurement