History of rocketfuel
The changes to the source code of this entry in the repository, newest first, each with the differences it made to the entry's files. They are also listed on GitLab.
[entries] Add the date when each entry was added, and remove icon_hash beff7c9
The dates are those of the commits that first added the entries' files (2020-08-11 for the entries of the initial commit); renamed entries keep the date of their first name. Changed: none
diff --git a/entries/rocketfuel/__init__.py b/entries/rocketfuel/__init__.py
index 4cf0b8f..287794b 100644
--- a/entries/rocketfuel/__init__.py
+++ b/entries/rocketfuel/__init__.py
@@ -67,7 +67,7 @@ url = ['https://research.cs.washington.edu/networking/rocketfuel/',
citation = [('N. Spring, R. Mahajan, D. Wetherall, "Measuring ISP topologies with Rocketfuel." Proceedings of ACM SIGCOMM 2002, 133-145 (2002)', 'https://doi.org/10.1145/633025.633039'),
('R. Mahajan, N. Spring, D. Wetherall, T. Anderson, "Inferring link weights using end-to-end measurements." Proceedings of the 2nd ACM SIGCOMM Workshop on Internet Measurement (IMW 2002), 231-236 (2002)', 'https://doi.org/10.1145/637201.637237'),
('N. Spring, R. Mahajan, T. Anderson, "The causes of path inflation." Proceedings of ACM SIGCOMM 2003, 113-124 (2003)', 'https://doi.org/10.1145/863955.863970')]
-icon_hash = None
+date_added = '2026-10-27'
upstream_license = None
# Pinned Internet Archive snapshots of the files of the Rocketfuel page
upstream_prefix = None
[entries] Prefix the properties in the descriptions with (Node), (Edge) or (Graph) bc3dbbd
The lists of properties at the end of the descriptions marked edge properties with "Edge" but left node properties unmarked, or grouped them in "Nodes: `a`, ...; `b`, ..." items, so the kind of a property was often unclear. Every item now starts with (Node), (Edge) or (Graph), followed by the property names, a colon and the explanation, one item per property or group of properties; qualifications (kinds of nodes, networks) follow the names in parentheses. Properties that were only named got a short explanation, and a few items that named the wrong kind of property were corrected.
diff --git a/entries/rocketfuel/__init__.py b/entries/rocketfuel/__init__.py
index 9a8e4e1..4cf0b8f 100644
--- a/entries/rocketfuel/__init__.py
+++ b/entries/rocketfuel/__init__.py
@@ -41,9 +41,26 @@ description = r"""Router-level and PoP-level maps of Internet service providers
The networks have the following properties:
-- `routers_<AS>`: the network property `isp`, the name of the ISP. Nodes: `uid`, the router identifier of the source; `name`, the DNS name of the router (or one of its IP addresses, if it has no name); `location`, the location of the router, as `city, state/country` (`T` if it is ambiguous, e.g. connected to routers in different locations, and `?` if it is unknown); `location_from_dns`, whether the location was derived from the DNS name (true) or from the connectivity of the router (false); `backbone`, whether it is a backbone router (connected to other cities), as opposed to access (gateway) and customer routers; `no_alias_response`, whether the router did not respond to alias resolution (and might not be a distinct router); `radius`, the distance in hops from a named backbone or gateway router (capped at 5); `n_external`, the number of the router's links to addresses outside the ISP.
-- `backbone_<AS>`: the network property `isp`, the name of the ISP. Nodes: `name`, the name of the router in the source (its location and an identifier); `location`, its location. Edges: `weight`, the inferred link weight; `latency`, the latency of the link, in milliseconds, estimated from the geographical distance (rounded up).
-- `pops`: Nodes: `name`, the name of the PoP in the source (`AS:city`); `asn`, the AS number of the ISP; `city`, the city of the PoP. Edges: `count`, the number of times the link was observed in the traceroutes; `latency`, the latency of the link, in milliseconds, estimated from the geographical distance (zero for most links between ISPs in the same city); `weight`, the inferred link weight (`NaN` for links between ISPs); `peering`, whether the link connects two ISPs."""
+- (Graph) `isp` (`routers_<AS>` and `backbone_<AS>`): the name of the ISP.
+- (Node) `uid` (`routers_<AS>`): the router identifier of the source.
+- (Node) `name` (`routers_<AS>`): the DNS name of the router (or one of its IP addresses, if it has no name).
+- (Node) `location` (`routers_<AS>`): the location of the router, as `city, state/country` (`T` if it is ambiguous, e.g. connected to routers in different locations, and `?` if it is unknown).
+- (Node) `location_from_dns` (`routers_<AS>`): whether the location was derived from the DNS name (true) or from the connectivity of the router (false).
+- (Node) `backbone` (`routers_<AS>`): whether it is a backbone router (connected to other cities), as opposed to access (gateway) and customer routers.
+- (Node) `no_alias_response` (`routers_<AS>`): whether the router did not respond to alias resolution (and might not be a distinct router).
+- (Node) `radius` (`routers_<AS>`): the distance in hops from a named backbone or gateway router (capped at 5).
+- (Node) `n_external` (`routers_<AS>`): the number of the router's links to addresses outside the ISP.
+- (Node) `name` (`backbone_<AS>`): the name of the router in the source (its location and an identifier).
+- (Node) `location` (`backbone_<AS>`): its location.
+- (Edge) `weight` (`backbone_<AS>`): the inferred link weight.
+- (Edge) `latency` (`backbone_<AS>`): the latency of the link, in milliseconds, estimated from the geographical distance (rounded up).
+- (Node) `name` (`pops`): the name of the PoP in the source (`AS:city`).
+- (Node) `asn` (`pops`): the AS number of the ISP.
+- (Node) `city` (`pops`): the city of the PoP.
+- (Edge) `count` (`pops`): the number of times the link was observed in the traceroutes.
+- (Edge) `latency` (`pops`): the latency of the link, in milliseconds, estimated from the geographical distance (zero for most links between ISPs in the same city).
+- (Edge) `weight` (`pops`): the inferred link weight (`NaN` for links between ISPs).
+- (Edge) `peering` (`pops`): whether the link connects two ISPs."""
tags = ['Technological', 'Communication', 'Weighted', 'Metadata']
url = ['https://research.cs.washington.edu/networking/rocketfuel/',
'https://web.archive.org/web/2026/https://research.cs.washington.edu/networking/rocketfuel/']
[entries] Remove unneeded code from the entries 2c69f19
Unneeded "global files" statements, unused imports and variables, f-strings without placeholders, duplicate keys in foodweb_little_rock (keeping the order and the values used), and the second definition of get_lines() in twitter_higgs. Nothing changes in the networks or descriptions. Changed: none
diff --git a/entries/rocketfuel/__init__.py b/entries/rocketfuel/__init__.py
index 9c49567..9a8e4e1 100644
--- a/entries/rocketfuel/__init__.py
+++ b/entries/rocketfuel/__init__.py
@@ -126,7 +126,6 @@ def read_pop_edges(f):
@coerce_props()
@annotate()
def parse(alts=None):
- global files
name = __name__.split(".")[-1]
for fnames, alt, fmt in files:
if alts is not None and alt not in alts:
[entries] New entry: rocketfuel 0da3229
diff --git a/entries/rocketfuel/__init__.py b/entries/rocketfuel/__init__.py
new file mode 100644
index 0000000..9c49567
--- /dev/null
+++ b/entries/rocketfuel/__init__.py
@@ -0,0 +1,191 @@
+#! /usr/bin/env python
+# -*- coding: utf-8 -*-
+#
+# Copyright (C) 2026 Tiago de Paula Peixoto <tiago@skewed.de>
+#
+# This program is free software: you can redistribute it and/or modify it under
+# the terms of the GNU Affero General Public License as published by the Free
+# Software Foundation, either version 3 of the License, or (at your option) any
+# later version.
+#
+# This program is distributed in the hope that it will be useful, but WITHOUT
+# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
+# FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
+# details.
+#
+# You should have received a copy of the GNU Affero General Public License along
+# with this program. If not, see <http://www.gnu.org/licenses/>.
+
+
+import re
+import tarfile
+
+from .. import *
+
+# The ten ISPs mapped by Rocketfuel (AS number, name)
+isps = [(1221, "Telstra (Australia)"), (1239, "Sprintlink (US)"),
+ (1755, "Ebone (Europe)"), (2914, "Verio (US)"),
+ (3257, "Tiscali (Europe)"), (3356, "Level 3 (US)"),
+ (3967, "Exodus (US)"), (4755, "VSNL (India)"),
+ (6461, "Abovenet (US)"), (7018, "AT&T (US)")]
+isp_names = dict(isps)
+# The ISPs whose backbones were annotated with inferred link weights
+weight_isps = [1221, 1239, 1755, 3257, 3967, 6461]
+
+title = "Rocketfuel ISP topologies (2002-2003)"
+description = r"""Router-level and PoP-level maps of Internet service providers (ISPs), measured by Spring, Mahajan, Wetherall and Anderson with Rocketfuel, an ISP topology mapping engine. Rocketfuel ran traceroutes from about 800 vantage points (public traceroute servers), chosen with BGP routing information so as to cross the ISP being mapped, recognized the routers of the ISP and their locations from their DNS names (with ISP-specific naming conventions), and resolved the aliases (the IP addresses of different interfaces of the same router) with a technique of its own. The data set contains three kinds of networks:
+
+- `routers_<AS>` (e.g. `routers_1239`): the router-level maps of ten ISPs (SIGCOMM 2002; files of 2003), identified by their AS numbers: 1221 (Telstra, Australia), 1239 (Sprintlink, US), 1755 (Ebone, Europe), 2914 (Verio, US), 3257 (Tiscali, Europe), 3356 (Level 3, US), 3967 (Exodus, US), 4755 (VSNL, India), 6461 (Abovenet, US) and 7018 (AT&T, US). Nodes are the routers of the ISP (the full maps of the source, the `.cch` files), and an undirected edge connects two routers seen directly connected in some traceroute. The maps include the routers that could belong to the ISP, such as chains of routers leading to customers; the vertex property `radius` allows restricting them to the backbone and gateway routers (radius 0), or to these and their direct neighbors (radius at most 1, the maps mostly used in the paper). The links of the routers to external addresses (outside the ISP) are not included as edges; their number is stored in `n_external`.
+- `backbone_<AS>` (e.g. `backbone_1239`): the backbone topologies of six of these ISPs (1221, 1239, 1755, 3257, 3967 and 6461), annotated with link weights inferred from the observed paths, assuming shortest-path routing, and with geographical latencies (IMW 2002). Nodes are backbone routers, named by their location and an identifier (these identifiers are not the same as the router identifiers of the `routers_<AS>` networks, which were assigned anew), and undirected edges are the backbone links.
+- `pops`: a PoP-level map of 67 ISPs and of the links between them (SIGCOMM 2003), including the PoPs of two more ISPs that appear only in links to these. Nodes are points of presence (PoPs), i.e. the cities where an ISP has routers, and an undirected edge connects two PoPs that were seen connected in some traceroute, either within an ISP, with link weights inferred as above, or between two ISPs (peering links).
+
+The networks have the following properties:
+
+- `routers_<AS>`: the network property `isp`, the name of the ISP. Nodes: `uid`, the router identifier of the source; `name`, the DNS name of the router (or one of its IP addresses, if it has no name); `location`, the location of the router, as `city, state/country` (`T` if it is ambiguous, e.g. connected to routers in different locations, and `?` if it is unknown); `location_from_dns`, whether the location was derived from the DNS name (true) or from the connectivity of the router (false); `backbone`, whether it is a backbone router (connected to other cities), as opposed to access (gateway) and customer routers; `no_alias_response`, whether the router did not respond to alias resolution (and might not be a distinct router); `radius`, the distance in hops from a named backbone or gateway router (capped at 5); `n_external`, the number of the router's links to addresses outside the ISP.
+- `backbone_<AS>`: the network property `isp`, the name of the ISP. Nodes: `name`, the name of the router in the source (its location and an identifier); `location`, its location. Edges: `weight`, the inferred link weight; `latency`, the latency of the link, in milliseconds, estimated from the geographical distance (rounded up).
+- `pops`: Nodes: `name`, the name of the PoP in the source (`AS:city`); `asn`, the AS number of the ISP; `city`, the city of the PoP. Edges: `count`, the number of times the link was observed in the traceroutes; `latency`, the latency of the link, in milliseconds, estimated from the geographical distance (zero for most links between ISPs in the same city); `weight`, the inferred link weight (`NaN` for links between ISPs); `peering`, whether the link connects two ISPs."""
+tags = ['Technological', 'Communication', 'Weighted', 'Metadata']
+url = ['https://research.cs.washington.edu/networking/rocketfuel/',
+ 'https://web.archive.org/web/2026/https://research.cs.washington.edu/networking/rocketfuel/']
+citation = [('N. Spring, R. Mahajan, D. Wetherall, "Measuring ISP topologies with Rocketfuel." Proceedings of ACM SIGCOMM 2002, 133-145 (2002)', 'https://doi.org/10.1145/633025.633039'),
+ ('R. Mahajan, N. Spring, D. Wetherall, T. Anderson, "Inferring link weights using end-to-end measurements." Proceedings of the 2nd ACM SIGCOMM Workshop on Internet Measurement (IMW 2002), 231-236 (2002)', 'https://doi.org/10.1145/637201.637237'),
+ ('N. Spring, R. Mahajan, T. Anderson, "The causes of path inflation." Proceedings of ACM SIGCOMM 2003, 113-124 (2003)', 'https://doi.org/10.1145/863955.863970')]
+icon_hash = None
+upstream_license = None
+# Pinned Internet Archive snapshots of the files of the Rocketfuel page
+upstream_prefix = None
+archived_urls = {
+ 'rocketfuel_maps_cch.tar.gz': 'https://web.archive.org/web/20260111123036id_/https://research.cs.washington.edu/networking/rocketfuel/maps/rocketfuel_maps_cch.tar.gz',
+ 'weights-dist.tar.gz': 'https://web.archive.org/web/20260221052901id_/https://research.cs.washington.edu/networking/rocketfuel/maps/weights-dist.tar.gz',
+ 'policy-dist.tar.gz': 'https://web.archive.org/web/20260112074048id_/https://research.cs.washington.edu/networking/rocketfuel/maps/policy-dist.tar.gz'}
+
+files = ([(f"rocketfuel_maps_cch.tar.gz:{asn}.cch", f"routers_{asn}", None)
+ for asn, _ in isps] +
+ [((f"weights-dist.tar.gz:{asn}/weights.intra",
+ f"weights-dist.tar.gz:{asn}/latencies.intra"), f"backbone_{asn}", None)
+ for asn in weight_isps] +
+ [("policy-dist.tar.gz", "pops", None)])
+
+def fetch_upstream(force=False):
+ return fetch_archived_files(__name__.split(".")[-1], archived_urls, force,
+ tries=4, timeout=600)
+
+# A router of a .cch file (see README.cch in the archive):
+# uid @loc [+] [bb] (num_neigh) [&ext] -> <nuid> ... {-euid} ... =name[!] rn
+cch_line = re.compile(r"^(\d+) @(\S*)\s+(\+ )?(bb)?\s*\((\d+)\)\s*(?:&(\d+))?\s*->(.*?)\s*=(\S*?)(!)?\s+r(\d+)\s*$")
+
+def parse_cch(f):
+ g = Graph(directed=False)
+ for p, t in [("uid", "int"), ("name", "string"), ("location", "string"),
+ ("location_from_dns", "bool"), ("backbone", "bool"),
+ ("no_alias_response", "bool"), ("radius", "int"),
+ ("n_external", "int")]:
+ g.vp[p] = g.new_vp(t)
+ idx = {}
+ edges = []
+ for line in f:
+ if not line[:1].isdigit():
+ continue # external addresses (negative uids) and comments
+ m = cch_line.match(line.rstrip("\n"))
+ if m is None:
+ raise ValueError(f"unexpected line: {line!r}")
+ uid = int(m.group(1))
+ v = g.add_vertex()
+ idx[uid] = int(v)
+ g.vp.uid[v] = uid
+ g.vp.location[v] = m.group(2).replace("+", " ")
+ g.vp.location_from_dns[v] = m.group(3) is not None
+ g.vp.backbone[v] = m.group(4) is not None
+ g.vp.n_external[v] = int(m.group(6) or 0)
+ g.vp.name[v] = m.group(8)
+ g.vp.no_alias_response[v] = m.group(9) is not None
+ g.vp.radius[v] = int(m.group(10))
+ neighbors = [int(x) for x in re.findall(r"<(\d+)>", m.group(7))]
+ assert len(neighbors) == int(m.group(5))
+ # The neighbor lists are symmetric: each link is kept once
+ edges.extend((uid, w) for w in neighbors if uid < w)
+ g.add_edge_list((idx[u], idx[w]) for u, w in edges)
+ return g
+
+def read_metric(f):
+ "Edge lists with a metric in the third column, listed in both directions."
+ vals = {}
+ for line in f:
+ u, v, x = line.split()
+ if u < v:
+ vals[(u, v)] = float(x)
+ return vals
+
+def read_pop_edges(f):
+ for line in f:
+ u, rest = line.split(" -> ")
+ v, x = rest.rsplit(" ", 1)
+ yield u, v, float(x)
+
+@cache_network()
+@coerce_props()
+@annotate()
+def parse(alts=None):
+ global files
+ name = __name__.split(".")[-1]
+ for fnames, alt, fmt in files:
+ if alts is not None and alt not in alts:
+ continue
+ if alt.startswith("routers_"):
+ with open_upstream_file(name, fnames, "r") as f:
+ g = parse_cch(f)
+ g.gp.isp = g.new_gp("string", isp_names[int(alt.split("_")[1])])
+ elif alt.startswith("backbone_"):
+ with open_upstream_file(name, fnames[0], "r") as f:
+ weights = read_metric(f)
+ with open_upstream_file(name, fnames[1], "r") as f:
+ latencies = read_metric(f)
+ assert weights.keys() == latencies.keys()
+ g = Graph(directed=False)
+ g.ep.weight = g.new_ep("double")
+ g.ep.latency = g.new_ep("double")
+ g.vp.name = g.add_edge_list(((u, v, w, latencies[(u, v)])
+ for (u, v), w in weights.items()),
+ hashed=True, hash_type="string",
+ eprops=[g.ep.weight, g.ep.latency])
+ g.vp.location = g.new_vp("string")
+ for v in g.vertices():
+ g.vp.location[v] = re.sub(r"\d+$", "", g.vp.name[v]).replace("+", " ")
+ g.gp.isp = g.new_gp("string", isp_names[int(alt.split("_")[1])])
+ else:
+ # Each directory "x:y" of the archive holds the links of ISP x
+ # (x == y) or between ISPs x and y, each listed in both
+ # directions, with the same values
+ count, latency, weight = {}, {}, {}
+ with open_upstream_file(name, fnames, "rb") as f:
+ with tarfile.open(fileobj=f, mode="r|") as tf:
+ for m in tf:
+ if not m.isfile() or ":" not in m.name:
+ continue
+ kind = m.name.rsplit("/", 1)[1]
+ target = {"edges": count, "edges.lat": latency,
+ "edges.wt": weight}[kind]
+ lines = tf.extractfile(m).read().decode("utf8").splitlines()
+ for u, v, x in read_pop_edges(lines):
+ if u < v:
+ target[(u, v)] = x
+ assert latency.keys() == count.keys() and weight.keys() <= count.keys()
+ g = Graph(directed=False)
+ g.ep.count = g.new_ep("int")
+ g.ep.latency = g.new_ep("double")
+ g.ep.weight = g.new_ep("double")
+ g.vp.name = g.add_edge_list(((u, v, c, latency[(u, v)],
+ weight.get((u, v), numpy.nan))
+ for (u, v), c in sorted(count.items())),
+ hashed=True, hash_type="string",
+ eprops=[g.ep.count, g.ep.latency,
+ g.ep.weight])
+ g.vp.asn = g.new_vp("int")
+ g.vp.city = g.new_vp("string")
+ for v in g.vertices():
+ asn, city = g.vp.name[v].split(":", 1)
+ g.vp.asn[v] = int(asn)
+ g.vp.city[v] = city
+ g.ep.peering = g.new_ep("bool")
+ g.ep.peering.a = g.vp.asn.a[g.get_edges()[:, 0]] != g.vp.asn.a[g.get_edges()[:, 1]]
+ yield alt, g