logo Netzschleuder network catalog, repository, and centrifuge

History of citibike

The changes to the source code of this entry in the repository, newest first, each with the differences it made to the entry's files. They are also listed on GitLab.

[entries] Add the date when each entry was added, and remove icon_hash by Tiago de Paula Peixoto beff7c9
The dates are those of the commits that first added the entries' files
(2020-08-11 for the entries of the initial commit); renamed entries keep
the date of their first name.

Changed: none
diff --git a/entries/citibike/__init__.py b/entries/citibike/__init__.py
index d6b78c6..d122740 100644
--- a/entries/citibike/__init__.py
+++ b/entries/citibike/__init__.py
@@ -40,7 +40,7 @@ The networks have the following properties:
 tags = ['Transportation', 'Travel', 'Weighted', 'Metadata']
 url = ['https://citibikenyc.com/system-data', 'https://s3.amazonaws.com/tripdata/index.html']
 citation = [('Lyft Bikes and Scooters, "Citi Bike System Data." (2026)', 'https://citibikenyc.com/system-data')]
-icon_hash = None
+date_added = '2026-10-27'
 upstream_license = ("Citi Bike Data License Agreement (no redistribution as a stand-alone dataset)", "https://citibikenyc.com/data-sharing-policy")
 restricted = True
 
[entries] Prefix the properties in the descriptions with (Node), (Edge) or (Graph) by Tiago de Paula Peixoto bc3dbbd
The lists of properties at the end of the descriptions marked edge
properties with "Edge" but left node properties unmarked, or grouped
them in "Nodes: `a`, ...; `b`, ..." items, so the kind of a property
was often unclear. Every item now starts with (Node), (Edge) or (Graph),
followed by the property names, a colon and the explanation, one item
per property or group of properties; qualifications (kinds of nodes,
networks) follow the names in parentheses. Properties that were only
named got a short explanation, and a few items that named the wrong
kind of property were corrected.
diff --git a/entries/citibike/__init__.py b/entries/citibike/__init__.py
index ab07465..d6b78c6 100644
--- a/entries/citibike/__init__.py
+++ b/entries/citibike/__init__.py
@@ -32,8 +32,11 @@ The station-to-station networks are not planar spatial networks (trips may conne
 
 The networks have the following properties:
 
-- Nodes: `station_id`, the id of the station; `name`, its name (the most common one in the month); `lat` and `lon`, its geographic coordinates (NaN if unknown).
-- Edges: `weight`, the number of trips; `member_trips`, the number of trips by members, i.e. subscribers with an annual membership (the others are by casual users, with single rides or day passes)."""
+- (Node) `station_id`: the id of the station.
+- (Node) `name`: its name (the most common one in the month).
+- (Node) `lat` and `lon`: its geographic coordinates (NaN if unknown).
+- (Edge) `weight`: the number of trips.
+- (Edge) `member_trips`: the number of trips by members, i.e. subscribers with an annual membership (the others are by casual users, with single rides or day passes)."""
 tags = ['Transportation', 'Travel', 'Weighted', 'Metadata']
 url = ['https://citibikenyc.com/system-data', 'https://s3.amazonaws.com/tripdata/index.html']
 citation = [('Lyft Bikes and Scooters, "Citi Bike System Data." (2026)', 'https://citibikenyc.com/system-data')]
[entries] Remove unneeded code from the entries by Tiago de Paula Peixoto 2c69f19
Unneeded "global files" statements, unused imports and variables,
f-strings without placeholders, duplicate keys in foodweb_little_rock
(keeping the order and the values used), and the second definition of
get_lines() in twitter_higgs. Nothing changes in the networks or
descriptions.

Changed: none
diff --git a/entries/citibike/__init__.py b/entries/citibike/__init__.py
index a2a6d63..ab07465 100644
--- a/entries/citibike/__init__.py
+++ b/entries/citibike/__init__.py
@@ -196,7 +196,6 @@ def parse_trips(csvs):
 @coerce_props(vforce={"station_id": "string"})
 @annotate()
 def parse(alts=None):
-    global files
     name = __name__.split(".")[-1]
     for fname, alt, fmt in files:
         if alts is not None and alt not in alts:
[entries] Remove the remarks on the distribution of the data from the descriptions of beer_reviews, bluesky_failla, citibike, divvy, facebook100, goodreads, hiv_transmission, hollywood_actors, mind_news, movielens_10m_tags, movielens_32m, movielens_ratings, netflix_prize, opensky_flights, opensky_flights_full, regulondb, statsbomb, yelp_restaurants and yelp_reviews by Tiago de Paula Peixoto 8be09c8
The descriptions said whether the data may be distributed (access
tokens, restrictions, redistribution bans, take-downs); the site shows
restricted entries itself, and the descriptions now only describe the
data.
diff --git a/entries/citibike/__init__.py b/entries/citibike/__init__.py
index 229ecdc..a2a6d63 100644
--- a/entries/citibike/__init__.py
+++ b/entries/citibike/__init__.py
@@ -30,8 +30,6 @@ The station ids are numbers in the older files (for New York City until 2019, an
 
 The station-to-station networks are not planar spatial networks (trips may connect any two stations), so they are laid out as other networks, with the coordinates as vertex properties.
 
-The files of this entry are available only with an access token, since the Citi Bike Data License Agreement forbids redistributing the data as a stand-alone dataset.
-
 The networks have the following properties:
 
 - Nodes: `station_id`, the id of the station; `name`, its name (the most common one in the month); `lat` and `lon`, its geographic coordinates (NaN if unknown).
[entries] New entry: citibike by Tiago de Paula Peixoto 7c74f80
diff --git a/entries/citibike/__init__.py b/entries/citibike/__init__.py
new file mode 100644
index 0000000..229ecdc
--- /dev/null
+++ b/entries/citibike/__init__.py
@@ -0,0 +1,213 @@
+#! /usr/bin/env python
+# -*- coding: utf-8 -*-
+#
+# Copyright (C) 2026 Tiago de Paula Peixoto <tiago@skewed.de>
+#
+# This program is free software: you can redistribute it and/or modify it under
+# the terms of the GNU Affero General Public License as published by the Free
+# Software Foundation, either version 3 of the License, or (at your option) any
+# later version.
+#
+# This program is distributed in the hope that it will be useful, but WITHOUT
+# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
+# FOR A PARTICULAR PURPOSE.  See the GNU Affero General Public License for more
+# details.
+#
+# You should have received a copy of the GNU Affero General Public License along
+# with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+
+from .. import *
+import re
+from collections import Counter
+
+title = "Citi Bike trips (New York City and Jersey City, 2013-2026)"
+description = """Monthly networks of the trips made with Citi Bike, the bicycle sharing system of New York City (operated by Lyft), from the trip data published by the operator: one network for each month from June 2013, when the system opened, to September 2026, named `nyc_<year>_<month>` (e.g. `nyc_2013_06`, 160 networks), and one for each month of its extension to Jersey City and Hoboken (New Jersey), from September 2015 to September 2026, named `jc_<year>_<month>` (133 networks), whose trips are published separately.
+
+Each node is a station, and each directed edge goes from the station where a bicycle was taken out to the station where it was returned, weighted by the number of such trips in the month (trips that return to the same station are self-loops). The networks contain 328,053,007 trips in New York City and 6,995,957 in New Jersey. Left out are the 769,188 trips without a start or an end station (e.g. electric bicycles left away from a station), and the details of individual trips (times, durations, bicycles, and the birth year and gender of the riders, given in the older files).
+
+The station ids are numbers in the older files (for New York City until 2019, and for New Jersey until January 2021) and codes such as `5329.03` or `JC105` in the newer ones (the trips of 2020 in New York City were republished in the newer format), so the stations of different months can be matched by `station_id` only within each period. The coordinates of a station are the most common ones given for the trips that start or end there (in the newer files, those of electric bicycles are recorded by GPS). The yearly archives of the older files contain some months twice (the whole month, and the same trips split into several files); each trip is counted once.
+
+The station-to-station networks are not planar spatial networks (trips may connect any two stations), so they are laid out as other networks, with the coordinates as vertex properties.
+
+The files of this entry are available only with an access token, since the Citi Bike Data License Agreement forbids redistributing the data as a stand-alone dataset.
+
+The networks have the following properties:
+
+- Nodes: `station_id`, the id of the station; `name`, its name (the most common one in the month); `lat` and `lon`, its geographic coordinates (NaN if unknown).
+- Edges: `weight`, the number of trips; `member_trips`, the number of trips by members, i.e. subscribers with an annual membership (the others are by casual users, with single rides or day passes)."""
+tags = ['Transportation', 'Travel', 'Weighted', 'Metadata']
+url = ['https://citibikenyc.com/system-data', 'https://s3.amazonaws.com/tripdata/index.html']
+citation = [('Lyft Bikes and Scooters, "Citi Bike System Data." (2026)', 'https://citibikenyc.com/system-data')]
+icon_hash = None
+upstream_license = ("Citi Bike Data License Agreement (no redistribution as a stand-alone dataset)", "https://citibikenyc.com/data-sharing-policy")
+restricted = True
+
+upstream_prefix = 'https://s3.amazonaws.com/tripdata'
+
+def months(start, end):
+    y, m = start
+    while (y, m) <= end:
+        yield y, m
+        y, m = (y, m + 1) if m < 12 else (y + 1, 1)
+
+# Trips in New York City: yearly archives until 2023 (with the monthly files
+# inside, as CSV files in a folder per month until 2019, and as zip
+# archives from 2020), then one archive per month
+nyc_archives = {}
+for y, m in months((2013, 6), (2026, 9)):
+    nyc_archives[(y, m)] = (f'{y}-citibike-tripdata.zip' if y <= 2023 else
+                            f'{y}{m:02d}-citibike-tripdata.zip')
+# Trips in Jersey City and Hoboken: one archive per month (a few named
+# irregularly; the space in one of the names is written escaped)
+jc_irregular = {(2017, 8): 'JC-201708%20citibike-tripdata.csv.zip',
+                (2022, 7): 'JC-202207-citbike-tripdata.csv.zip',
+                (2025, 10): 'JC-202510-citibike-tripdata.zip',
+                (2026, 1): 'JC-202601-citibike-tripdata.zip',
+                (2026, 4): 'JC-202604-citibike-tripdata.zip'}
+jc_archives = {}
+for y, m in months((2015, 9), (2026, 9)):
+    jc_archives[(y, m)] = jc_irregular.get((y, m),
+                                           f'JC-{y}{m:02d}-citibike-tripdata.csv.zip')
+
+files = ([(a, f'nyc_{y}_{m:02d}', None) for (y, m), a in nyc_archives.items()] +
+         [(a, f'jc_{y}_{m:02d}', None) for (y, m), a in jc_archives.items()])
+
+def fetch_upstream(force=False):
+    return fetch_upstream_files(__name__.split(".")[-1], upstream_prefix, files,
+                                force)
+
+MONTH_FOLDERS = ['January', 'February', 'March', 'April', 'May', 'June', 'July',
+                 'August', 'September', 'October', 'November', 'December']
+
+def is_month_csv(member, y, m, monthly):
+    """Whether the archive member is a CSV file with trips of month (y, m).
+    All the CSV files of the monthly archives are (some are named
+    irregularly, e.g. "JC-20161-citibike-tripdata.csv"). In the yearly
+    archives until 2019, the trips of each month are in the CSV files of the
+    folder of the month (e.g. "6_June/"); the copies of some months at the
+    top level are ignored."""
+    base = member.rsplit("/", 1)[-1]
+    if member.startswith("__MACOSX") or base.startswith("."):
+        return False
+    if monthly:
+        return True
+    if f"{y}{m:02d}" not in base:
+        return False
+    if y <= 2019 and len(member.split("/")) > 1:
+        return f"/{m}_{MONTH_FOLDERS[m - 1]}/" in member
+    return True
+
+@contextmanager
+def open_zip(f):
+    "Open the zip archive in the binary file `f` (copied to a temporary file)."
+    os.makedirs("cache/temp", exist_ok=True)
+    with tempfile.NamedTemporaryFile(dir="cache/temp") as tmp:
+        shutil.copyfileobj(f, tmp, 1 << 24)
+        tmp.flush()
+        with zipfile.ZipFile(tmp.name) as z:
+            yield z
+
+def month_csvs(z, y, m, stack, monthly=False):
+    """The CSV files (opened in text mode) of the trips of month (y, m) in the
+    zip archive `z`, including those in zip archives inside it; `monthly` is
+    True if `z` contains only trips of that month."""
+    for member in sorted(z.namelist()):
+        base = member.rsplit("/", 1)[-1]
+        if member.startswith("__MACOSX") or base.startswith("."):
+            continue
+        if base.endswith(".zip") and f"{y}{m:02d}" in base:
+            inner = stack.enter_context(open_zip(z.open(member)))
+            yield from month_csvs(inner, y, m, stack, True)
+        elif base.endswith(".csv") and is_month_csv(member, y, m, monthly):
+            yield member, io.TextIOWrapper(z.open(member), "utf-8-sig",
+                                           errors="replace")
+
+# Normalized column names (lower case, without spaces and underscores) of
+# the two formats of the trip files: until January 2021, and since
+COLUMNS = {"start_id": ("startstationid",), "start_name": ("startstationname",),
+           "start_lat": ("startstationlatitude", "startlat"),
+           "start_lon": ("startstationlongitude", "startlng"),
+           "end_id": ("endstationid",), "end_name": ("endstationname",),
+           "end_lat": ("endstationlatitude", "endlat"),
+           "end_lon": ("endstationlongitude", "endlng"),
+           "user": ("usertype", "membercasual")}
+
+def parse_trips(csvs):
+    """Aggregate the trips in the CSV files `csvs` into a directed network of
+    stations, weighted by the number of trips."""
+    counts = defaultdict(lambda: [0, 0])
+    names = defaultdict(Counter)
+    coords = defaultdict(Counter)
+    nskipped = ntrips = 0
+    for member, f in csvs:
+        reader = csv.reader(f)
+        header = [re.sub(r"[\s_]", "", h.lower()) for h in next(reader)]
+        idx = {}
+        for k, cols in COLUMNS.items():
+            idx[k] = [header.index(c) for c in cols if c in header][0]
+        for row in reader:
+            if len(row) < len(header):
+                if not row:
+                    continue
+                nskipped += 1
+                continue
+            s, t = row[idx["start_id"]].strip(), row[idx["end_id"]].strip()
+            if s in ("", "NULL") or t in ("", "NULL"):
+                nskipped += 1
+                continue
+            s = re.sub(r"\.0+$", "", s) if s[-2:] == ".0" else s
+            t = re.sub(r"\.0+$", "", t) if t[-2:] == ".0" else t
+            c = counts[(s, t)]
+            c[0] += 1
+            if row[idx["user"]].strip().lower() in ("subscriber", "member"):
+                c[1] += 1
+            ntrips += 1
+            for v, p in ((s, "start"), (t, "end")):
+                names[v][row[idx[f"{p}_name"]].strip()] += 1
+                coords[v][(row[idx[f"{p}_lat"]], row[idx[f"{p}_lon"]])] += 1
+    g = Graph(directed=True)
+    g.vp.station_id = g.new_vp("string")
+    g.vp.name = g.new_vp("string")
+    g.vp.lat = g.new_vp("double")
+    g.vp.lon = g.new_vp("double")
+    vs = {}
+    for v in sorted(names):
+        u = vs[v] = int(g.add_vertex())
+        g.vp.station_id[u] = v
+        g.vp.name[u] = names[v].most_common(1)[0][0]
+        lat, lon = numpy.nan, numpy.nan
+        for (a, b), n in coords[v].most_common():
+            try:
+                lat, lon = float(a), float(b)
+            except ValueError:
+                continue
+            if lat != 0 and lon != 0:
+                break
+            lat, lon = numpy.nan, numpy.nan
+        g.vp.lat[u] = lat
+        g.vp.lon[u] = lon
+    g.ep.weight = g.new_ep("int")
+    g.ep.member_trips = g.new_ep("int")
+    g.add_edge_list(((vs[s], vs[t], c[0], c[1])
+                     for (s, t), c in sorted(counts.items())),
+                    eprops=[g.ep.weight, g.ep.member_trips])
+    return g, ntrips, nskipped
+
+@cache_network()
+@coerce_props(vforce={"station_id": "string"})
+@annotate()
+def parse(alts=None):
+    global files
+    name = __name__.split(".")[-1]
+    for fname, alt, fmt in files:
+        if alts is not None and alt not in alts:
+            continue
+        y, m = int(alt[-7:-3]), int(alt[-2:])
+        with ExitStack() as stack:
+            f = stack.enter_context(open_upstream_file(name, fname, "rb"))
+            z = stack.enter_context(zipfile.ZipFile(f))
+            monthly = not re.match(r"\d{4}-citibike", fname)
+            g, ntrips, nskipped = parse_trips(month_csvs(z, y, m, stack,
+                                                         monthly))
+        yield alt, g