logo Netzschleuder network catalog, repository, and centrifuge

History of nyc_taxi_trips

The changes to the source code of this entry in the repository, newest first, each with the differences it made to the entry's files. They are also listed on GitLab.

[entries] Remove superfluous details of the upstream distribution from the descriptions by Tiago de Paula Peixoto dc07aa0
The descriptions mentioned incidental details of how the upstream data
are made available: the file formats (CSV, JSON, XML, GEXF, SBML, HIF,
Pajek, UCINET, ...), file names, where the files were published or
obtained from (Zenodo, figshare, Kaggle, the Pajek datasets page, the
Internet Archive), and that some networks are only available in the gt
format. These are left out; caveats about the data themselves are kept,
referring to "the data" instead of "the file". The description of
sp_baboons now lists its properties instead of the upstream columns,
and says that events without a recipient are left out.
diff --git a/entries/nyc_taxi_trips/__init__.py b/entries/nyc_taxi_trips/__init__.py
index 03a622a..aed8899 100644
--- a/entries/nyc_taxi_trips/__init__.py
+++ b/entries/nyc_taxi_trips/__init__.py
@@ -30,7 +30,7 @@ description = r"""Daily flows of the trips of taxis and for-hire vehicles (ride-
 - `fhv_<year>` (2015 to 2025): for-hire vehicles dispatched by bases (car services, black cars, limousines; before February 2019 these include the trips of the ride-hailing companies, which were then separated into the next category). The zones of many of these trips are not known (see below), and their drop-off times are missing in some years.
 - `fhvhv_<year>` (February 2019 to 2025): high-volume for-hire services, i.e. the ride-hailing companies with more than 10,000 trips per day (Uber, Lyft, Via and Juno).
 
-Nodes are the taxi zones (all 265 zones of the TLC in every network, including the two pseudo-zones for unknown locations and for locations outside the city), and a directed edge from zone $i$ to zone $j$ aggregates the trips picked up in $i$ and dropped off in $j$ on the same day (the day of the pickup, in local time), so that two zones are connected by one edge per day with such trips (a multigraph), with the number of trips and their totals of distance, duration, passengers and fares; in `fhvhv_<year>`, the trips of each company are aggregated separately. Individual trips (hundreds of millions per year) are not kept. Trips with a pickup time outside the year of the network (data errors, or trips of the last day of the previous year) are left out. Trips whose pickup or drop-off zone is missing in the data (null, or 0 in some files, and -1 in a few `fhv` trips of December 2017; a large fraction of the `fhv` trips, whose drop-off zones are not recorded before 2017) are assigned to the zone of unknown locations (264), as the TLC does in the other records; otherwise the records are taken as they are, including obviously erroneous values (e.g. negative durations or fares). The yellow taxi records of 2009 and 2010, which give the pickup and drop-off coordinates instead of the zones, are not included; neither are the payment types, rate codes, tolls, surcharges and taxes, the vendors and dispatching bases, and the request flags of the trips. The zones are polygons (neighborhoods, airports, parks), so that the networks are not spatial in the sense of nearly planar networks: they are dense networks of flows. The monthly files of the TLC are republished in place when they are corrected, so the data are those of the time of download.
+Nodes are the taxi zones (all 265 zones of the TLC in every network, including the two pseudo-zones for unknown locations and for locations outside the city), and a directed edge from zone $i$ to zone $j$ aggregates the trips picked up in $i$ and dropped off in $j$ on the same day (the day of the pickup, in local time), so that two zones are connected by one edge per day with such trips (a multigraph), with the number of trips and their totals of distance, duration, passengers and fares; in `fhvhv_<year>`, the trips of each company are aggregated separately. Individual trips (hundreds of millions per year) are not kept. Trips with a pickup time outside the year of the network (data errors, or trips of the last day of the previous year) are left out. Trips whose pickup or drop-off zone is missing in the data (null, or 0 in part of the data, and -1 in a few `fhv` trips of December 2017; a large fraction of the `fhv` trips, whose drop-off zones are not recorded before 2017) are assigned to the zone of unknown locations (264), as the TLC does in the other records; otherwise the records are taken as they are, including obviously erroneous values (e.g. negative durations or fares). The yellow taxi records of 2009 and 2010, which give the pickup and drop-off coordinates instead of the zones, are not included; neither are the payment types, rate codes, tolls, surcharges and taxes, the vendors and dispatching bases, and the request flags of the trips. The zones are polygons (neighborhoods, airports, parks), so that the networks are not spatial in the sense of nearly planar networks: they are dense networks of flows. The TLC corrects its records retroactively, so the data are those of the time of download.
 
 The networks have the following properties:
 
[entries] Add the date when each entry was added, and remove icon_hash by Tiago de Paula Peixoto beff7c9
The dates are those of the commits that first added the entries' files
(2020-08-11 for the entries of the initial commit); renamed entries keep
the date of their first name.

Changed: none
diff --git a/entries/nyc_taxi_trips/__init__.py b/entries/nyc_taxi_trips/__init__.py
index 7ede87c..03a622a 100644
--- a/entries/nyc_taxi_trips/__init__.py
+++ b/entries/nyc_taxi_trips/__init__.py
@@ -51,7 +51,7 @@ tags = ['Transportation', 'Travel', 'Multigraph', 'Weighted', 'Timestamps', 'Tem
 url = ['https://www.nyc.gov/site/tlc/about/tlc-trip-record-data.page',
        'https://d37ci6vzurychx.cloudfront.net/trip-data/']
 citation = [('NYC Taxi and Limousine Commission, "TLC Trip Record Data." (2011-2025)', 'https://www.nyc.gov/site/tlc/about/tlc-trip-record-data.page')]
-icon_hash = None
+date_added = '2026-10-27'
 upstream_license = ("NYC Open Data Terms of Use", "https://www.nyc.gov/home/terms-of-use.page")
 
 upstream_prefix = 'https://d37ci6vzurychx.cloudfront.net'
[entries] Prefix the properties in the descriptions with (Node), (Edge) or (Graph) by Tiago de Paula Peixoto bc3dbbd
The lists of properties at the end of the descriptions marked edge
properties with "Edge" but left node properties unmarked, or grouped
them in "Nodes: `a`, ...; `b`, ..." items, so the kind of a property
was often unclear. Every item now starts with (Node), (Edge) or (Graph),
followed by the property names, a colon and the explanation, one item
per property or group of properties; qualifications (kinds of nodes,
networks) follow the names in parentheses. Properties that were only
named got a short explanation, and a few items that named the wrong
kind of property were corrected.
diff --git a/entries/nyc_taxi_trips/__init__.py b/entries/nyc_taxi_trips/__init__.py
index 1d6d777..7ede87c 100644
--- a/entries/nyc_taxi_trips/__init__.py
+++ b/entries/nyc_taxi_trips/__init__.py
@@ -34,19 +34,19 @@ Nodes are the taxi zones (all 265 zones of the TLC in every network, including t
 
 The networks have the following properties:
 
-- `location_id`: the number of the taxi zone (from 1 to 265; 264 is unknown, 265 outside New York City).
-- `zone`: the name of the zone (e.g. `JFK Airport`, `Upper East Side North`).
-- `borough`: the borough of the zone (`Manhattan`, `Brooklyn`, `Queens`, `Bronx`, `Staten Island`, `EWR` for Newark Airport).
-- `service_zone`: the service zone (`Yellow Zone` for central Manhattan, `Boro Zone` elsewhere, `Airports`, `EWR`).
-- Edge `time`: the day of the pickups, as the Unix time of its beginning (midnight in New York).
-- Edge `trips`: the number of trips.
-- Edge `duration`: the total duration of the trips (from pickup to drop-off) with a recorded drop-off time, in seconds (-1 if none of the trips has one, as in the `fhv` records of 2015 and 2016).
-- Edge `distance` (not `fhv`): the total distance of the trips, in miles.
-- Edge `passengers` (`yellow` and `green` only): the total number of passengers, as entered by the drivers (missing counts are taken as zero).
-- Edge `fare` (not `fhv`): the total of the fares (for `fhvhv`, the base passenger fares), in US dollars, before tolls, surcharges, taxes and tips.
-- Edge `tips` (not `fhv`): the total of the tips, in US dollars (for `yellow` and `green`, only tips paid by credit card are recorded).
-- Edge `total_amount` (`yellow` and `green` only): the total amount charged to the passengers, in US dollars.
-- Edge `company` (`fhvhv` only): the ride-hailing company of the trips (`Uber`, `Lyft`, `Via` or `Juno`)."""
+- (Node) `location_id`: the number of the taxi zone (from 1 to 265; 264 is unknown, 265 outside New York City).
+- (Node) `zone`: the name of the zone (e.g. `JFK Airport`, `Upper East Side North`).
+- (Node) `borough`: the borough of the zone (`Manhattan`, `Brooklyn`, `Queens`, `Bronx`, `Staten Island`, `EWR` for Newark Airport).
+- (Node) `service_zone`: the service zone (`Yellow Zone` for central Manhattan, `Boro Zone` elsewhere, `Airports`, `EWR`).
+- (Edge) `time`: the day of the pickups, as the Unix time of its beginning (midnight in New York).
+- (Edge) `trips`: the number of trips.
+- (Edge) `duration`: the total duration of the trips (from pickup to drop-off) with a recorded drop-off time, in seconds (-1 if none of the trips has one, as in the `fhv` records of 2015 and 2016).
+- (Edge) `distance` (not `fhv`): the total distance of the trips, in miles.
+- (Edge) `passengers` (`yellow` and `green` only): the total number of passengers, as entered by the drivers (missing counts are taken as zero).
+- (Edge) `fare` (not `fhv`): the total of the fares (for `fhvhv`, the base passenger fares), in US dollars, before tolls, surcharges, taxes and tips.
+- (Edge) `tips` (not `fhv`): the total of the tips, in US dollars (for `yellow` and `green`, only tips paid by credit card are recorded).
+- (Edge) `total_amount` (`yellow` and `green` only): the total amount charged to the passengers, in US dollars.
+- (Edge) `company` (`fhvhv` only): the ride-hailing company of the trips (`Uber`, `Lyft`, `Via` or `Juno`)."""
 tags = ['Transportation', 'Travel', 'Multigraph', 'Weighted', 'Timestamps', 'Temporal', 'Metadata']
 url = ['https://www.nyc.gov/site/tlc/about/tlc-trip-record-data.page',
        'https://d37ci6vzurychx.cloudfront.net/trip-data/']
[entries] nyc_taxi_trips: count the negative zones of a few trips as unknown by Tiago de Paula Peixoto f6afd25
Four fhv trips of December 2017 have the pickup zone -1, which failed
the check of the zones. Like the missing zones (null, NaN or 0), they
are assigned to the zone of unknown locations (264).
diff --git a/entries/nyc_taxi_trips/__init__.py b/entries/nyc_taxi_trips/__init__.py
index 73e0a91..1d6d777 100644
--- a/entries/nyc_taxi_trips/__init__.py
+++ b/entries/nyc_taxi_trips/__init__.py
@@ -30,7 +30,7 @@ description = r"""Daily flows of the trips of taxis and for-hire vehicles (ride-
 - `fhv_<year>` (2015 to 2025): for-hire vehicles dispatched by bases (car services, black cars, limousines; before February 2019 these include the trips of the ride-hailing companies, which were then separated into the next category). The zones of many of these trips are not known (see below), and their drop-off times are missing in some years.
 - `fhvhv_<year>` (February 2019 to 2025): high-volume for-hire services, i.e. the ride-hailing companies with more than 10,000 trips per day (Uber, Lyft, Via and Juno).
 
-Nodes are the taxi zones (all 265 zones of the TLC in every network, including the two pseudo-zones for unknown locations and for locations outside the city), and a directed edge from zone $i$ to zone $j$ aggregates the trips picked up in $i$ and dropped off in $j$ on the same day (the day of the pickup, in local time), so that two zones are connected by one edge per day with such trips (a multigraph), with the number of trips and their totals of distance, duration, passengers and fares; in `fhvhv_<year>`, the trips of each company are aggregated separately. Individual trips (hundreds of millions per year) are not kept. Trips with a pickup time outside the year of the network (data errors, or trips of the last day of the previous year) are left out. Trips whose pickup or drop-off zone is missing in the data (null, or 0 in some files; a large fraction of the `fhv` trips, whose drop-off zones are not recorded before 2017) are assigned to the zone of unknown locations (264), as the TLC does in the other records; otherwise the records are taken as they are, including obviously erroneous values (e.g. negative durations or fares). The yellow taxi records of 2009 and 2010, which give the pickup and drop-off coordinates instead of the zones, are not included; neither are the payment types, rate codes, tolls, surcharges and taxes, the vendors and dispatching bases, and the request flags of the trips. The zones are polygons (neighborhoods, airports, parks), so that the networks are not spatial in the sense of nearly planar networks: they are dense networks of flows. The monthly files of the TLC are republished in place when they are corrected, so the data are those of the time of download.
+Nodes are the taxi zones (all 265 zones of the TLC in every network, including the two pseudo-zones for unknown locations and for locations outside the city), and a directed edge from zone $i$ to zone $j$ aggregates the trips picked up in $i$ and dropped off in $j$ on the same day (the day of the pickup, in local time), so that two zones are connected by one edge per day with such trips (a multigraph), with the number of trips and their totals of distance, duration, passengers and fares; in `fhvhv_<year>`, the trips of each company are aggregated separately. Individual trips (hundreds of millions per year) are not kept. Trips with a pickup time outside the year of the network (data errors, or trips of the last day of the previous year) are left out. Trips whose pickup or drop-off zone is missing in the data (null, or 0 in some files, and -1 in a few `fhv` trips of December 2017; a large fraction of the `fhv` trips, whose drop-off zones are not recorded before 2017) are assigned to the zone of unknown locations (264), as the TLC does in the other records; otherwise the records are taken as they are, including obviously erroneous values (e.g. negative durations or fares). The yellow taxi records of 2009 and 2010, which give the pickup and drop-off coordinates instead of the zones, are not included; neither are the payment types, rate codes, tolls, surcharges and taxes, the vendors and dispatching bases, and the request flags of the trips. The zones are polygons (neighborhoods, airports, parks), so that the networks are not spatial in the sense of nearly planar networks: they are dense networks of flows. The monthly files of the TLC are republished in place when they are corrected, so the data are those of the time of download.
 
 The networks have the following properties:
 
@@ -116,7 +116,7 @@ def _col(names, *cands):
 def _zone(c):
     import polars as pl
     c = c.cast(pl.Float64).fill_nan(None).fill_null(264)
-    return pl.when(c == 0).then(264).otherwise(c).cast(pl.Int32)
+    return pl.when(c <= 0).then(264).otherwise(c).cast(pl.Int32)
 
 def _read_month(path, service, year):
     "The trips of a monthly file, aggregated by pickup and drop-off zone and day."
@@ -129,7 +129,7 @@ def _read_month(path, service, year):
     tgt = _col(names, "DOLocationID")
     cols = {"pu": pl.col(pu).cast(pl.Datetime("us")),
             "do": pl.col(do).cast(pl.Datetime("us")),
-            # missing zones (null, NaN or 0 in some files) are the zone of
+            # missing zones (null, NaN, 0 or -1 in some files) are the zone of
             # unknown locations, 264
             "s": _zone(pl.col(src)), "t": _zone(pl.col(tgt))}
     aggs = [pl.len().alias("trips"),
[entries] New entry: nyc_taxi_trips by Tiago de Paula Peixoto 03bb746
diff --git a/entries/nyc_taxi_trips/__init__.py b/entries/nyc_taxi_trips/__init__.py
new file mode 100644
index 0000000..73e0a91
--- /dev/null
+++ b/entries/nyc_taxi_trips/__init__.py
@@ -0,0 +1,219 @@
+#! /usr/bin/env python
+# -*- coding: utf-8 -*-
+#
+# Copyright (C) 2026 Tiago de Paula Peixoto <tiago@skewed.de>
+#
+# This program is free software: you can redistribute it and/or modify it under
+# the terms of the GNU Affero General Public License as published by the Free
+# Software Foundation, either version 3 of the License, or (at your option) any
+# later version.
+#
+# This program is distributed in the hope that it will be useful, but WITHOUT
+# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
+# FOR A PARTICULAR PURPOSE.  See the GNU Affero General Public License for more
+# details.
+#
+# You should have received a copy of the GNU Affero General Public License along
+# with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import time
+import urllib.error
+
+from .. import *
+import process_entry
+
+title = "New York City taxi and ride-hailing trips (2011-2025)"
+description = r"""Daily flows of the trips of taxis and for-hire vehicles (ride-hailing services such as Uber and Lyft, car services, limousines) in New York City, between the 263 taxi zones of the city, from the trip records published by the NYC Taxi and Limousine Commission (TLC). The records are collected from the taximeters and the dispatching companies under the rules of the TLC, and contain the pickup and drop-off times and zones of each trip, and, depending on the service, the distance, fare and number of passengers. There is one network per service and calendar year:
+
+- `yellow_<year>` (2011 to 2025): yellow taxis ("medallion" cabs), which pick up passengers hailing them on the street anywhere in the city.
+- `green_<year>` (2014 to 2025): green taxis ("boro" cabs), which can pick up hailing passengers only outside central Manhattan and the airports.
+- `fhv_<year>` (2015 to 2025): for-hire vehicles dispatched by bases (car services, black cars, limousines; before February 2019 these include the trips of the ride-hailing companies, which were then separated into the next category). The zones of many of these trips are not known (see below), and their drop-off times are missing in some years.
+- `fhvhv_<year>` (February 2019 to 2025): high-volume for-hire services, i.e. the ride-hailing companies with more than 10,000 trips per day (Uber, Lyft, Via and Juno).
+
+Nodes are the taxi zones (all 265 zones of the TLC in every network, including the two pseudo-zones for unknown locations and for locations outside the city), and a directed edge from zone $i$ to zone $j$ aggregates the trips picked up in $i$ and dropped off in $j$ on the same day (the day of the pickup, in local time), so that two zones are connected by one edge per day with such trips (a multigraph), with the number of trips and their totals of distance, duration, passengers and fares; in `fhvhv_<year>`, the trips of each company are aggregated separately. Individual trips (hundreds of millions per year) are not kept. Trips with a pickup time outside the year of the network (data errors, or trips of the last day of the previous year) are left out. Trips whose pickup or drop-off zone is missing in the data (null, or 0 in some files; a large fraction of the `fhv` trips, whose drop-off zones are not recorded before 2017) are assigned to the zone of unknown locations (264), as the TLC does in the other records; otherwise the records are taken as they are, including obviously erroneous values (e.g. negative durations or fares). The yellow taxi records of 2009 and 2010, which give the pickup and drop-off coordinates instead of the zones, are not included; neither are the payment types, rate codes, tolls, surcharges and taxes, the vendors and dispatching bases, and the request flags of the trips. The zones are polygons (neighborhoods, airports, parks), so that the networks are not spatial in the sense of nearly planar networks: they are dense networks of flows. The monthly files of the TLC are republished in place when they are corrected, so the data are those of the time of download.
+
+The networks have the following properties:
+
+- `location_id`: the number of the taxi zone (from 1 to 265; 264 is unknown, 265 outside New York City).
+- `zone`: the name of the zone (e.g. `JFK Airport`, `Upper East Side North`).
+- `borough`: the borough of the zone (`Manhattan`, `Brooklyn`, `Queens`, `Bronx`, `Staten Island`, `EWR` for Newark Airport).
+- `service_zone`: the service zone (`Yellow Zone` for central Manhattan, `Boro Zone` elsewhere, `Airports`, `EWR`).
+- Edge `time`: the day of the pickups, as the Unix time of its beginning (midnight in New York).
+- Edge `trips`: the number of trips.
+- Edge `duration`: the total duration of the trips (from pickup to drop-off) with a recorded drop-off time, in seconds (-1 if none of the trips has one, as in the `fhv` records of 2015 and 2016).
+- Edge `distance` (not `fhv`): the total distance of the trips, in miles.
+- Edge `passengers` (`yellow` and `green` only): the total number of passengers, as entered by the drivers (missing counts are taken as zero).
+- Edge `fare` (not `fhv`): the total of the fares (for `fhvhv`, the base passenger fares), in US dollars, before tolls, surcharges, taxes and tips.
+- Edge `tips` (not `fhv`): the total of the tips, in US dollars (for `yellow` and `green`, only tips paid by credit card are recorded).
+- Edge `total_amount` (`yellow` and `green` only): the total amount charged to the passengers, in US dollars.
+- Edge `company` (`fhvhv` only): the ride-hailing company of the trips (`Uber`, `Lyft`, `Via` or `Juno`)."""
+tags = ['Transportation', 'Travel', 'Multigraph', 'Weighted', 'Timestamps', 'Temporal', 'Metadata']
+url = ['https://www.nyc.gov/site/tlc/about/tlc-trip-record-data.page',
+       'https://d37ci6vzurychx.cloudfront.net/trip-data/']
+citation = [('NYC Taxi and Limousine Commission, "TLC Trip Record Data." (2011-2025)', 'https://www.nyc.gov/site/tlc/about/tlc-trip-record-data.page')]
+icon_hash = None
+upstream_license = ("NYC Open Data Terms of Use", "https://www.nyc.gov/home/terms-of-use.page")
+
+upstream_prefix = 'https://d37ci6vzurychx.cloudfront.net'
+zone_file = "misc/taxi_zone_lookup.csv"
+services = {"yellow": range(2011, 2026), "green": range(2014, 2026),
+            "fhv": range(2015, 2026), "fhvhv": range(2019, 2026)}
+
+def _months(service, year):
+    # the high-volume services are separate from February 2019
+    first = 2 if (service, year) == ("fhvhv", 2019) else 1
+    return [f"trip-data/{service}_tripdata_{year}-{m:02d}.parquet"
+            for m in range(first, 13)]
+
+files = [(tuple(_months(s, y)) + (zone_file,), f"{s}_{y}", None)
+         for s, ys in services.items() for y in ys]
+
+companies = {"HV0002": "Juno", "HV0003": "Uber", "HV0004": "Via",
+             "HV0005": "Lyft"}
+
+def _upstream_dir():
+    return f"cache/upstream/{__name__.split('.')[-1]}"
+
+def _get(url, target, tries=6, pause=60):
+    # The host (CloudFront) answers 403 for a while after many requests
+    for i in range(tries):
+        try:
+            with open_url(url) as r, open(target + ".part", "wb") as out:
+                shutil.copyfileobj(r, out, 1 << 24)
+            os.replace(target + ".part", target)
+            return
+        except urllib.error.HTTPError as e:
+            if e.code not in (403, 429, 500, 502, 503, 504) or i == tries - 1:
+                raise
+        time.sleep(pause * (i + 1))
+
+def fetch_upstream(force=False):
+    name = __name__.split(".")[-1]
+    fetch_upstream_files(name, upstream_prefix, [((zone_file,), None, None)],
+                         force)
+    # The parquet files (already compressed) are stored as they are
+    months = [f for fnames, alt, fmt in files for f in fnames[:-1]]
+    if process_entry._check_health:
+        for f in process_entry.health_sample(months):
+            with open_url(f"{upstream_prefix}/{f}") as r:
+                r.read(1)
+        return
+    for f in months:
+        target = f"{_upstream_dir()}/{f}"
+        if not force and os.path.exists(target):
+            continue
+        os.makedirs(os.path.dirname(target), exist_ok=True)
+        _get(f"{upstream_prefix}/{f}", target)
+
+def _col(names, *cands):
+    "The column among `names` that matches one of `cands`, ignoring case."
+    low = {n.lower(): n for n in names}
+    for c in cands:
+        if c.lower() in low:
+            return low[c.lower()]
+    return None
+
+def _zone(c):
+    import polars as pl
+    c = c.cast(pl.Float64).fill_nan(None).fill_null(264)
+    return pl.when(c == 0).then(264).otherwise(c).cast(pl.Int32)
+
+def _read_month(path, service, year):
+    "The trips of a monthly file, aggregated by pickup and drop-off zone and day."
+    import polars as pl
+    import pyarrow.parquet as pq
+    names = pq.read_schema(path).names
+    pu = _col(names, "tpep_pickup_datetime", "lpep_pickup_datetime", "pickup_datetime")
+    do = _col(names, "tpep_dropoff_datetime", "lpep_dropoff_datetime", "dropoff_datetime")
+    src = _col(names, "PULocationID")
+    tgt = _col(names, "DOLocationID")
+    cols = {"pu": pl.col(pu).cast(pl.Datetime("us")),
+            "do": pl.col(do).cast(pl.Datetime("us")),
+            # missing zones (null, NaN or 0 in some files) are the zone of
+            # unknown locations, 264
+            "s": _zone(pl.col(src)), "t": _zone(pl.col(tgt))}
+    aggs = [pl.len().alias("trips"),
+            pl.col("dur").sum().alias("duration"),
+            pl.col("dur").count().alias("timed")]
+    keys = ["s", "t", "day"]
+    if service in ("yellow", "green"):
+        cols.update(distance=pl.col(_col(names, "trip_distance")).cast(pl.Float64),
+                    passengers=pl.col(_col(names, "passenger_count")).cast(pl.Float64),
+                    fare=pl.col(_col(names, "fare_amount")).cast(pl.Float64),
+                    tips=pl.col(_col(names, "tip_amount")).cast(pl.Float64),
+                    total_amount=pl.col(_col(names, "total_amount")).cast(pl.Float64))
+    elif service == "fhvhv":
+        cols.update(distance=pl.col(_col(names, "trip_miles")).cast(pl.Float64),
+                    fare=pl.col(_col(names, "base_passenger_fare")).cast(pl.Float64),
+                    tips=pl.col(_col(names, "tips")).cast(pl.Float64),
+                    company=pl.col(_col(names, "hvfhs_license_num")))
+        keys.append("company")
+    for c in ["distance", "passengers", "fare", "tips", "total_amount"]:
+        if c in cols:
+            aggs.append(pl.col(c).sum().alias(c))
+    df = (pl.scan_parquet(path)
+          .select(**cols)
+          .filter(pl.col("pu").dt.year() == year)
+          .with_columns(day=pl.col("pu").dt.date(),
+                        # drop-off times are missing (1989-01-01) in some files
+                        dur=pl.when(pl.col("do").dt.year() >= 2000)
+                              .then((pl.col("do") - pl.col("pu")).dt.total_seconds()))
+          .group_by(keys)
+          .agg(aggs)
+          .collect())
+    return df
+
+@cache_network()
+@coerce_props(eforce={"company": "string"})
+@annotate()
+def parse(alts=None):
+    import polars as pl
+    name = __name__.split(".")[-1]
+    zones = None
+    for fnames, alt, fmt in files:
+        if alts is not None and alt not in alts:
+            continue
+        if zones is None:
+            with open_upstream_file(name, zone_file, "rb") as f:
+                zones = pl.read_csv(f, infer_schema_length=None)
+            ids = zones["LocationID"].to_numpy()
+            if not numpy.array_equal(ids, numpy.arange(1, len(ids) + 1)):
+                raise ValueError("unexpected zone numbers")
+        service, year = alt.rsplit("_", 1)
+        year = int(year)
+        months = [_read_month(f"{_upstream_dir()}/{f}", service, year)
+                  for f in fnames[:-1]]
+        keys = ["s", "t", "day"] + (["company"] if service == "fhvhv" else [])
+        sums = [c for c in months[0].columns if c not in keys]
+        # trips of the same day may be in two monthly files
+        df = (pl.concat(months).group_by(keys).agg([pl.col(c).sum() for c in sums])
+              .sort(["day", "s", "t"] + keys[3:]))
+        del months
+        s, t = df["s"].to_numpy(), df["t"].to_numpy()
+        if min(s.min(), t.min()) < 1 or max(s.max(), t.max()) > len(ids):
+            raise ValueError(f"unknown zones in {alt}")
+
+        g = Graph(directed=True)
+        g.add_vertex(len(ids))
+        g.vp.location_id = g.new_vp("int32_t", vals=ids)
+        g.vp.zone = g.new_vp("string", vals=zones["Zone"].fill_null("").to_list())
+        g.vp.borough = g.new_vp("string", vals=zones["Borough"].fill_null("").to_list())
+        g.vp.service_zone = g.new_vp("string",
+                                     vals=zones["service_zone"].fill_null("").to_list())
+        g.add_edge_list(numpy.column_stack((s - 1, t - 1)))
+        day = (df["day"].cast(pl.Datetime("us"))
+               .dt.replace_time_zone("America/New_York", ambiguous="earliest")
+               .dt.epoch("s"))
+        g.ep.time = g.new_ep("int64_t", vals=day.to_numpy())
+        g.ep.trips = g.new_ep("int64_t", vals=df["trips"].to_numpy())
+        dur = df["duration"].to_numpy().copy()
+        dur[df["timed"].to_numpy() == 0] = -1
+        g.ep.duration = g.new_ep("int64_t", vals=dur)
+        for c in ["distance", "passengers", "fare", "tips", "total_amount"]:
+            if c in df.columns:
+                g.ep[c] = g.new_ep("double", vals=df[c].fill_null(0).to_numpy())
+        if service == "fhvhv":
+            g.ep.company = g.new_ep("string",
+                                    vals=[companies.get(c, c) for c in df["company"].to_list()])
+        del df
+        yield alt, g