History of fec_contributions
The changes to the source code of this entry in the repository, newest first, each with the differences it made to the entry's files. They are also listed on GitLab.
[entries] Remove superfluous details of the upstream distribution from the descriptions dc07aa0
The descriptions mentioned incidental details of how the upstream data are made available: the file formats (CSV, JSON, XML, GEXF, SBML, HIF, Pajek, UCINET, ...), file names, where the files were published or obtained from (Zenodo, figshare, Kaggle, the Pajek datasets page, the Internet Archive), and that some networks are only available in the gt format. These are left out; caveats about the data themselves are kept, referring to "the data" instead of "the file". The description of sp_baboons now lists its properties instead of the upstream columns, and says that events without a recipient are left out.
diff --git a/entries/fec_contributions/__init__.py b/entries/fec_contributions/__init__.py
index de893a0..2e2530b 100644
--- a/entries/fec_contributions/__init__.py
+++ b/entries/fec_contributions/__init__.py
@@ -22,9 +22,9 @@ import pyarrow
import pyarrow.csv
title = "US campaign contributions of individuals to political committees (FEC, 1980-2026)"
-description = r"""Bipartite networks of the contributions of individuals to the political committees of US federal elections (the campaign committees of candidates, party committees and political action committees), from the bulk data of the US Federal Election Commission (FEC). The committees must report to the FEC the contributions they receive from individuals that exceed 200 dollars in an election cycle (and many report smaller ones too, in particular the conduit committees, such as ActBlue and WinRed, through which individuals give to candidates); the file of "contributions by individuals" contains these itemized receipts, as reported, for each two-year election cycle. There is one network per cycle, from `1980` (1979-1980) to `2026` (2025-2026, as of the download date, October 2026; the files of the recent cycles are updated by the FEC as new reports are filed).
+description = r"""Bipartite networks of the contributions of individuals to the political committees of US federal elections (the campaign committees of candidates, party committees and political action committees), from the bulk data of the US Federal Election Commission (FEC). The committees must report to the FEC the contributions they receive from individuals that exceed 200 dollars in an election cycle (and many report smaller ones too, in particular the conduit committees, such as ActBlue and WinRed, through which individuals give to candidates); the data of "contributions by individuals" contain these itemized receipts, as reported, for each two-year election cycle. There is one network per cycle, from `1980` (1979-1980) to `2026` (2025-2026, as of the download date, October 2026; the data of the recent cycles are updated by the FEC as new reports are filed).
-The nodes are the contributors and the committees, and an edge from a contributor to a committee aggregates all the reported receipts of the committee from the contributor in the cycle, with their number and total amount (since the files of the recent cycles have more than 10^8 rows, the individual receipts are not kept as edges). The committees are identified by their FEC id, and described by the committee master file of the cycle. The FEC gives no identifier of the contributors, only their name, address, employer and occupation as reported by each committee: the contributors are therefore merged by their name (upper-cased, with spaces normalized), the first five digits of their ZIP code and their employer (upper-cased) together, so that the same person with differently written names or employers (or who moved) is split into several nodes, and different persons with the same name, ZIP code and employer are merged. The file also contains some contributions of organizations and other entities, which are treated in the same way (see `entity_type`).
+The nodes are the contributors and the committees, and an edge from a contributor to a committee aggregates all the reported receipts of the committee from the contributor in the cycle, with their number and total amount (since the recent cycles have more than 10^8 receipts, the individual receipts are not kept as edges). The committees are identified by their FEC id, and described by the committee master file of the cycle. The FEC gives no identifier of the contributors, only their name, address, employer and occupation as reported by each committee: the contributors are therefore merged by their name (upper-cased, with spaces normalized), the first five digits of their ZIP code and their employer (upper-cased) together, so that the same person with differently written names or employers (or who moved) is split into several nodes, and different persons with the same name, ZIP code and employer are merged. The data also contain some contributions of organizations and other entities, which are treated in the same way (see `entity_type`).
The receipts are as reported, including refunds (negative amounts) and the memo entries (`memo` code `X`), which are not counted in the totals of the reports: e.g. the receipts that a candidate committee reports as earmarked through a conduit committee also appear as receipts of the conduit, so that the same money can appear on two edges. Their numbers and amounts are given separately on the edges. Note that the law (52 U.S.C. 30111(a)(4)) forbids the use of the names and addresses of contributors from these data for soliciting contributions or for commercial purposes.
[entries] Add the date when each entry was added, and remove icon_hash beff7c9
The dates are those of the commits that first added the entries' files (2020-08-11 for the entries of the initial commit); renamed entries keep the date of their first name. Changed: none
diff --git a/entries/fec_contributions/__init__.py b/entries/fec_contributions/__init__.py
index 2b6fc58..de893a0 100644
--- a/entries/fec_contributions/__init__.py
+++ b/entries/fec_contributions/__init__.py
@@ -45,7 +45,7 @@ tags = ['Social', 'Political', 'Economic', 'Bipartite', 'Weighted', 'Metadata']
url = ['https://www.fec.gov/data/browse-data/?tab=bulk-data']
citation = [('Federal Election Commission, "Contributions by individuals" and "Committee master" bulk data files (1980-2026)',
'https://www.fec.gov/data/browse-data/?tab=bulk-data')]
-icon_hash = None
+date_added = '2026-10-31'
upstream_license = ("Public domain (US government work); 52 U.S.C. 30111(a)(4) forbids the use of contributor names and addresses for solicitation or commercial purposes",
"https://www.fec.gov/updates/sale-or-use-contributor-information/")
[entries] Prefix the properties in the descriptions with (Node), (Edge) or (Graph) bc3dbbd
The lists of properties at the end of the descriptions marked edge properties with "Edge" but left node properties unmarked, or grouped them in "Nodes: `a`, ...; `b`, ..." items, so the kind of a property was often unclear. Every item now starts with (Node), (Edge) or (Graph), followed by the property names, a colon and the explanation, one item per property or group of properties; qualifications (kinds of nodes, networks) follow the names in parentheses. Properties that were only named got a short explanation, and a few items that named the wrong kind of property were corrected.
diff --git a/entries/fec_contributions/__init__.py b/entries/fec_contributions/__init__.py
index cf29abe..2b6fc58 100644
--- a/entries/fec_contributions/__init__.py
+++ b/entries/fec_contributions/__init__.py
@@ -30,17 +30,17 @@ The receipts are as reported, including refunds (negative amounts) and the memo
The networks have the following properties:
-- `is_committee`: whether the node is a committee (otherwise it is a contributor).
-- `committee_id` (committees): the FEC id of the committee (e.g. `C00401224`).
-- `name`: the name of the committee, or of the contributor (as merged, upper-cased).
-- `zip5`, `employer` (contributors): the first five digits of the ZIP code and the employer, as merged.
-- `city`, `state`: the city and state of the contributor (as first reported) or of the committee.
-- `occupation` (contributors): the occupation (as first reported).
-- `entity_type` (contributors): the type of entity (`IND` individual, `ORG` organization, `CAN` candidate, ...; empty in the older cycles).
-- `committee_type`, `committee_designation`, `party`, `connected_organization`, `candidate_id` (committees): the type of the committee (e.g. `H`, `S`, `P` for the campaign committees of candidates to the House, Senate and presidency; `Q`, `N` for PACs; `X`, `Y` for party committees; `O` for super PACs), its designation (e.g. `P` principal campaign committee, `J` joint fundraiser, `U` unauthorized), the party affiliation, the connected organization and the candidate id (from the committee master file; empty for committees that are not in it).
-- Edge `count`, `amount`: the number of receipts that are not memo entries, and their total amount in US dollars.
-- Edge `memo_count`, `memo_amount`: the number and total amount of the memo entries.
-- Edge `first_date`, `last_date`: the dates of the first and the last receipt (`YYYY-MM-DD`; empty if not given)."""
+- (Node) `is_committee`: whether the node is a committee (otherwise it is a contributor).
+- (Node) `committee_id` (committees): the FEC id of the committee (e.g. `C00401224`).
+- (Node) `name`: the name of the committee, or of the contributor (as merged, upper-cased).
+- (Node) `zip5`, `employer` (contributors): the first five digits of the ZIP code and the employer, as merged.
+- (Node) `city`, `state`: the city and state of the contributor (as first reported) or of the committee.
+- (Node) `occupation` (contributors): the occupation (as first reported).
+- (Node) `entity_type` (contributors): the type of entity (`IND` individual, `ORG` organization, `CAN` candidate, ...; empty in the older cycles).
+- (Node) `committee_type`, `committee_designation`, `party`, `connected_organization`, `candidate_id` (committees): the type of the committee (e.g. `H`, `S`, `P` for the campaign committees of candidates to the House, Senate and presidency; `Q`, `N` for PACs; `X`, `Y` for party committees; `O` for super PACs), its designation (e.g. `P` principal campaign committee, `J` joint fundraiser, `U` unauthorized), the party affiliation, the connected organization and the candidate id (from the committee master file; empty for committees that are not in it).
+- (Edge) `count`, `amount`: the number of receipts that are not memo entries, and their total amount in US dollars.
+- (Edge) `memo_count`, `memo_amount`: the number and total amount of the memo entries.
+- (Edge) `first_date`, `last_date`: the dates of the first and the last receipt (`YYYY-MM-DD`; empty if not given)."""
tags = ['Social', 'Political', 'Economic', 'Bipartite', 'Weighted', 'Metadata']
url = ['https://www.fec.gov/data/browse-data/?tab=bulk-data']
citation = [('Federal Election Commission, "Contributions by individuals" and "Committee master" bulk data files (1980-2026)',
[entries] New entry: fec_contributions 2e2f6d9
diff --git a/entries/fec_contributions/__init__.py b/entries/fec_contributions/__init__.py
new file mode 100644
index 0000000..cf29abe
--- /dev/null
+++ b/entries/fec_contributions/__init__.py
@@ -0,0 +1,193 @@
+#! /usr/bin/env python
+# -*- coding: utf-8 -*-
+#
+# Copyright (C) 2026 Tiago de Paula Peixoto <tiago@skewed.de>
+#
+# This program is free software: you can redistribute it and/or modify it under
+# the terms of the GNU Affero General Public License as published by the Free
+# Software Foundation, either version 3 of the License, or (at your option) any
+# later version.
+#
+# This program is distributed in the hope that it will be useful, but WITHOUT
+# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
+# FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
+# details.
+#
+# You should have received a copy of the GNU Affero General Public License along
+# with this program. If not, see <http://www.gnu.org/licenses/>.
+
+
+from .. import *
+import pyarrow
+import pyarrow.csv
+
+title = "US campaign contributions of individuals to political committees (FEC, 1980-2026)"
+description = r"""Bipartite networks of the contributions of individuals to the political committees of US federal elections (the campaign committees of candidates, party committees and political action committees), from the bulk data of the US Federal Election Commission (FEC). The committees must report to the FEC the contributions they receive from individuals that exceed 200 dollars in an election cycle (and many report smaller ones too, in particular the conduit committees, such as ActBlue and WinRed, through which individuals give to candidates); the file of "contributions by individuals" contains these itemized receipts, as reported, for each two-year election cycle. There is one network per cycle, from `1980` (1979-1980) to `2026` (2025-2026, as of the download date, October 2026; the files of the recent cycles are updated by the FEC as new reports are filed).
+
+The nodes are the contributors and the committees, and an edge from a contributor to a committee aggregates all the reported receipts of the committee from the contributor in the cycle, with their number and total amount (since the files of the recent cycles have more than 10^8 rows, the individual receipts are not kept as edges). The committees are identified by their FEC id, and described by the committee master file of the cycle. The FEC gives no identifier of the contributors, only their name, address, employer and occupation as reported by each committee: the contributors are therefore merged by their name (upper-cased, with spaces normalized), the first five digits of their ZIP code and their employer (upper-cased) together, so that the same person with differently written names or employers (or who moved) is split into several nodes, and different persons with the same name, ZIP code and employer are merged. The file also contains some contributions of organizations and other entities, which are treated in the same way (see `entity_type`).
+
+The receipts are as reported, including refunds (negative amounts) and the memo entries (`memo` code `X`), which are not counted in the totals of the reports: e.g. the receipts that a candidate committee reports as earmarked through a conduit committee also appear as receipts of the conduit, so that the same money can appear on two edges. Their numbers and amounts are given separately on the edges. Note that the law (52 U.S.C. 30111(a)(4)) forbids the use of the names and addresses of contributors from these data for soliciting contributions or for commercial purposes.
+
+The networks have the following properties:
+
+- `is_committee`: whether the node is a committee (otherwise it is a contributor).
+- `committee_id` (committees): the FEC id of the committee (e.g. `C00401224`).
+- `name`: the name of the committee, or of the contributor (as merged, upper-cased).
+- `zip5`, `employer` (contributors): the first five digits of the ZIP code and the employer, as merged.
+- `city`, `state`: the city and state of the contributor (as first reported) or of the committee.
+- `occupation` (contributors): the occupation (as first reported).
+- `entity_type` (contributors): the type of entity (`IND` individual, `ORG` organization, `CAN` candidate, ...; empty in the older cycles).
+- `committee_type`, `committee_designation`, `party`, `connected_organization`, `candidate_id` (committees): the type of the committee (e.g. `H`, `S`, `P` for the campaign committees of candidates to the House, Senate and presidency; `Q`, `N` for PACs; `X`, `Y` for party committees; `O` for super PACs), its designation (e.g. `P` principal campaign committee, `J` joint fundraiser, `U` unauthorized), the party affiliation, the connected organization and the candidate id (from the committee master file; empty for committees that are not in it).
+- Edge `count`, `amount`: the number of receipts that are not memo entries, and their total amount in US dollars.
+- Edge `memo_count`, `memo_amount`: the number and total amount of the memo entries.
+- Edge `first_date`, `last_date`: the dates of the first and the last receipt (`YYYY-MM-DD`; empty if not given)."""
+tags = ['Social', 'Political', 'Economic', 'Bipartite', 'Weighted', 'Metadata']
+url = ['https://www.fec.gov/data/browse-data/?tab=bulk-data']
+citation = [('Federal Election Commission, "Contributions by individuals" and "Committee master" bulk data files (1980-2026)',
+ 'https://www.fec.gov/data/browse-data/?tab=bulk-data')]
+icon_hash = None
+upstream_license = ("Public domain (US government work); 52 U.S.C. 30111(a)(4) forbids the use of contributor names and addresses for solicitation or commercial purposes",
+ "https://www.fec.gov/updates/sale-or-use-contributor-information/")
+
+# The files of the open cycles are replaced in place as new reports are
+# filed (those of the older cycles are stable)
+upstream_prefix = 'https://www.fec.gov/files/bulk-downloads'
+cycles = range(1980, 2028, 2)
+files = [((f"{c}/indiv{c % 100:02d}.zip:itcont.txt", f"{c}/cm{c % 100:02d}.zip:cm.txt"),
+ str(c), None) for c in cycles]
+
+def fetch_upstream(force=False):
+ return fetch_upstream_files(__name__.split(".")[-1], upstream_prefix, files,
+ force)
+
+# For checks only: if set, only (about) this many rows are read per cycle
+_check_rows = None
+
+_cols = ["CMTE_ID", "AMNDT_IND", "RPT_TP", "TRANSACTION_PGI", "IMAGE_NUM",
+ "TRANSACTION_TP", "ENTITY_TP", "NAME", "CITY", "STATE", "ZIP_CODE",
+ "EMPLOYER", "OCCUPATION", "TRANSACTION_DT", "TRANSACTION_AMT",
+ "OTHER_ID", "TRAN_ID", "FILE_NUM", "MEMO_CD", "MEMO_TEXT", "SUB_ID"]
+_cm_cols = ["CMTE_ID", "CMTE_NM", "TRES_NM", "CMTE_ST1", "CMTE_ST2",
+ "CMTE_CITY", "CMTE_ST", "CMTE_ZIP", "CMTE_DSGN", "CMTE_TP",
+ "CMTE_PTY_AFFILIATION", "CMTE_FILING_FREQ", "ORG_TP",
+ "CONNECTED_ORG_NM", "CAND_ID"]
+
+def _read_pipe(f, cols, block=1 << 26):
+ "Batches of a '|'-separated file without quoting, as string columns."
+ return pyarrow.csv.open_csv(
+ f, read_options=pyarrow.csv.ReadOptions(column_names=cols,
+ block_size=block,
+ encoding="latin-1"),
+ parse_options=pyarrow.csv.ParseOptions(delimiter="|", quote_char=False,
+ newlines_in_values=False,
+ invalid_row_handler=lambda r: "skip"),
+ convert_options=pyarrow.csv.ConvertOptions(
+ column_types={c: pyarrow.string() for c in cols}))
+
+@cache_network()
+@coerce_props()
+@annotate()
+def parse(alts=None):
+ import polars as pl
+ name = __name__.split(".")[-1]
+ norm = lambda c: (pl.col(c).fill_null("").str.to_uppercase()
+ .str.replace_all(r"\s+", " ").str.strip_chars())
+ for fnames, alt, fmt in files:
+ if alts is not None and alt not in alts:
+ continue
+ parts = []
+ first = []
+ n = 0
+ with open_upstream_file(name, fnames[0], "rb") as f:
+ for batch in _read_pipe(f, _cols):
+ df = pl.from_arrow(pyarrow.Table.from_batches([batch]))
+ n += len(df)
+ df = df.select(
+ norm("NAME").alias("name"),
+ norm("ZIP_CODE").str.slice(0, 5).alias("zip5"),
+ norm("EMPLOYER").alias("employer"),
+ pl.col("CMTE_ID").str.strip_chars().alias("cmte"),
+ pl.col("TRANSACTION_AMT").str.strip_chars()
+ .cast(pl.Float64, strict=False).fill_null(0).alias("amt"),
+ (pl.col("MEMO_CD").fill_null("").str.strip_chars() == "X").alias("memo"),
+ pl.col("TRANSACTION_DT").str.strip_chars()
+ .str.to_date("%m%d%Y", strict=False).alias("date"),
+ norm("CITY").alias("city"), norm("STATE").alias("state"),
+ norm("OCCUPATION").alias("occupation"),
+ norm("ENTITY_TP").alias("entity_type"))
+ key = ["name", "zip5", "employer"]
+ first.append(df.select(key + ["city", "state", "occupation",
+ "entity_type"])
+ .unique(subset=key, keep="first", maintain_order=True))
+ parts.append(
+ df.group_by(key + ["cmte"]).agg(
+ count=(~pl.col("memo")).sum(),
+ amount=pl.col("amt").filter(~pl.col("memo")).sum(),
+ memo_count=pl.col("memo").sum(),
+ memo_amount=pl.col("amt").filter(pl.col("memo")).sum(),
+ first_date=pl.col("date").min(),
+ last_date=pl.col("date").max()))
+ del df
+ if len(parts) >= 8:
+ parts = [_combine(pl.concat(parts))]
+ first = [pl.concat(first).unique(subset=key, keep="first",
+ maintain_order=True)]
+ if _check_rows is not None and n >= _check_rows:
+ break
+ edges = _combine(pl.concat(parts))
+ del parts
+ donors = (pl.concat(first).unique(subset=["name", "zip5", "employer"],
+ keep="first", maintain_order=True)
+ .with_row_index("s"))
+ del first
+ with open_upstream_file(name, fnames[1], "rb") as f:
+ cm = pl.from_arrow(_read_pipe(f, _cm_cols).read_all())
+ cm = cm.unique(subset="CMTE_ID", keep="last")
+ cmtes = (edges.select("cmte").unique().sort("cmte")
+ .join(cm, left_on="cmte", right_on="CMTE_ID", how="left")
+ .with_row_index("t"))
+ del cm
+ n_d = len(donors)
+ edges = (edges.join(donors.select("name", "zip5", "employer", "s"),
+ on=["name", "zip5", "employer"])
+ .join(cmtes.select("cmte", "t"), on="cmte"))
+
+ g = Graph(directed=True)
+ g.add_vertex(n_d + len(cmtes))
+ g.add_edge_list(numpy.column_stack((edges["s"].to_numpy().astype("int64"),
+ edges["t"].to_numpy().astype("int64") + n_d)))
+ g.vp.is_committee = g.new_vp("bool", vals=numpy.arange(g.num_vertices()) >= n_d)
+ s = lambda df, c: df[c].fill_null("").str.strip_chars().to_list()
+ empty_d = [""] * n_d
+ empty_c = [""] * len(cmtes)
+ g.vp.committee_id = g.new_vp("string", vals=empty_d + cmtes["cmte"].to_list())
+ g.vp.name = g.new_vp("string", vals=donors["name"].to_list() + s(cmtes, "CMTE_NM"))
+ g.vp.zip5 = g.new_vp("string", vals=donors["zip5"].to_list() + empty_c)
+ g.vp.employer = g.new_vp("string", vals=donors["employer"].to_list() + empty_c)
+ g.vp.city = g.new_vp("string", vals=donors["city"].to_list() + s(cmtes, "CMTE_CITY"))
+ g.vp.state = g.new_vp("string", vals=donors["state"].to_list() + s(cmtes, "CMTE_ST"))
+ for k in ["occupation", "entity_type"]:
+ g.vp[k] = g.new_vp("string", vals=donors[k].to_list() + empty_c)
+ for k, c in [("committee_type", "CMTE_TP"),
+ ("committee_designation", "CMTE_DSGN"),
+ ("party", "CMTE_PTY_AFFILIATION"),
+ ("connected_organization", "CONNECTED_ORG_NM"),
+ ("candidate_id", "CAND_ID")]:
+ g.vp[k] = g.new_vp("string", vals=empty_d + s(cmtes, c))
+ del donors, cmtes
+ for k in ["count", "memo_count"]:
+ g.ep[k] = g.new_ep("int64_t", vals=edges[k].to_numpy())
+ for k in ["amount", "memo_amount"]:
+ g.ep[k] = g.new_ep("double", vals=edges[k].to_numpy())
+ for k in ["first_date", "last_date"]:
+ g.ep[k] = g.new_ep("string", vals=edges[k].cast(pl.String).fill_null("").to_list())
+ del edges
+ yield alt, g
+
+def _combine(df):
+ "Combine the partial aggregates of the receipts."
+ import polars as pl
+ return df.group_by(["name", "zip5", "employer", "cmte"]).agg(
+ pl.col("count").sum(), pl.col("amount").sum(),
+ pl.col("memo_count").sum(), pl.col("memo_amount").sum(),
+ pl.col("first_date").min(), pl.col("last_date").max())