History of convote_mentions
The changes to the source code of this entry in the repository, newest first, each with the differences it made to the entry's files. They are also listed on GitLab.
[entries] Add the date when each entry was added, and remove icon_hash beff7c9
The dates are those of the commits that first added the entries' files (2020-08-11 for the entries of the initial commit); renamed entries keep the date of their first name. Changed: none
diff --git a/entries/convote_mentions/__init__.py b/entries/convote_mentions/__init__.py
index 6695d37..f5e4aed 100644
--- a/entries/convote_mentions/__init__.py
+++ b/entries/convote_mentions/__init__.py
@@ -39,7 +39,7 @@ The network has the following properties:
tags = ['Social', 'Political', 'Signed', 'Multigraph', 'Metadata']
url = 'https://www.cs.cornell.edu/home/llee/data/convote.html'
citation = [('M. Thomas, B. Pang, L. Lee, "Get out the vote: Determining support or opposition from congressional floor-debate transcripts." Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing, 327-335 (2006)', 'https://doi.org/10.3115/1610075.1610122')]
-icon_hash = None
+date_added = '2026-10-27'
upstream_license = None
upstream_prefix = 'https://www.cs.cornell.edu/home/llee/data/convote'
files = [('convote_v1.1.tar.gz', None, None)]
[entries] Prefix the properties in the descriptions with (Node), (Edge) or (Graph) bc3dbbd
The lists of properties at the end of the descriptions marked edge properties with "Edge" but left node properties unmarked, or grouped them in "Nodes: `a`, ...; `b`, ..." items, so the kind of a property was often unclear. Every item now starts with (Node), (Edge) or (Graph), followed by the property names, a colon and the explanation, one item per property or group of properties; qualifications (kinds of nodes, networks) follow the names in parentheses. Properties that were only named got a short explanation, and a few items that named the wrong kind of property were corrected.
diff --git a/entries/convote_mentions/__init__.py b/entries/convote_mentions/__init__.py
index 2f7c86f..6695d37 100644
--- a/entries/convote_mentions/__init__.py
+++ b/entries/convote_mentions/__init__.py
@@ -27,8 +27,15 @@ The network contains the references of the 14 debates of the development and tes
The network has the following properties:
-- Nodes: `speaker_id`, the index of the speaker in the corpus; `party`, the party of the speaker ("D" for Democratic, "R" for Republican, "I" for independent).
-- Edges: `sign`, +1 for a reference between speakers who voted the same way on the bill (agreement), -1 otherwise (disagreement); `debate`, the index of the bill under discussion (identifying the debate); `source_vote` and `target_vote`, the votes ("yes" or "no") of the referring and the referenced speaker on the bill; `raw_score`, the score of the agreement classifier (positive for references classified as agreements); `normalized_score`, the score divided by the standard deviation of the scores of the references of the debate; `strength`, the strength of the link between the speakers in the authors' graph, 2500 times the normalized score (0 for negative normalized scores); `normalized_score_high_precision` and `strength_high_precision`, the same, but with the score minus the average score of the debate, which keeps only the references with above-average scores in the graph."""
+- (Node) `speaker_id`: the index of the speaker in the corpus.
+- (Node) `party`: the party of the speaker ("D" for Democratic, "R" for Republican, "I" for independent).
+- (Edge) `sign`: +1 for a reference between speakers who voted the same way on the bill (agreement), -1 otherwise (disagreement).
+- (Edge) `debate`: the index of the bill under discussion (identifying the debate).
+- (Edge) `source_vote` and `target_vote`: the votes ("yes" or "no") of the referring and the referenced speaker on the bill.
+- (Edge) `raw_score`: the score of the agreement classifier (positive for references classified as agreements).
+- (Edge) `normalized_score`: the score divided by the standard deviation of the scores of the references of the debate.
+- (Edge) `strength`: the strength of the link between the speakers in the authors' graph, 2500 times the normalized score (0 for negative normalized scores).
+- (Edge) `normalized_score_high_precision` and `strength_high_precision`: the same, but with the score minus the average score of the debate, which keeps only the references with above-average scores in the graph."""
tags = ['Social', 'Political', 'Signed', 'Multigraph', 'Metadata']
url = 'https://www.cs.cornell.edu/home/llee/data/convote.html'
citation = [('M. Thomas, B. Pang, L. Lee, "Get out the vote: Determining support or opposition from congressional floor-debate transcripts." Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing, 327-335 (2006)', 'https://doi.org/10.3115/1610075.1610122')]
[entries] Remove unneeded code from the entries 2c69f19
Unneeded "global files" statements, unused imports and variables, f-strings without placeholders, duplicate keys in foodweb_little_rock (keeping the order and the values used), and the second definition of get_lines() in twitter_higgs. Nothing changes in the networks or descriptions. Changed: none
diff --git a/entries/convote_mentions/__init__.py b/entries/convote_mentions/__init__.py
index d4686df..2f7c86f 100644
--- a/entries/convote_mentions/__init__.py
+++ b/entries/convote_mentions/__init__.py
@@ -47,7 +47,6 @@ def fetch_upstream(force=False):
@coerce_props()
@annotate()
def parse(alts=None):
- global files
name = __name__.split(".")[-1]
for fname, alt, fmt in files:
if alts is not None and alt not in alts:
[entries] Move contact, convote_mentions, douban, email_company, epinions, flixster, football_league_results, hyves, lastfm, libimseti, livemocha, reuters, sa_companies, sister_cities, tripadvisor_ratings, wikilens_ratings and wikipedia_discussions_de to their original sources 8f097de
These entries downloaded KONECT's copies of the data. They now download the original data (live, from pinned Wayback Machine snapshots, or by re-extracting from public dumps), gaining the metadata KONECT dropped (names, identifiers, timestamps, attributes), with descriptions and citations rewritten from the original sources and factual errors fixed (years and periods of contact, email_company, douban, flixster, libimseti, sa_companies; the rated objects of epinions; the sign of convote_mentions; songs merged by title alone in lastfm). New networks: the trust network of epinions, the links of wikilens_ratings, the JSON crawl of tripadvisor_ratings, and current versions of sister_cities and wikipedia_discussions_de next to the ones KONECT used. asu_social_networks only loses its mention of KONECT. Changed: upstream contact convote_mentions douban email_company epinions flixster football_league_results hyves lastfm libimseti livemocha reuters sa_companies sister_cities tripadvisor_ratings wikilens_ratings wikipedia_discussions_de
diff --git a/entries/convote_mentions/__init__.py b/entries/convote_mentions/__init__.py
index a002086..d4686df 100644
--- a/entries/convote_mentions/__init__.py
+++ b/entries/convote_mentions/__init__.py
@@ -16,29 +16,28 @@
# You should have received a copy of the GNU Affero General Public License along
# with this program. If not, see <http://www.gnu.org/licenses/>.
-
-
from .. import *
+import tarfile
title = 'US House floor debate mentions (ConVote, 2005)'
-description = """A directed signed network of mentions between the speakers of the floor debates of the US House of Representatives in 2005, from the ConVote corpus of Thomas, Pang and Lee (2006), as distributed by KONECT. Each node is a speaker (a member of the House), and a directed edge from speaker A to speaker B is a mention of B in a speech of A, labeled as expressing support for or opposition to B. Pairs of speakers may be connected by several edges, and speakers may mention themselves. The data identify the speakers only by indices.
+description = """A directed signed network of the by-name references between the speakers of floor debates of the US House of Representatives in 2005, from the ConVote corpus of Thomas, Pang and Lee (2006). The authors downloaded the 2005 record of the House from govtrack.us, associated each page with the bill it discussed most, kept the bills that came to a vote, split the pages into speech segments, and labeled each segment by the vote (yes or no) of its speaker on the bill; speakers who abstained were discarded, and only the debates (the speeches on a bill) with at least 20% of speeches of each label were kept, 53 debates in total. In the speeches, the references by name to other members of the House who spoke in the same debate were identified, and a classifier (a support vector machine) was trained on those of the training set (39 debates) to tell whether a reference expresses agreement or disagreement with the referenced member.
+
+The network contains the references of the 14 debates of the development and test sets (the references of the training set are not distributed), with the scores of the classifier. Each node is a speaker (a member of the House), and a directed edge from speaker A to speaker B is a reference to B in a speech of A in a debate. Pairs of speakers may be connected by several edges, and speakers may refer to themselves. The sign of an edge is the true label of the reference: agreement if both speakers voted the same way on the bill, and disagreement otherwise. Only the speakers making or receiving some reference are included (219 of the 302 speakers of these debates). The texts of the speeches, the scores of the individual speeches by the classifier of support or opposition (and the strengths of the corresponding links of the authors' graphs), and the indicators of whether a speech mentions the bill directly are not included.
The network has the following properties:
-- Edge `sign`: +1 for a mention in support, -1 for a mention in opposition."""
-tags = ['Social', 'Political', 'Signed', 'Multigraph']
-url = ['http://konect.cc/networks/convote', 'https://www.cs.cornell.edu/home/llee/data/convote.html']
-citation = [('M. Thomas, B. Pang, L. Lee, "Get out the vote: Determining support or opposition from congressional floor-debate transcripts." Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing, 327-335 (2006)', 'https://doi.org/10.3115/1610075.1610122'),
- ('J. Kunegis, "KONECT -- The Koblenz Network Collection." Proceedings of the 22nd International Conference on World Wide Web Companion, 1343-1350 (2013)', 'https://doi.org/10.1145/2487788.2488173')]
+- Nodes: `speaker_id`, the index of the speaker in the corpus; `party`, the party of the speaker ("D" for Democratic, "R" for Republican, "I" for independent).
+- Edges: `sign`, +1 for a reference between speakers who voted the same way on the bill (agreement), -1 otherwise (disagreement); `debate`, the index of the bill under discussion (identifying the debate); `source_vote` and `target_vote`, the votes ("yes" or "no") of the referring and the referenced speaker on the bill; `raw_score`, the score of the agreement classifier (positive for references classified as agreements); `normalized_score`, the score divided by the standard deviation of the scores of the references of the debate; `strength`, the strength of the link between the speakers in the authors' graph, 2500 times the normalized score (0 for negative normalized scores); `normalized_score_high_precision` and `strength_high_precision`, the same, but with the score minus the average score of the debate, which keeps only the references with above-average scores in the graph."""
+tags = ['Social', 'Political', 'Signed', 'Multigraph', 'Metadata']
+url = 'https://www.cs.cornell.edu/home/llee/data/convote.html'
+citation = [('M. Thomas, B. Pang, L. Lee, "Get out the vote: Determining support or opposition from congressional floor-debate transcripts." Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing, 327-335 (2006)', 'https://doi.org/10.3115/1610075.1610122')]
icon_hash = None
upstream_license = None
-upstream_prefix = 'http://konect.cc/files'
-files = [('download.tsv.convote.tar.bz2', None, 'konect')]
+upstream_prefix = 'https://www.cs.cornell.edu/home/llee/data/convote'
+files = [('convote_v1.1.tar.gz', None, None)]
-def postprocess(alt, g):
- g.ep.sign = g.ep.weight.copy("int")
- del g.ep["weight"]
+PREFIX = "convote_v1.1/graph_edge_data/edges_reference_set_"
def fetch_upstream(force=False):
return fetch_upstream_files(__name__.split(".")[-1], upstream_prefix, files,
@@ -53,7 +52,62 @@ def parse(alts=None):
for fname, alt, fmt in files:
if alts is not None and alt not in alts:
continue
+ party = {}
+ votes = {}
+ refs = {}
with open_upstream_file(name, fname, "rb") as f:
- g = parse_graph([f], fmt, directed=None)
- postprocess(alt, g)
+ with tarfile.open(fileobj=f, mode="r|") as tf:
+ for m in tf:
+ base = os.path.basename(m.name)
+ # speech segments: <debate>_<speaker>_<page and
+ # position>_<party><bill mentioned><vote>.txt
+ s = re.fullmatch(r"(\d{3})_(\d{6})_\d{7}_([A-Z])[MO]([YN])\.txt",
+ base)
+ if m.name.startswith("convote_v1.1/data_stage_") and s:
+ debate, speaker, p, v = s.groups()
+ assert party.setdefault(int(speaker), p) == p
+ v = "yes" if v == "Y" else "no"
+ assert votes.setdefault((int(debate), int(speaker)), v) == v
+ elif m.name.startswith(PREFIX) and m.name.endswith(".csv"):
+ kind = m.name.removeprefix(PREFIX).split(".")[0]
+ text = tf.extractfile(m).read().decode()
+ refs[kind] = list(csv.reader(io.StringIO(text)))
+ full = refs["full"]
+ hp = refs["high_precision"]
+ assert len(full) == len(hp)
+ g = Graph(directed=True)
+ vs = {}
+ def get_v(s):
+ if s not in vs:
+ vs[s] = g.add_vertex()
+ return vs[s]
+ g.ep.sign = g.new_ep("int")
+ g.ep.debate = g.new_ep("int")
+ g.ep.source_vote = g.new_ep("string")
+ g.ep.target_vote = g.new_ep("string")
+ g.ep.raw_score = g.new_ep("double")
+ g.ep.normalized_score = g.new_ep("double")
+ g.ep.strength = g.new_ep("int")
+ g.ep.normalized_score_high_precision = g.new_ep("double")
+ g.ep.strength_high_precision = g.new_ep("int")
+ # <true label>,<debate>,<referring speaker>,<referenced speaker>,
+ # <raw score>,<normalized score>,<edge strength>
+ for r, rh in zip(full, hp):
+ assert len(r) == 7 and r[:5] == rh[:5]
+ label, debate, s, t = (int(x) for x in r[:4])
+ e = g.add_edge(get_v(s), get_v(t))
+ g.ep.sign[e] = label
+ g.ep.debate[e] = debate
+ g.ep.source_vote[e] = votes[(debate, s)]
+ g.ep.target_vote[e] = votes[(debate, t)]
+ g.ep.raw_score[e] = float(r[4])
+ g.ep.normalized_score[e] = float(r[5])
+ g.ep.strength[e] = int(r[6])
+ g.ep.normalized_score_high_precision[e] = float(rh[5])
+ g.ep.strength_high_precision[e] = int(rh[6])
+ g.vp.speaker_id = g.new_vp("int")
+ g.vp.party = g.new_vp("string")
+ for s, v in vs.items():
+ g.vp.speaker_id[v] = s
+ g.vp.party[v] = party[s]
yield alt, g
[entries] New entry: convote_mentions c042bb8
diff --git a/entries/convote_mentions/__init__.py b/entries/convote_mentions/__init__.py
new file mode 100644
index 0000000..a002086
--- /dev/null
+++ b/entries/convote_mentions/__init__.py
@@ -0,0 +1,59 @@
+#! /usr/bin/env python
+# -*- coding: utf-8 -*-
+#
+# Copyright (C) 2026 Tiago de Paula Peixoto <tiago@skewed.de>
+#
+# This program is free software: you can redistribute it and/or modify it under
+# the terms of the GNU Affero General Public License as published by the Free
+# Software Foundation, either version 3 of the License, or (at your option) any
+# later version.
+#
+# This program is distributed in the hope that it will be useful, but WITHOUT
+# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
+# FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
+# details.
+#
+# You should have received a copy of the GNU Affero General Public License along
+# with this program. If not, see <http://www.gnu.org/licenses/>.
+
+
+
+from .. import *
+
+
+title = 'US House floor debate mentions (ConVote, 2005)'
+description = """A directed signed network of mentions between the speakers of the floor debates of the US House of Representatives in 2005, from the ConVote corpus of Thomas, Pang and Lee (2006), as distributed by KONECT. Each node is a speaker (a member of the House), and a directed edge from speaker A to speaker B is a mention of B in a speech of A, labeled as expressing support for or opposition to B. Pairs of speakers may be connected by several edges, and speakers may mention themselves. The data identify the speakers only by indices.
+
+The network has the following properties:
+
+- Edge `sign`: +1 for a mention in support, -1 for a mention in opposition."""
+tags = ['Social', 'Political', 'Signed', 'Multigraph']
+url = ['http://konect.cc/networks/convote', 'https://www.cs.cornell.edu/home/llee/data/convote.html']
+citation = [('M. Thomas, B. Pang, L. Lee, "Get out the vote: Determining support or opposition from congressional floor-debate transcripts." Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing, 327-335 (2006)', 'https://doi.org/10.3115/1610075.1610122'),
+ ('J. Kunegis, "KONECT -- The Koblenz Network Collection." Proceedings of the 22nd International Conference on World Wide Web Companion, 1343-1350 (2013)', 'https://doi.org/10.1145/2487788.2488173')]
+icon_hash = None
+upstream_license = None
+upstream_prefix = 'http://konect.cc/files'
+files = [('download.tsv.convote.tar.bz2', None, 'konect')]
+
+def postprocess(alt, g):
+ g.ep.sign = g.ep.weight.copy("int")
+ del g.ep["weight"]
+
+def fetch_upstream(force=False):
+ return fetch_upstream_files(__name__.split(".")[-1], upstream_prefix, files,
+ force)
+
+@cache_network()
+@coerce_props()
+@annotate()
+def parse(alts=None):
+ global files
+ name = __name__.split(".")[-1]
+ for fname, alt, fmt in files:
+ if alts is not None and alt not in alts:
+ continue
+ with open_upstream_file(name, fname, "rb") as f:
+ g = parse_graph([f], fmt, directed=None)
+ postprocess(alt, g)
+ yield alt, g