logo Netzschleuder network catalog, repository, and centrifuge

History of convote_mentions

The changes to the source code of this entry in the repository, newest first, each with the differences it made to the entry's files. They are also listed on GitLab.

[entries] Add the date when each entry was added, and remove icon_hash by Tiago de Paula Peixoto beff7c9
The dates are those of the commits that first added the entries' files
(2020-08-11 for the entries of the initial commit); renamed entries keep
the date of their first name.

Changed: none
diff --git a/entries/convote_mentions/__init__.py b/entries/convote_mentions/__init__.py
index 6695d37..f5e4aed 100644
--- a/entries/convote_mentions/__init__.py
+++ b/entries/convote_mentions/__init__.py
@@ -39,7 +39,7 @@ The network has the following properties:
 tags = ['Social', 'Political', 'Signed', 'Multigraph', 'Metadata']
 url = 'https://www.cs.cornell.edu/home/llee/data/convote.html'
 citation = [('M. Thomas, B. Pang, L. Lee, "Get out the vote: Determining support or opposition from congressional floor-debate transcripts." Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing, 327-335 (2006)', 'https://doi.org/10.3115/1610075.1610122')]
-icon_hash = None
+date_added = '2026-10-27'
 upstream_license = None
 upstream_prefix = 'https://www.cs.cornell.edu/home/llee/data/convote'
 files = [('convote_v1.1.tar.gz', None, None)]
[entries] Prefix the properties in the descriptions with (Node), (Edge) or (Graph) by Tiago de Paula Peixoto bc3dbbd
The lists of properties at the end of the descriptions marked edge
properties with "Edge" but left node properties unmarked, or grouped
them in "Nodes: `a`, ...; `b`, ..." items, so the kind of a property
was often unclear. Every item now starts with (Node), (Edge) or (Graph),
followed by the property names, a colon and the explanation, one item
per property or group of properties; qualifications (kinds of nodes,
networks) follow the names in parentheses. Properties that were only
named got a short explanation, and a few items that named the wrong
kind of property were corrected.
diff --git a/entries/convote_mentions/__init__.py b/entries/convote_mentions/__init__.py
index 2f7c86f..6695d37 100644
--- a/entries/convote_mentions/__init__.py
+++ b/entries/convote_mentions/__init__.py
@@ -27,8 +27,15 @@ The network contains the references of the 14 debates of the development and tes
 
 The network has the following properties:
 
-- Nodes: `speaker_id`, the index of the speaker in the corpus; `party`, the party of the speaker ("D" for Democratic, "R" for Republican, "I" for independent).
-- Edges: `sign`, +1 for a reference between speakers who voted the same way on the bill (agreement), -1 otherwise (disagreement); `debate`, the index of the bill under discussion (identifying the debate); `source_vote` and `target_vote`, the votes ("yes" or "no") of the referring and the referenced speaker on the bill; `raw_score`, the score of the agreement classifier (positive for references classified as agreements); `normalized_score`, the score divided by the standard deviation of the scores of the references of the debate; `strength`, the strength of the link between the speakers in the authors' graph, 2500 times the normalized score (0 for negative normalized scores); `normalized_score_high_precision` and `strength_high_precision`, the same, but with the score minus the average score of the debate, which keeps only the references with above-average scores in the graph."""
+- (Node) `speaker_id`: the index of the speaker in the corpus.
+- (Node) `party`: the party of the speaker ("D" for Democratic, "R" for Republican, "I" for independent).
+- (Edge) `sign`: +1 for a reference between speakers who voted the same way on the bill (agreement), -1 otherwise (disagreement).
+- (Edge) `debate`: the index of the bill under discussion (identifying the debate).
+- (Edge) `source_vote` and `target_vote`: the votes ("yes" or "no") of the referring and the referenced speaker on the bill.
+- (Edge) `raw_score`: the score of the agreement classifier (positive for references classified as agreements).
+- (Edge) `normalized_score`: the score divided by the standard deviation of the scores of the references of the debate.
+- (Edge) `strength`: the strength of the link between the speakers in the authors' graph, 2500 times the normalized score (0 for negative normalized scores).
+- (Edge) `normalized_score_high_precision` and `strength_high_precision`: the same, but with the score minus the average score of the debate, which keeps only the references with above-average scores in the graph."""
 tags = ['Social', 'Political', 'Signed', 'Multigraph', 'Metadata']
 url = 'https://www.cs.cornell.edu/home/llee/data/convote.html'
 citation = [('M. Thomas, B. Pang, L. Lee, "Get out the vote: Determining support or opposition from congressional floor-debate transcripts." Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing, 327-335 (2006)', 'https://doi.org/10.3115/1610075.1610122')]
[entries] Remove unneeded code from the entries by Tiago de Paula Peixoto 2c69f19
Unneeded "global files" statements, unused imports and variables,
f-strings without placeholders, duplicate keys in foodweb_little_rock
(keeping the order and the values used), and the second definition of
get_lines() in twitter_higgs. Nothing changes in the networks or
descriptions.

Changed: none
diff --git a/entries/convote_mentions/__init__.py b/entries/convote_mentions/__init__.py
index d4686df..2f7c86f 100644
--- a/entries/convote_mentions/__init__.py
+++ b/entries/convote_mentions/__init__.py
@@ -47,7 +47,6 @@ def fetch_upstream(force=False):
 @coerce_props()
 @annotate()
 def parse(alts=None):
-    global files
     name = __name__.split(".")[-1]
     for fname, alt, fmt in files:
         if alts is not None and alt not in alts:
[entries] Move contact, convote_mentions, douban, email_company, epinions, flixster, football_league_results, hyves, lastfm, libimseti, livemocha, reuters, sa_companies, sister_cities, tripadvisor_ratings, wikilens_ratings and wikipedia_discussions_de to their original sources by Tiago de Paula Peixoto 8f097de
These entries downloaded KONECT's copies of the data. They now download
the original data (live, from pinned Wayback Machine snapshots, or by
re-extracting from public dumps), gaining the metadata KONECT dropped
(names, identifiers, timestamps, attributes), with descriptions and
citations rewritten from the original sources and factual errors fixed
(years and periods of contact, email_company, douban, flixster,
libimseti, sa_companies; the rated objects of epinions; the sign of
convote_mentions; songs merged by title alone in lastfm).

New networks: the trust network of epinions, the links of
wikilens_ratings, the JSON crawl of tripadvisor_ratings, and current
versions of sister_cities and wikipedia_discussions_de next to the ones
KONECT used. asu_social_networks only loses its mention of KONECT.

Changed: upstream contact convote_mentions douban email_company epinions flixster football_league_results hyves lastfm libimseti livemocha reuters sa_companies sister_cities tripadvisor_ratings wikilens_ratings wikipedia_discussions_de
diff --git a/entries/convote_mentions/__init__.py b/entries/convote_mentions/__init__.py
index a002086..d4686df 100644
--- a/entries/convote_mentions/__init__.py
+++ b/entries/convote_mentions/__init__.py
@@ -16,29 +16,28 @@
 # You should have received a copy of the GNU Affero General Public License along
 # with this program.  If not, see <http://www.gnu.org/licenses/>.
 
-
-
 from .. import *
 
+import tarfile
 
 title = 'US House floor debate mentions (ConVote, 2005)'
-description = """A directed signed network of mentions between the speakers of the floor debates of the US House of Representatives in 2005, from the ConVote corpus of Thomas, Pang and Lee (2006), as distributed by KONECT. Each node is a speaker (a member of the House), and a directed edge from speaker A to speaker B is a mention of B in a speech of A, labeled as expressing support for or opposition to B. Pairs of speakers may be connected by several edges, and speakers may mention themselves. The data identify the speakers only by indices.
+description = """A directed signed network of the by-name references between the speakers of floor debates of the US House of Representatives in 2005, from the ConVote corpus of Thomas, Pang and Lee (2006). The authors downloaded the 2005 record of the House from govtrack.us, associated each page with the bill it discussed most, kept the bills that came to a vote, split the pages into speech segments, and labeled each segment by the vote (yes or no) of its speaker on the bill; speakers who abstained were discarded, and only the debates (the speeches on a bill) with at least 20% of speeches of each label were kept, 53 debates in total. In the speeches, the references by name to other members of the House who spoke in the same debate were identified, and a classifier (a support vector machine) was trained on those of the training set (39 debates) to tell whether a reference expresses agreement or disagreement with the referenced member.
+
+The network contains the references of the 14 debates of the development and test sets (the references of the training set are not distributed), with the scores of the classifier. Each node is a speaker (a member of the House), and a directed edge from speaker A to speaker B is a reference to B in a speech of A in a debate. Pairs of speakers may be connected by several edges, and speakers may refer to themselves. The sign of an edge is the true label of the reference: agreement if both speakers voted the same way on the bill, and disagreement otherwise. Only the speakers making or receiving some reference are included (219 of the 302 speakers of these debates). The texts of the speeches, the scores of the individual speeches by the classifier of support or opposition (and the strengths of the corresponding links of the authors' graphs), and the indicators of whether a speech mentions the bill directly are not included.
 
 The network has the following properties:
 
-- Edge `sign`: +1 for a mention in support, -1 for a mention in opposition."""
-tags = ['Social', 'Political', 'Signed', 'Multigraph']
-url = ['http://konect.cc/networks/convote', 'https://www.cs.cornell.edu/home/llee/data/convote.html']
-citation = [('M. Thomas, B. Pang, L. Lee, "Get out the vote: Determining support or opposition from congressional floor-debate transcripts." Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing, 327-335 (2006)', 'https://doi.org/10.3115/1610075.1610122'),
-            ('J. Kunegis, "KONECT -- The Koblenz Network Collection." Proceedings of the 22nd International Conference on World Wide Web Companion, 1343-1350 (2013)', 'https://doi.org/10.1145/2487788.2488173')]
+- Nodes: `speaker_id`, the index of the speaker in the corpus; `party`, the party of the speaker ("D" for Democratic, "R" for Republican, "I" for independent).
+- Edges: `sign`, +1 for a reference between speakers who voted the same way on the bill (agreement), -1 otherwise (disagreement); `debate`, the index of the bill under discussion (identifying the debate); `source_vote` and `target_vote`, the votes ("yes" or "no") of the referring and the referenced speaker on the bill; `raw_score`, the score of the agreement classifier (positive for references classified as agreements); `normalized_score`, the score divided by the standard deviation of the scores of the references of the debate; `strength`, the strength of the link between the speakers in the authors' graph, 2500 times the normalized score (0 for negative normalized scores); `normalized_score_high_precision` and `strength_high_precision`, the same, but with the score minus the average score of the debate, which keeps only the references with above-average scores in the graph."""
+tags = ['Social', 'Political', 'Signed', 'Multigraph', 'Metadata']
+url = 'https://www.cs.cornell.edu/home/llee/data/convote.html'
+citation = [('M. Thomas, B. Pang, L. Lee, "Get out the vote: Determining support or opposition from congressional floor-debate transcripts." Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing, 327-335 (2006)', 'https://doi.org/10.3115/1610075.1610122')]
 icon_hash = None
 upstream_license = None
-upstream_prefix = 'http://konect.cc/files'
-files = [('download.tsv.convote.tar.bz2', None, 'konect')]
+upstream_prefix = 'https://www.cs.cornell.edu/home/llee/data/convote'
+files = [('convote_v1.1.tar.gz', None, None)]
 
-def postprocess(alt, g):
-    g.ep.sign = g.ep.weight.copy("int")
-    del g.ep["weight"]
+PREFIX = "convote_v1.1/graph_edge_data/edges_reference_set_"
 
 def fetch_upstream(force=False):
     return fetch_upstream_files(__name__.split(".")[-1], upstream_prefix, files,
@@ -53,7 +52,62 @@ def parse(alts=None):
     for fname, alt, fmt in files:
         if alts is not None and alt not in alts:
             continue
+        party = {}
+        votes = {}
+        refs = {}
         with open_upstream_file(name, fname, "rb") as f:
-            g = parse_graph([f], fmt, directed=None)
-        postprocess(alt, g)
+            with tarfile.open(fileobj=f, mode="r|") as tf:
+                for m in tf:
+                    base = os.path.basename(m.name)
+                    # speech segments: <debate>_<speaker>_<page and
+                    # position>_<party><bill mentioned><vote>.txt
+                    s = re.fullmatch(r"(\d{3})_(\d{6})_\d{7}_([A-Z])[MO]([YN])\.txt",
+                                     base)
+                    if m.name.startswith("convote_v1.1/data_stage_") and s:
+                        debate, speaker, p, v = s.groups()
+                        assert party.setdefault(int(speaker), p) == p
+                        v = "yes" if v == "Y" else "no"
+                        assert votes.setdefault((int(debate), int(speaker)), v) == v
+                    elif m.name.startswith(PREFIX) and m.name.endswith(".csv"):
+                        kind = m.name.removeprefix(PREFIX).split(".")[0]
+                        text = tf.extractfile(m).read().decode()
+                        refs[kind] = list(csv.reader(io.StringIO(text)))
+        full = refs["full"]
+        hp = refs["high_precision"]
+        assert len(full) == len(hp)
+        g = Graph(directed=True)
+        vs = {}
+        def get_v(s):
+            if s not in vs:
+                vs[s] = g.add_vertex()
+            return vs[s]
+        g.ep.sign = g.new_ep("int")
+        g.ep.debate = g.new_ep("int")
+        g.ep.source_vote = g.new_ep("string")
+        g.ep.target_vote = g.new_ep("string")
+        g.ep.raw_score = g.new_ep("double")
+        g.ep.normalized_score = g.new_ep("double")
+        g.ep.strength = g.new_ep("int")
+        g.ep.normalized_score_high_precision = g.new_ep("double")
+        g.ep.strength_high_precision = g.new_ep("int")
+        # <true label>,<debate>,<referring speaker>,<referenced speaker>,
+        # <raw score>,<normalized score>,<edge strength>
+        for r, rh in zip(full, hp):
+            assert len(r) == 7 and r[:5] == rh[:5]
+            label, debate, s, t = (int(x) for x in r[:4])
+            e = g.add_edge(get_v(s), get_v(t))
+            g.ep.sign[e] = label
+            g.ep.debate[e] = debate
+            g.ep.source_vote[e] = votes[(debate, s)]
+            g.ep.target_vote[e] = votes[(debate, t)]
+            g.ep.raw_score[e] = float(r[4])
+            g.ep.normalized_score[e] = float(r[5])
+            g.ep.strength[e] = int(r[6])
+            g.ep.normalized_score_high_precision[e] = float(rh[5])
+            g.ep.strength_high_precision[e] = int(rh[6])
+        g.vp.speaker_id = g.new_vp("int")
+        g.vp.party = g.new_vp("string")
+        for s, v in vs.items():
+            g.vp.speaker_id[v] = s
+            g.vp.party[v] = party[s]
         yield alt, g
[entries] New entry: convote_mentions by Tiago de Paula Peixoto c042bb8
diff --git a/entries/convote_mentions/__init__.py b/entries/convote_mentions/__init__.py
new file mode 100644
index 0000000..a002086
--- /dev/null
+++ b/entries/convote_mentions/__init__.py
@@ -0,0 +1,59 @@
+#! /usr/bin/env python
+# -*- coding: utf-8 -*-
+#
+# Copyright (C) 2026 Tiago de Paula Peixoto <tiago@skewed.de>
+#
+# This program is free software: you can redistribute it and/or modify it under
+# the terms of the GNU Affero General Public License as published by the Free
+# Software Foundation, either version 3 of the License, or (at your option) any
+# later version.
+#
+# This program is distributed in the hope that it will be useful, but WITHOUT
+# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
+# FOR A PARTICULAR PURPOSE.  See the GNU Affero General Public License for more
+# details.
+#
+# You should have received a copy of the GNU Affero General Public License along
+# with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+
+
+from .. import *
+
+
+title = 'US House floor debate mentions (ConVote, 2005)'
+description = """A directed signed network of mentions between the speakers of the floor debates of the US House of Representatives in 2005, from the ConVote corpus of Thomas, Pang and Lee (2006), as distributed by KONECT. Each node is a speaker (a member of the House), and a directed edge from speaker A to speaker B is a mention of B in a speech of A, labeled as expressing support for or opposition to B. Pairs of speakers may be connected by several edges, and speakers may mention themselves. The data identify the speakers only by indices.
+
+The network has the following properties:
+
+- Edge `sign`: +1 for a mention in support, -1 for a mention in opposition."""
+tags = ['Social', 'Political', 'Signed', 'Multigraph']
+url = ['http://konect.cc/networks/convote', 'https://www.cs.cornell.edu/home/llee/data/convote.html']
+citation = [('M. Thomas, B. Pang, L. Lee, "Get out the vote: Determining support or opposition from congressional floor-debate transcripts." Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing, 327-335 (2006)', 'https://doi.org/10.3115/1610075.1610122'),
+            ('J. Kunegis, "KONECT -- The Koblenz Network Collection." Proceedings of the 22nd International Conference on World Wide Web Companion, 1343-1350 (2013)', 'https://doi.org/10.1145/2487788.2488173')]
+icon_hash = None
+upstream_license = None
+upstream_prefix = 'http://konect.cc/files'
+files = [('download.tsv.convote.tar.bz2', None, 'konect')]
+
+def postprocess(alt, g):
+    g.ep.sign = g.ep.weight.copy("int")
+    del g.ep["weight"]
+
+def fetch_upstream(force=False):
+    return fetch_upstream_files(__name__.split(".")[-1], upstream_prefix, files,
+                                force)
+
+@cache_network()
+@coerce_props()
+@annotate()
+def parse(alts=None):
+    global files
+    name = __name__.split(".")[-1]
+    for fname, alt, fmt in files:
+        if alts is not None and alt not in alts:
+            continue
+        with open_upstream_file(name, fname, "rb") as f:
+            g = parse_graph([f], fmt, directed=None)
+        postprocess(alt, g)
+        yield alt, g