History of gab_followers
The changes to the source code of this entry in the repository, newest first, each with the differences it made to the entry's files. They are also listed on GitLab.
[entries] Add the date when each entry was added, and remove icon_hash beff7c9
The dates are those of the commits that first added the entries' files (2020-08-11 for the entries of the initial commit); renamed entries keep the date of their first name. Changed: none
diff --git a/entries/gab_followers/__init__.py b/entries/gab_followers/__init__.py
index adea2aa..161b43c 100644
--- a/entries/gab_followers/__init__.py
+++ b/entries/gab_followers/__init__.py
@@ -49,7 +49,7 @@ citation = [('S. Zannettou, B. Bradlyn, E. De Cristofaro, H. Kwak, M. Sirivianos
'https://doi.org/10.1145/3184558.3191531')]
upstream_license = ('Creative Commons Attribution 4.0 International (CC BY 4.0)',
'https://creativecommons.org/licenses/by/4.0/')
-icon_hash = None
+date_added = '2026-10-31'
# Zenodo record; the posts file is a single JSON file of 50 GB (6 GB
# compressed), read as a stream
[entries] Prefix the properties in the descriptions with (Node), (Edge) or (Graph) bc3dbbd
The lists of properties at the end of the descriptions marked edge properties with "Edge" but left node properties unmarked, or grouped them in "Nodes: `a`, ...; `b`, ..." items, so the kind of a property was often unclear. Every item now starts with (Node), (Edge) or (Graph), followed by the property names, a colon and the explanation, one item per property or group of properties; qualifications (kinds of nodes, networks) follow the names in parentheses. Properties that were only named got a short explanation, and a few items that named the wrong kind of property were corrected.
diff --git a/entries/gab_followers/__init__.py b/entries/gab_followers/__init__.py
index 66d7326..adea2aa 100644
--- a/entries/gab_followers/__init__.py
+++ b/entries/gab_followers/__init__.py
@@ -36,8 +36,13 @@ The posts file lists the posts of each user's timeline, including the user's rep
The networks have the following properties:
-- `follows`: nodes have `username`; edges have `in_followers` and `in_followings`, whether the relation is in the list of followers of v, and in the list of followings of u, and `time`, the time when the relation was first collected (Unix time, from the MongoDB object id).
-- `reposts`, `replies` and `quotes`: nodes have `user_id`, the user's id on Gab, `username`, and `verified`, `is_pro`, `is_donor`, `is_investor` and `is_private`, the flags of the account (as in the last post seen); edges have `time`, the time of the repost or of the post (Unix time), `post_id`, the id of the reposted post, or of the reply or quote, and in `replies` and `quotes`, `parent_id`, the id of the parent post."""
+- (Node) `username`: the user's name.
+- (Node) `user_id` (`reposts`, `replies` and `quotes`): the user's id on Gab.
+- (Node) `verified`, `is_pro`, `is_donor`, `is_investor`, `is_private` (`reposts`, `replies` and `quotes`): the flags of the account (as in the last post seen).
+- (Edge) `in_followers`, `in_followings` (`follows`): whether the relation is in the list of followers of v, and in the list of followings of u.
+- (Edge) `time`: in `follows`, the time when the relation was first collected (Unix time, from the MongoDB object id); in `reposts`, `replies` and `quotes`, the time of the repost or of the post (Unix time).
+- (Edge) `post_id` (`reposts`, `replies` and `quotes`): the id of the reposted post, or of the reply or quote.
+- (Edge) `parent_id` (`replies` and `quotes`): the id of the parent post."""
tags = ['Social', 'Online', 'Political', 'Unweighted', 'Multigraph', 'Timestamps', 'Metadata']
url = 'https://doi.org/10.5281/zenodo.3460400'
citation = [('S. Zannettou, B. Bradlyn, E. De Cristofaro, H. Kwak, M. Sirivianos, G. Stringhini, J. Blackburn, "What is Gab: A Bastion of Free Speech or an Alt-Right Echo Chamber." Companion Proceedings of The Web Conference 2018, 1007-1014 (2018)',
[entries] Remove unneeded code from the entries 2c69f19
Unneeded "global files" statements, unused imports and variables, f-strings without placeholders, duplicate keys in foodweb_little_rock (keeping the order and the values used), and the second definition of get_lines() in twitter_higgs. Nothing changes in the networks or descriptions. Changed: none
diff --git a/entries/gab_followers/__init__.py b/entries/gab_followers/__init__.py
index 2ec11e2..66d7326 100644
--- a/entries/gab_followers/__init__.py
+++ b/entries/gab_followers/__init__.py
@@ -124,7 +124,6 @@ def read_posts(name, fname, alts):
@coerce_props()
@annotate()
def parse(alts=None):
- global files
name = __name__.split(".")[-1]
posts = None
for fnames, alt, fmt in files:
[entries] New entry: gab_followers 4e22594
diff --git a/entries/gab_followers/__init__.py b/entries/gab_followers/__init__.py
new file mode 100644
index 0000000..2ec11e2
--- /dev/null
+++ b/entries/gab_followers/__init__.py
@@ -0,0 +1,194 @@
+#! /usr/bin/env python
+# -*- coding: utf-8 -*-
+#
+# Copyright (C) 2026 Tiago de Paula Peixoto <tiago@skewed.de>
+#
+# This program is free software: you can redistribute it and/or modify it under
+# the terms of the GNU Affero General Public License as published by the Free
+# Software Foundation, either version 3 of the License, or (at your option) any
+# later version.
+#
+# This program is distributed in the hope that it will be useful, but WITHOUT
+# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
+# FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
+# details.
+#
+# You should have received a copy of the GNU Affero General Public License along
+# with this program. If not, see <http://www.gnu.org/licenses/>.
+
+from .. import *
+try:
+ import orjson as fastjson
+except ImportError:
+ fastjson = json
+
+title = "Gab followers, reposts, replies and quotes (2016-2018)"
+description = """Networks of the users of Gab, a microblogging platform popular with the alt-right, from the dataset of Zannettou et al., who crawled Gab through its API with a snowball method: starting from the most popular users, they collected iteratively the followers and followings of all users found (lists of February 1, 2018), and then all posts of the 336,752 users found (22 million posts, from August 2016 to January 2018).
+
+The entry contains four networks:
+
+- `follows`: a directed edge from user u to user v means that u follows v, from the lists of followers and of followings (the union of both; users are identified by their usernames).
+- `reposts`: a directed edge from a user to the author of a post they reposted, for each repost.
+- `replies`: a directed edge from the author of a reply to the author of the post replied to, for each reply.
+- `quotes`: a directed edge from the author of a quote to the author of the quoted post, for each quote.
+
+The posts file lists the posts of each user's timeline, including the user's reposts, so that posts appear several times: in `replies` and `quotes`, each post is counted once (by its id), and in `reposts`, each pair of reposting user and post once. Replies and quotes whose parent post is not given are left out. Repeated interactions between the same users appear as multiple edges, and self-interactions as self-loops. The `follows` network and the others use different identifiers (usernames, and numeric ids with the usernames at the time of the post), and their nodes are not matched. Left out are the texts of the posts, their topics, attachments and the users' profile pictures.
+
+The networks have the following properties:
+
+- `follows`: nodes have `username`; edges have `in_followers` and `in_followings`, whether the relation is in the list of followers of v, and in the list of followings of u, and `time`, the time when the relation was first collected (Unix time, from the MongoDB object id).
+- `reposts`, `replies` and `quotes`: nodes have `user_id`, the user's id on Gab, `username`, and `verified`, `is_pro`, `is_donor`, `is_investor` and `is_private`, the flags of the account (as in the last post seen); edges have `time`, the time of the repost or of the post (Unix time), `post_id`, the id of the reposted post, or of the reply or quote, and in `replies` and `quotes`, `parent_id`, the id of the parent post."""
+tags = ['Social', 'Online', 'Political', 'Unweighted', 'Multigraph', 'Timestamps', 'Metadata']
+url = 'https://doi.org/10.5281/zenodo.3460400'
+citation = [('S. Zannettou, B. Bradlyn, E. De Cristofaro, H. Kwak, M. Sirivianos, G. Stringhini, J. Blackburn, "What is Gab: A Bastion of Free Speech or an Alt-Right Echo Chamber." Companion Proceedings of The Web Conference 2018, 1007-1014 (2018)',
+ 'https://doi.org/10.1145/3184558.3191531')]
+upstream_license = ('Creative Commons Attribution 4.0 International (CC BY 4.0)',
+ 'https://creativecommons.org/licenses/by/4.0/')
+icon_hash = None
+
+# Zenodo record; the posts file is a single JSON file of 50 GB (6 GB
+# compressed), read as a stream
+upstream_prefix = 'https://zenodo.org/records/3460400/files'
+NET = 'gab_network.tar'
+# (read as a stream, from the tar archive decompressed on the fly: opening
+# the member by name would make tarfile scan the whole archive first)
+POSTS = 'gab_posts_jan_2018.json.tar.gz'
+files = [((f'{NET}:followers_2018_02_01.json', f'{NET}:followings_2018_02_01.json'), 'follows', None),
+ (POSTS, 'reposts', None),
+ (POSTS, 'replies', None),
+ (POSTS, 'quotes', None)]
+
+# Number of lines read per file (None for all; set only for checks)
+_check_rows = None
+
+def fetch_upstream(force=False):
+ return fetch_upstream_files(__name__.split(".")[-1], upstream_prefix, files,
+ force)
+
+def lines(name, fname):
+ with open_upstream_file(name, fname, "rb") as f:
+ if fname == POSTS:
+ tf = tarfile.open(fileobj=f, mode="r|")
+ f = tf.extractfile(tf.next())
+ for i, line in enumerate(f):
+ if _check_rows is not None and i >= _check_rows:
+ break
+ yield line
+
+def unix(s):
+ return int(numpy.datetime64(s[:19], "s").astype("int64"))
+
+FLAGS = ["verified", "is_pro", "is_donor", "is_investor", "is_private"]
+
+def read_posts(name, fname, alts):
+ """The users and edges of the networks `alts` (among reposts, replies
+ and quotes), in one pass over the posts: a dict mapping each network to
+ (users, edges), where users maps Gab ids to [index, username, flags]."""
+ out = {}
+ for alt in alts:
+ out[alt] = ({}, [], set())
+ def user(users, u):
+ x = users.get(u["id"])
+ if x is None:
+ x = users[u["id"]] = [len(users), "", ()]
+ x[1] = u.get("username") or x[1]
+ x[2] = tuple(int(bool(u.get(f))) for f in FLAGS)
+ return x[0]
+ for line in lines(name, fname):
+ d = fastjson.loads(line)
+ p = d["post"]
+ for alt, (users, edges, seen) in out.items():
+ if alt == "reposts":
+ if d.get("type") != "repost":
+ continue
+ key = (d["actuser"]["id"], p["id"])
+ if key in seen:
+ continue
+ seen.add(key)
+ edges.append((user(users, d["actuser"]), user(users, p["user"]),
+ unix(d["published_at"]), p["id"], -1))
+ else:
+ if not p.get("is_reply" if alt == "replies" else "is_quote"):
+ continue
+ parent = p.get("parent")
+ if not parent or not parent.get("user"):
+ continue
+ if p["id"] in seen:
+ continue
+ seen.add(p["id"])
+ edges.append((user(users, p["user"]), user(users, parent["user"]),
+ unix(p["created_at"]), p["id"], parent["id"]))
+ return {alt: (users, edges) for alt, (users, edges, seen) in out.items()}
+
+@cache_network()
+@coerce_props()
+@annotate()
+def parse(alts=None):
+ global files
+ name = __name__.split(".")[-1]
+ posts = None
+ for fnames, alt, fmt in files:
+ if alts is not None and alt not in alts:
+ continue
+ if alt == "follows":
+ idx = {}
+ edges = {} # (u, v) -> [in_followers, in_followings, time]
+ for fname, (a, b), flag in [(fnames[0], ("follower", "user"), 0),
+ (fnames[1], ("user", "following"), 1)]:
+ for line in lines(name, fname):
+ d = fastjson.loads(line)
+ u = idx.setdefault(d[a], len(idx))
+ v = idx.setdefault(d[b], len(idx))
+ t = int(d["_id"]["$oid"][:8], 16)
+ e = edges.get((u, v))
+ if e is None:
+ e = edges[(u, v)] = [0, 0, t]
+ e[flag] = 1
+ e[2] = min(e[2], t)
+ g = Graph(directed=True)
+ g.add_vertex(len(idx))
+ g.vp.username = g.new_vp("string")
+ for k, i in idx.items():
+ g.vp.username[i] = k
+ del idx
+ g.ep.in_followers = g.new_ep("bool")
+ g.ep.in_followings = g.new_ep("bool")
+ g.ep.time = g.new_ep("int64_t")
+ e = numpy.array([k + tuple(v) for k, v in edges.items()],
+ dtype="int64").reshape(-1, 5)
+ del edges
+ g.add_edge_list(e, eprops=[g.ep.in_followers, g.ep.in_followings,
+ g.ep.time])
+ yield alt, g
+ continue
+
+ if posts is None:
+ posts = read_posts(name, fnames, [a for f, a, m in files
+ if a != "follows" and
+ (alts is None or a in alts)])
+ users, edges = posts.pop(alt)
+ g = Graph(directed=True)
+ g.add_vertex(len(users))
+ g.vp.user_id = g.new_vp("int64_t")
+ g.vp.username = g.new_vp("string")
+ fl = [g.new_vp("bool") for f in FLAGS]
+ for uid, (i, uname, flags) in users.items():
+ g.vp.user_id[i] = uid
+ g.vp.username[i] = uname
+ for p, x in zip(fl, flags):
+ p[i] = x
+ for f, p in zip(FLAGS, fl):
+ g.vp[f] = p
+ del users
+ g.ep.time = g.new_ep("int64_t")
+ g.ep.post_id = g.new_ep("int64_t")
+ eprops = [g.ep.time, g.ep.post_id]
+ e = numpy.array(edges, dtype="int64").reshape(-1, 5)
+ del edges
+ if alt != "reposts":
+ g.ep.parent_id = g.new_ep("int64_t")
+ eprops.append(g.ep.parent_id)
+ else:
+ e = e[:, :4]
+ g.add_edge_list(e, eprops=eprops)
+ yield alt, g