History of sp_south_african_households
The changes to the source code of this entry in the repository, newest first, each with the differences it made to the entry's files. They are also listed on GitLab.
[entries] Remove superfluous details of the upstream distribution from the descriptions dc07aa0
The descriptions mentioned incidental details of how the upstream data are made available: the file formats (CSV, JSON, XML, GEXF, SBML, HIF, Pajek, UCINET, ...), file names, where the files were published or obtained from (Zenodo, figshare, Kaggle, the Pajek datasets page, the Internet Archive), and that some networks are only available in the gt format. These are left out; caveats about the data themselves are kept, referring to "the data" instead of "the file". The description of sp_baboons now lists its properties instead of the upstream columns, and says that events without a recipient are left out.
diff --git a/entries/sp_south_african_households/__init__.py b/entries/sp_south_african_households/__init__.py
index 4bc8786..e0e223e 100644
--- a/entries/sp_south_african_households/__init__.py
+++ b/entries/sp_south_african_households/__init__.py
@@ -24,12 +24,12 @@ import datetime
title = "South African household contacts (2020-2021)"
description = r"""Temporal network of close-range proximity contacts between members of 88 households in two urban communities of South Africa, Klerksdorp (North West Province) and Soweto (Gauteng Province), collected in the SARS-CoV-2 household transmission study SA-S-HTS led by the National Institute for Communicable Diseases (NICD), in collaboration with SocioPatterns. After a symptomatic adult SARS-CoV-2 index case was identified at a clinic, the members of their household were followed up, and wearable proximity sensors were deployed for about two weeks (between November 2020 and August 2021) to measure close-range interactions (within about 1.5 m) between household members. Sensors exchange low-power radio packets, and a contact event between two sensors is a sequence of consecutive 20-second intervals in each of which at least one packet was exchanged. Events outside the deployment period and during 5-minute periods in which the sensor's accelerometer detected no movement were removed, as were individuals who did not comply or had sensor failures. Due to a technical error some sensors in Klerksdorp did not record valid timestamps; these were aligned relative to each other by cross-correlating packet time series, and their times start at the UNIX epoch (1 January 1970) instead of the real date. The data include 340 individuals (88 index cases and 252 household contacts), all of whom were included in the analysis of Kleynhans et al. (2023); contacts were only measured within households, so the network has one connected component per household.
-Nodes are individuals, and each edge is a contact event between two members of the same household; two individuals are connected by as many edges as the number of their contact events. (The upstream file lists every event twice, once for each direction; here each event appears once.) The network has the following properties:
+Nodes are individuals, and each edge is a contact event between two members of the same household; two individuals are connected by as many edges as the number of their contact events. (The upstream data list every event twice, once for each direction; here each event appears once.) The network has the following properties:
- (Node) `id`: the identifier of the individual (site letter, `J` for Klerksdorp or `S` for Soweto, household number and individual number).
- (Node) `household`: the household identifier.
- (Node) `site`: the study site (`Klerksdorp` or `Soweto`).
-- (Node) `age_group`: the age group in years (`<5`, `5-12`, `13-17`, `18-34`, `35-59` or `>=60`; the latter is written `=60` in the upstream file).
+- (Node) `age_group`: the age group in years (`<5`, `5-12`, `13-17`, `18-34`, `35-59` or `>=60`; the latter is written `=60` in the upstream data).
- (Node) `sex`: the sex of the individual (`Male` or `Female`).
- (Node) `role`: whether the individual is the SARS-CoV-2 index case of the household (`Index`) or a household contact (`Contact`).
- (Node) `sars_cov_2`: the SARS-CoV-2 infection status of the individual during follow-up (`Positive` or `Negative`).
[entries] Add the date when each entry was added, and remove icon_hash beff7c9
The dates are those of the commits that first added the entries' files (2020-08-11 for the entries of the initial commit); renamed entries keep the date of their first name. Changed: none
diff --git a/entries/sp_south_african_households/__init__.py b/entries/sp_south_african_households/__init__.py
index a09e37b..4bc8786 100644
--- a/entries/sp_south_african_households/__init__.py
+++ b/entries/sp_south_african_households/__init__.py
@@ -48,7 +48,7 @@ tags = ['Social', 'Offline', 'Temporal', 'Timestamps', 'Multigraph', 'Unweighted
url = 'https://sociopatterns.org/datasets/south-african-households-contact-networks/'
citation = [('J. Kleynhans, L. Dall\'Amico, L. Gauvin, M. Tizzoni, L. Maloma, S. Walaza, N. A. Martinson, A. von Gottberg, N. Wolter, M. Makhasi, C. Cohen, C. Cattuto, S. Tempia, "Association of close-range contact patterns with SARS-CoV-2: a household transmission study." eLife 12, e84753 (2023)', 'https://doi.org/10.7554/eLife.84753'),
('J. Kleynhans et al., "Household Transmission of Severe Acute Respiratory Syndrome Coronavirus 2 From Adult Index Cases With and Without Human Immunodeficiency Virus in South Africa, 2020-2021: A Case-Ascertained, Prospective, Observational Household Transmission Study." Clinical Infectious Diseases 76(3), e71-e81 (2023)', 'https://doi.org/10.1093/cid/ciac640')]
-icon_hash = None
+date_added = '2026-10-25'
upstream_license = ("Creative Commons CC0 1.0 Universal (Public Domain Dedication)",
"https://creativecommons.org/publicdomain/zero/1.0/")
upstream_prefix = 'https://raw.githubusercontent.com/crdm-nicd/sashts/7b43d0ceebc66b4fe879ff6aa314e9d725ca4bd1'
[entries] Prefix the properties in the descriptions with (Node), (Edge) or (Graph) bc3dbbd
The lists of properties at the end of the descriptions marked edge properties with "Edge" but left node properties unmarked, or grouped them in "Nodes: `a`, ...; `b`, ..." items, so the kind of a property was often unclear. Every item now starts with (Node), (Edge) or (Graph), followed by the property names, a colon and the explanation, one item per property or group of properties; qualifications (kinds of nodes, networks) follow the names in parentheses. Properties that were only named got a short explanation, and a few items that named the wrong kind of property were corrected.
diff --git a/entries/sp_south_african_households/__init__.py b/entries/sp_south_african_households/__init__.py
index 9d0a27e..a09e37b 100644
--- a/entries/sp_south_african_households/__init__.py
+++ b/entries/sp_south_african_households/__init__.py
@@ -26,24 +26,24 @@ description = r"""Temporal network of close-range proximity contacts between mem
Nodes are individuals, and each edge is a contact event between two members of the same household; two individuals are connected by as many edges as the number of their contact events. (The upstream file lists every event twice, once for each direction; here each event appears once.) The network has the following properties:
-- `id`: the identifier of the individual (site letter, `J` for Klerksdorp or `S` for Soweto, household number and individual number).
-- `household`: the household identifier.
-- `site`: the study site (`Klerksdorp` or `Soweto`).
-- `age_group`: the age group in years (`<5`, `5-12`, `13-17`, `18-34`, `35-59` or `>=60`; the latter is written `=60` in the upstream file).
-- `sex`: the sex of the individual (`Male` or `Female`).
-- `role`: whether the individual is the SARS-CoV-2 index case of the household (`Index`) or a household contact (`Contact`).
-- `sars_cov_2`: the SARS-CoV-2 infection status of the individual during follow-up (`Positive` or `Negative`).
-- `susceptible`: for household contacts, whether they were susceptible (`Susceptible`) or seropositive at baseline according to serology (empty for index cases).
-- `smoker`: whether the individual currently smokes cigarettes (`Yes`, `No` or `Unknown`).
-- `bmi`: body mass index category (`Underweight`, `Normal weight`, `Overweight`, `Obese` or `Unknown`).
-- `sleeps_with_index`: whether the individual sleeps in the same room as the index case (`Yes`, `No` or `Unknown`).
-- `cared_by_index`: whether the individual's main caregiver is the index case (`Yes` or `No`).
-- `index_age_group`, `index_sex`: age group and sex of the household's index case (empty for index cases).
-- `index_ct_cat1`, `index_ct_cat2`: category of the index case's minimum PCR cycle threshold (Ct) value during the episode, in two different binnings (`>35`, `25-35`, `<25` and `>35`, `30-35`, `<30`, respectively, or `Unknown`; empty for index cases). Lower values correspond to higher viral loads.
-- `index_variant`: the SARS-CoV-2 variant infecting the index case (`Alpha`, `Beta`, `Delta`, `non-Alpha/Beta/Delta` or `Variant Unknown`; empty for index cases).
-- `time`: the start of the contact event, as a UNIX timestamp in seconds (the upstream date-times are local times without a time zone, and are interpreted here as UTC). For the Klerksdorp sensors without a valid timestamp (see `timestamped`), times are relative, starting at 1 January 1970.
-- `duration`: the duration of the contact event in seconds (a multiple of 20 s).
-- `timestamped`: whether the time of the event is a real date-time (`True`) or a relative time of an unstamped Klerksdorp sensor (`False`)."""
+- (Node) `id`: the identifier of the individual (site letter, `J` for Klerksdorp or `S` for Soweto, household number and individual number).
+- (Node) `household`: the household identifier.
+- (Node) `site`: the study site (`Klerksdorp` or `Soweto`).
+- (Node) `age_group`: the age group in years (`<5`, `5-12`, `13-17`, `18-34`, `35-59` or `>=60`; the latter is written `=60` in the upstream file).
+- (Node) `sex`: the sex of the individual (`Male` or `Female`).
+- (Node) `role`: whether the individual is the SARS-CoV-2 index case of the household (`Index`) or a household contact (`Contact`).
+- (Node) `sars_cov_2`: the SARS-CoV-2 infection status of the individual during follow-up (`Positive` or `Negative`).
+- (Node) `susceptible`: for household contacts, whether they were susceptible (`Susceptible`) or seropositive at baseline according to serology (empty for index cases).
+- (Node) `smoker`: whether the individual currently smokes cigarettes (`Yes`, `No` or `Unknown`).
+- (Node) `bmi`: body mass index category (`Underweight`, `Normal weight`, `Overweight`, `Obese` or `Unknown`).
+- (Node) `sleeps_with_index`: whether the individual sleeps in the same room as the index case (`Yes`, `No` or `Unknown`).
+- (Node) `cared_by_index`: whether the individual's main caregiver is the index case (`Yes` or `No`).
+- (Node) `index_age_group`, `index_sex`: age group and sex of the household's index case (empty for index cases).
+- (Node) `index_ct_cat1`, `index_ct_cat2`: category of the index case's minimum PCR cycle threshold (Ct) value during the episode, in two different binnings (`>35`, `25-35`, `<25` and `>35`, `30-35`, `<30`, respectively, or `Unknown`; empty for index cases). Lower values correspond to higher viral loads.
+- (Node) `index_variant`: the SARS-CoV-2 variant infecting the index case (`Alpha`, `Beta`, `Delta`, `non-Alpha/Beta/Delta` or `Variant Unknown`; empty for index cases).
+- (Edge) `time`: the start of the contact event, as a UNIX timestamp in seconds (the upstream date-times are local times without a time zone, and are interpreted here as UTC). For the Klerksdorp sensors without a valid timestamp (see `timestamped`), times are relative, starting at 1 January 1970.
+- (Edge) `duration`: the duration of the contact event in seconds (a multiple of 20 s).
+- (Edge) `timestamped`: whether the time of the event is a real date-time (`True`) or a relative time of an unstamped Klerksdorp sensor (`False`)."""
tags = ['Social', 'Offline', 'Temporal', 'Timestamps', 'Multigraph', 'Unweighted', 'Metadata']
url = 'https://sociopatterns.org/datasets/south-african-households-contact-networks/'
citation = [('J. Kleynhans, L. Dall\'Amico, L. Gauvin, M. Tizzoni, L. Maloma, S. Walaza, N. A. Martinson, A. von Gottberg, N. Wolter, M. Makhasi, C. Cohen, C. Cattuto, S. Tempia, "Association of close-range contact patterns with SARS-CoV-2: a household transmission study." eLife 12, e84753 (2023)', 'https://doi.org/10.7554/eLife.84753'),
[entries] Remove unneeded code from the entries 2c69f19
Unneeded "global files" statements, unused imports and variables, f-strings without placeholders, duplicate keys in foodweb_little_rock (keeping the order and the values used), and the second definition of get_lines() in twitter_higgs. Nothing changes in the networks or descriptions. Changed: none
diff --git a/entries/sp_south_african_households/__init__.py b/entries/sp_south_african_households/__init__.py
index 7cc9f5e..9d0a27e 100644
--- a/entries/sp_south_african_households/__init__.py
+++ b/entries/sp_south_african_households/__init__.py
@@ -75,7 +75,6 @@ def _age(x):
@coerce_props()
@annotate()
def parse(alts=None):
- global files
name = __name__.split(".")[-1]
for fnames, alt, fmt in files:
if alts is not None and alt not in alts:
[entries] New entry: sp_south_african_households e2f4a01
diff --git a/entries/sp_south_african_households/__init__.py b/entries/sp_south_african_households/__init__.py
new file mode 100644
index 0000000..7cc9f5e
--- /dev/null
+++ b/entries/sp_south_african_households/__init__.py
@@ -0,0 +1,126 @@
+#! /usr/bin/env python
+# -*- coding: utf-8 -*-
+#
+# Copyright (C) 2020-2026 Tiago de Paula Peixoto <tiago@skewed.de>
+#
+# This program is free software: you can redistribute it and/or modify it under
+# the terms of the GNU Affero General Public License as published by the Free
+# Software Foundation, either version 3 of the License, or (at your option) any
+# later version.
+#
+# This program is distributed in the hope that it will be useful, but WITHOUT
+# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
+# FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
+# details.
+#
+# You should have received a copy of the GNU Affero General Public License along
+# with this program. If not, see <http://www.gnu.org/licenses/>.
+
+from .. import *
+
+import csv
+import datetime
+
+title = "South African household contacts (2020-2021)"
+description = r"""Temporal network of close-range proximity contacts between members of 88 households in two urban communities of South Africa, Klerksdorp (North West Province) and Soweto (Gauteng Province), collected in the SARS-CoV-2 household transmission study SA-S-HTS led by the National Institute for Communicable Diseases (NICD), in collaboration with SocioPatterns. After a symptomatic adult SARS-CoV-2 index case was identified at a clinic, the members of their household were followed up, and wearable proximity sensors were deployed for about two weeks (between November 2020 and August 2021) to measure close-range interactions (within about 1.5 m) between household members. Sensors exchange low-power radio packets, and a contact event between two sensors is a sequence of consecutive 20-second intervals in each of which at least one packet was exchanged. Events outside the deployment period and during 5-minute periods in which the sensor's accelerometer detected no movement were removed, as were individuals who did not comply or had sensor failures. Due to a technical error some sensors in Klerksdorp did not record valid timestamps; these were aligned relative to each other by cross-correlating packet time series, and their times start at the UNIX epoch (1 January 1970) instead of the real date. The data include 340 individuals (88 index cases and 252 household contacts), all of whom were included in the analysis of Kleynhans et al. (2023); contacts were only measured within households, so the network has one connected component per household.
+
+Nodes are individuals, and each edge is a contact event between two members of the same household; two individuals are connected by as many edges as the number of their contact events. (The upstream file lists every event twice, once for each direction; here each event appears once.) The network has the following properties:
+
+- `id`: the identifier of the individual (site letter, `J` for Klerksdorp or `S` for Soweto, household number and individual number).
+- `household`: the household identifier.
+- `site`: the study site (`Klerksdorp` or `Soweto`).
+- `age_group`: the age group in years (`<5`, `5-12`, `13-17`, `18-34`, `35-59` or `>=60`; the latter is written `=60` in the upstream file).
+- `sex`: the sex of the individual (`Male` or `Female`).
+- `role`: whether the individual is the SARS-CoV-2 index case of the household (`Index`) or a household contact (`Contact`).
+- `sars_cov_2`: the SARS-CoV-2 infection status of the individual during follow-up (`Positive` or `Negative`).
+- `susceptible`: for household contacts, whether they were susceptible (`Susceptible`) or seropositive at baseline according to serology (empty for index cases).
+- `smoker`: whether the individual currently smokes cigarettes (`Yes`, `No` or `Unknown`).
+- `bmi`: body mass index category (`Underweight`, `Normal weight`, `Overweight`, `Obese` or `Unknown`).
+- `sleeps_with_index`: whether the individual sleeps in the same room as the index case (`Yes`, `No` or `Unknown`).
+- `cared_by_index`: whether the individual's main caregiver is the index case (`Yes` or `No`).
+- `index_age_group`, `index_sex`: age group and sex of the household's index case (empty for index cases).
+- `index_ct_cat1`, `index_ct_cat2`: category of the index case's minimum PCR cycle threshold (Ct) value during the episode, in two different binnings (`>35`, `25-35`, `<25` and `>35`, `30-35`, `<30`, respectively, or `Unknown`; empty for index cases). Lower values correspond to higher viral loads.
+- `index_variant`: the SARS-CoV-2 variant infecting the index case (`Alpha`, `Beta`, `Delta`, `non-Alpha/Beta/Delta` or `Variant Unknown`; empty for index cases).
+- `time`: the start of the contact event, as a UNIX timestamp in seconds (the upstream date-times are local times without a time zone, and are interpreted here as UTC). For the Klerksdorp sensors without a valid timestamp (see `timestamped`), times are relative, starting at 1 January 1970.
+- `duration`: the duration of the contact event in seconds (a multiple of 20 s).
+- `timestamped`: whether the time of the event is a real date-time (`True`) or a relative time of an unstamped Klerksdorp sensor (`False`)."""
+tags = ['Social', 'Offline', 'Temporal', 'Timestamps', 'Multigraph', 'Unweighted', 'Metadata']
+url = 'https://sociopatterns.org/datasets/south-african-households-contact-networks/'
+citation = [('J. Kleynhans, L. Dall\'Amico, L. Gauvin, M. Tizzoni, L. Maloma, S. Walaza, N. A. Martinson, A. von Gottberg, N. Wolter, M. Makhasi, C. Cohen, C. Cattuto, S. Tempia, "Association of close-range contact patterns with SARS-CoV-2: a household transmission study." eLife 12, e84753 (2023)', 'https://doi.org/10.7554/eLife.84753'),
+ ('J. Kleynhans et al., "Household Transmission of Severe Acute Respiratory Syndrome Coronavirus 2 From Adult Index Cases With and Without Human Immunodeficiency Virus in South Africa, 2020-2021: A Case-Ascertained, Prospective, Observational Household Transmission Study." Clinical Infectious Diseases 76(3), e71-e81 (2023)', 'https://doi.org/10.1093/cid/ciac640')]
+icon_hash = None
+upstream_license = ("Creative Commons CC0 1.0 Universal (Public Domain Dedication)",
+ "https://creativecommons.org/publicdomain/zero/1.0/")
+upstream_prefix = 'https://raw.githubusercontent.com/crdm-nicd/sashts/7b43d0ceebc66b4fe879ff6aa314e9d725ca4bd1'
+files = [(('sashts_data.zip:sashts_contact_network.csv',
+ 'sashts_data.zip:sashts_metadata.csv'), None, None)]
+
+def fetch_upstream(force=False):
+ return fetch_upstream_files(__name__.split(".")[-1], upstream_prefix, files,
+ force)
+
+# upstream column -> vertex property
+_vprops = [("hhid", "household"), ("site", "site"), ("agegrp9", "age_group"),
+ ("sex", "sex"), ("index", "role"), ("sars", "sars_cov_2"),
+ ("sus", "susceptible"), ("smokecignow1", "smoker"),
+ ("bmicat", "bmi"), ("sleep_room_ix", "sleeps_with_index"),
+ ("cared_by_ix", "cared_by_index"), ("ixagegrp9", "index_age_group"),
+ ("ixsex", "index_sex"), ("ixminctcat1", "index_ct_cat1"),
+ ("ixminctcat2", "index_ct_cat2"), ("ixesarsvarf1", "index_variant")]
+
+def _age(x):
+ return ">=60" if x == "=60" else x
+
+@cache_network()
+@coerce_props()
+@annotate()
+def parse(alts=None):
+ global files
+ name = __name__.split(".")[-1]
+ for fnames, alt, fmt in files:
+ if alts is not None and alt not in alts:
+ continue
+ if isinstance(fnames, str):
+ fnames = [fnames]
+ with ExitStack() as stack:
+ fs = [stack.enter_context(open_upstream_file(name, fn, "r")) for fn in fnames]
+
+ g = Graph(directed=False)
+ g.ep.time = g.new_ep("int64_t")
+ g.ep.duration = g.new_ep("int")
+ g.ep.timestamped = g.new_ep("bool")
+ def get_edges():
+ for row in csv.DictReader(fs[0]):
+ u, v = row["indid1"], row["indid2"]
+ if u > v: # every event is listed in both directions
+ continue
+ t = datetime.datetime.strptime(row["t"], "%Y-%m-%d %H:%M:%S")
+ t = int(t.replace(tzinfo=datetime.timezone.utc).timestamp())
+ yield (u, v, t, int(row["duration_sec"]),
+ row["no_ts"].strip() != "1")
+ g.vp.id = g.add_edge_list(get_edges(), hashed=True,
+ hash_type="string",
+ eprops=[g.ep.time, g.ep.duration,
+ g.ep.timestamped])
+ vs = {g.vp.id[v] : v for v in g.vertices()}
+
+ for _, p in _vprops:
+ g.vp[p] = g.new_vp("string")
+ for row in csv.DictReader(fs[1]):
+ i = row["indid"]
+ if i not in vs:
+ v = g.add_vertex()
+ g.vp.id[v] = i
+ vs[i] = v
+ v = vs[i]
+ for k, p in _vprops:
+ x = row[k].strip()
+ if p in ("age_group", "index_age_group"):
+ x = _age(x)
+ g.vp[p][v] = x
+
+ # order nodes by identifier
+ rank = {x: r for r, x in enumerate(sorted(vs.keys()))}
+ order = g.new_vp("int", vals=[rank[g.vp.id[v]] for v in g.vertices()])
+ g = Graph(g, vorder=order)
+ yield alt, g