History of discogs_affiliation

The changes to the source code of this entry in the repository, newest first, each with the differences it made to the entry's files. They are also listed on GitLab.

2026-11-03 [entries] Add the date when each entry was added, and remove icon_hash by Tiago de Paula Peixoto beff7c9

The dates are those of the commits that first added the entries' files
(2020-08-11 for the entries of the initial commit); renamed entries keep
the date of their first name.

Changed: none

diff --git a/entries/discogs_affiliation/__init__.py b/entries/discogs_affiliation/__init__.py
index cf4a6d8..0da8b05 100644
--- a/entries/discogs_affiliation/__init__.py
+++ b/entries/discogs_affiliation/__init__.py
@@ -36,7 +36,7 @@ The network has the following properties:
 tags = ['Economic', 'Employment', 'Affiliation', 'Unweighted', 'Multigraph', 'Timestamps', 'Metadata']
 url = 'https://data.discogs.com/'
 citation = [('Discogs, "Discogs data dumps: releases, 1 October 2015."', 'https://data.discogs.com/')]
-icon_hash = '57342755461efe0c7a67830d'
+date_added = '2020-08-11'
 upstream_license = ("Creative Commons CC0 1.0 Universal (Public Domain Dedication)",
                     "https://creativecommons.org/publicdomain/zero/1.0/")
 dump = 'discogs_20151001_releases.xml.gz'

2026-11-03 [entries] Prefix the properties in the descriptions with (Node), (Edge) or (Graph) by Tiago de Paula Peixoto bc3dbbd

The lists of properties at the end of the descriptions marked edge
properties with "Edge" but left node properties unmarked, or grouped
them in "Nodes: `a`, ...; `b`, ..." items, so the kind of a property
was often unclear. Every item now starts with (Node), (Edge) or (Graph),
followed by the property names, a colon and the explanation, one item
per property or group of properties; qualifications (kinds of nodes,
networks) follow the names in parentheses. Properties that were only
named got a short explanation, and a few items that named the wrong
kind of property were corrected.

diff --git a/entries/discogs_affiliation/__init__.py b/entries/discogs_affiliation/__init__.py
index f676c7b..cf4a6d8 100644
--- a/entries/discogs_affiliation/__init__.py
+++ b/entries/discogs_affiliation/__init__.py
@@ -27,10 +27,12 @@ Artists are identified by their Discogs id, and their names are the Discogs name
 
 The network has the following properties:
 
-- `is_artist`: whether the node is an artist (otherwise it is a label).
-- `name`: the name of the artist or label.
-- `discogs_id`: the Discogs id of the artist (-1 for labels).
-- Edges: `release`, the Discogs id of the release; `year`, the release year as entered in Discogs (-1 if unknown; a few values are evidently mistyped, e.g. 198 or 4198); `extra`, whether the artist is credited only as an extra artist of the release (otherwise it is one of its main artists)."""
+- (Node) `is_artist`: whether the node is an artist (otherwise it is a label).
+- (Node) `name`: the name of the artist or label.
+- (Node) `discogs_id`: the Discogs id of the artist (-1 for labels).
+- (Edge) `release`: the Discogs id of the release.
+- (Edge) `year`: the release year as entered in Discogs (-1 if unknown; a few values are evidently mistyped, e.g. 198 or 4198).
+- (Edge) `extra`: whether the artist is credited only as an extra artist of the release (otherwise it is one of its main artists)."""
 tags = ['Economic', 'Employment', 'Affiliation', 'Unweighted', 'Multigraph', 'Timestamps', 'Metadata']
 url = 'https://data.discogs.com/'
 citation = [('Discogs, "Discogs data dumps: releases, 1 October 2015."', 'https://data.discogs.com/')]

2026-10-25 [entries] Move more KONECT-derived entries to their original sources by Tiago de Paula Peixoto bf9dc0e

The entries now download the data from where KONECT obtained them (live,
or via the Wayback Machine), using the same release as KONECT where it is
still available, and gain the metadata KONECT dropped; descriptions and
citations are rewritten from the original sources:

- Linton Freeman's UCINET data page: bible_nouns, bison, cattle, hens,
  highschool, kangaroo, macaques, moreno_sheep, physician_trust (the three
  relations and the physicians' attributes), rhesus_monkey,
  train_terrorists, windsurfers (now the networks `interaction` and
  `closeness`), college_freshmen, crime, interactome_yeast (from the
  primary source), moreno_taro (undirected), residence_hall, zebras
- other pages, archived: contiguous_usa, advogato, google, github,
  digg_reply, twitter, bookcrossing, dbtropes_feature (without KONECT's
  spurious "ItemName" tropes)
- movielens_100k (in fact the MovieLens 10M tag assignments)
- dblp release 2017-05-02: dblp_author_paper, dblp_cite, dblp_coauthor
- DBpedia 3.9: dbpedia_country, dbpedia_genre, dbpedia_location,
  dbpedia_occupation, dbpedia_producer, dbpedia_recordlabel,
  dbpedia_starring, dbpedia_team, dbpedia_writer, dbpedia_all, dbpedia_link
- Discogs releases dump of October 2015: discogs_affiliation,
  discogs_genre and discogs_label (now with genre and style networks)

diff --git a/entries/discogs_affiliation/__init__.py b/entries/discogs_affiliation/__init__.py
index 2e7e176..f676c7b 100644
--- a/entries/discogs_affiliation/__init__.py
+++ b/entries/discogs_affiliation/__init__.py
@@ -17,33 +17,112 @@
 # with this program.  If not, see <http://www.gnu.org/licenses/>.
 
 from .. import *
+import array
+from lxml import etree
 
-title = "Discogs artist-label affiliations"
-description = """A large bipartite network of the affiliations (contractual relations) among musical artists and record labels, as given in the discogs.com database. Each edge represents one release of an album by an artist with a particular label.[^icon]
-[^icon]: Description obtained from the [ICON](https://icon.colorado.edu) project."""
-tags = ['Economic', 'Employment', 'Unweighted', 'Multigraph']
-url = 'http://konect.cc/networks/discogs_affiliation'
-citation = [('J. Kunegis, "Discogs network dataset." KONECT (2016)', 'https://doi.org/10.1145/2487788.2488173')]
+title = "Discogs artist-label affiliations (2015)"
+description = """A bipartite network of musical artists and the record labels of their releases, from the Discogs database. Discogs (discogs.com) is a user-built online database of music recordings, whose contributors catalogue releases (albums, singles, compilations, etc.) together with their credits; its content is published as monthly XML dumps. This network is built from the releases dump of 1 October 2015, which contains the 6,379,659 releases accepted in the database at that time (drafts and the few rejected or deleted releases of the dump are excluded). Nodes are artists and labels, and an edge connects an artist to a label for each release of that label in which the artist is credited; repeated edges correspond to different releases. An artist is credited in a release either as one of its main artists or with a release-wide credit (e.g. producer, mixer, or performer on all tracks, the "extra artists"); credits limited to individual tracks are not included. When a release has several labels (co-releases, sublabels, distributors listed as labels), the artist is connected to each of them.
+
+Artists are identified by their Discogs id, and their names are the Discogs names, which include a numeric suffix (e.g. "John Smith (2)") to distinguish artists with the same name. The dump of 2015 does not contain label ids, and labels are therefore identified by their names, which are unique in Discogs. Discogs uses placeholder artists such as "Various" (for compilations) and "Unknown Artist", and placeholder labels such as "Not On Label" (with variants for self-released records), which are kept as nodes.
+
+The network has the following properties:
+
+- `is_artist`: whether the node is an artist (otherwise it is a label).
+- `name`: the name of the artist or label.
+- `discogs_id`: the Discogs id of the artist (-1 for labels).
+- Edges: `release`, the Discogs id of the release; `year`, the release year as entered in Discogs (-1 if unknown; a few values are evidently mistyped, e.g. 198 or 4198); `extra`, whether the artist is credited only as an extra artist of the release (otherwise it is one of its main artists)."""
+tags = ['Economic', 'Employment', 'Affiliation', 'Unweighted', 'Multigraph', 'Timestamps', 'Metadata']
+url = 'https://data.discogs.com/'
+citation = [('Discogs, "Discogs data dumps: releases, 1 October 2015."', 'https://data.discogs.com/')]
 icon_hash = '57342755461efe0c7a67830d'
-upstream_prefix = 'http://konect.cc/files'
-files = [('download.tsv.discogs_affiliation.tar.bz2', None, 'konect')]
+upstream_license = ("Creative Commons CC0 1.0 Universal (Public Domain Dedication)",
+                    "https://creativecommons.org/publicdomain/zero/1.0/")
+dump = 'discogs_20151001_releases.xml.gz'
+upstream_prefix = 'https://data.discogs.com/?download=data%2F2015%2F'
+files = [(dump, None, None)]
 
 def fetch_upstream(force=False):
-    return fetch_upstream_files(__name__.split(".")[-1], upstream_prefix, files,
-                                force)
+    # The files are served under a query string, so they are downloaded
+    # here, streaming to disk (the dump has 3.4 GB)
+    name = __name__.split(".")[-1]
+    fname = f"cache/upstream/{name}/{dump}"
+    if os.path.exists(fname) and not force:
+        return
+    os.makedirs(os.path.dirname(fname), exist_ok=True)
+    with open_url(upstream_prefix + dump) as f, open(fname + ".part", "wb") as out:
+        shutil.copyfileobj(f, out, 1 << 24)
+    os.replace(fname + ".part", fname)
+
+def read_releases(f):
+    """Iterate over the accepted releases of a Discogs releases XML dump, yielding
+    (id, year, labels, artists, extra_artists, genres, styles), where artists
+    are (id, name) pairs, and labels, genres and styles are lists of distinct
+    names."""
+    def year(s):
+        s = (s or "")[:4]
+        return int(s) if s.isdigit() and int(s) > 0 else -1
+    def distinct(xs):
+        return list(dict.fromkeys(x for x in xs if x))
+    def artists(el, path):
+        return [(int(a.findtext("id")), a.findtext("name") or "")
+                for a in el.iterfind(path) if (a.findtext("id") or "").isdigit()]
+    for ev, el in etree.iterparse(f, events=("end",), tag="release"):
+        # A few releases are drafts, or were rejected or deleted
+        if el.get("status") == "Accepted":
+            yield (int(el.get("id")), year(el.findtext("released")),
+                   distinct(l.get("name") for l in el.iterfind("labels/label")),
+                   artists(el, "artists/artist"),
+                   artists(el, "extraartists/artist"),
+                   distinct(x.text for x in el.iterfind("genres/genre")),
+                   distinct(x.text for x in el.iterfind("styles/style")))
+        el.clear()
+        while el.getprevious() is not None:
+            del el.getparent()[0]
 
 @cache_network()
-@coerce_props()
+@coerce_props(vforce={"name": "string"})
 @annotate()
 def parse(alts=None):
-    global files
     name = __name__.split(".")[-1]
-    for fnames, alt, fmt in files:
-        if alts is not None and alt not in alts:
-            continue
-        if isinstance(fnames, str):
-            fnames = [fnames]
-        with ExitStack() as stack:
-            fs = [stack.enter_context(open_upstream_file(name, fn, "rb")) for fn in fnames]
-            g = parse_graph(fs, fmt, directed=True)
-        yield alt, g
+    artists = {}                   # discogs id -> (index, name)
+    labels = {}                    # name -> index
+    src, tgt, rel, yr, extra = (array.array("l"), array.array("l"),
+                                array.array("l"), array.array("l"),
+                                array.array("b"))
+    with open_upstream_file(name, dump, "rb") as f:
+        for rid, year, ls, main, xtra, genres, styles in read_releases(f):
+            if not ls:
+                continue
+            credited = {}
+            for aid, aname in main:
+                credited.setdefault(aid, (aname, 0))
+            for aid, aname in xtra:
+                credited.setdefault(aid, (aname, 1))
+            lidx = [labels.setdefault(l, len(labels)) for l in ls]
+            for aid, (aname, x) in credited.items():
+                a = artists.setdefault(aid, (len(artists), aname))[0]
+                for l in lidx:
+                    src.append(a)
+                    tgt.append(l)
+                    rel.append(rid)
+                    yr.append(year)
+                    extra.append(x)
+    NA = len(artists)
+    g = Graph(directed=False)
+    g.add_vertex(NA + len(labels))
+    g.add_edge_list(numpy.column_stack((numpy.frombuffer(src, dtype="int64"),
+                                        numpy.frombuffer(tgt, dtype="int64") + NA)))
+    del src, tgt
+    g.ep.release = g.new_ep("int32_t", vals=numpy.frombuffer(rel, dtype="int64"))
+    g.ep.year = g.new_ep("int16_t", vals=numpy.frombuffer(yr, dtype="int64"))
+    g.ep.extra = g.new_ep("bool", vals=numpy.frombuffer(extra, dtype="int8"))
+    g.vp.is_artist = g.new_vp("bool")
+    g.vp.is_artist.a[:NA] = True
+    g.vp.name = g.new_vp("string")
+    g.vp.discogs_id = g.new_vp("int32_t", val=-1)
+    for aid, (i, aname) in artists.items():
+        g.vp.name[i] = aname
+        g.vp.discogs_id[i] = aid
+    for l, i in labels.items():
+        g.vp.name[NA + i] = l
+    yield None, g

2026-10-24 Update copyright information by Tiago de Paula Peixoto 73a4f0d

diff --git a/entries/discogs_affiliation/__init__.py b/entries/discogs_affiliation/__init__.py
index 799e01c..2e7e176 100644
--- a/entries/discogs_affiliation/__init__.py
+++ b/entries/discogs_affiliation/__init__.py
@@ -1,7 +1,7 @@
 #! /usr/bin/env python
 # -*- coding: utf-8 -*-
 #
-# Copyright (C) 2020-2024 Tiago de Paula Peixoto <tiago@skewed.de>
+# Copyright (C) 2020-2026 Tiago de Paula Peixoto <tiago@skewed.de>
 #
 # This program is free software: you can redistribute it and/or modify it under
 # the terms of the GNU Affero General Public License as published by the Free

2024-05-27 Update copyright information by Tiago de Paula Peixoto 3d288ea

diff --git a/entries/discogs_affiliation/__init__.py b/entries/discogs_affiliation/__init__.py
index 93609c2..799e01c 100644
--- a/entries/discogs_affiliation/__init__.py
+++ b/entries/discogs_affiliation/__init__.py
@@ -1,7 +1,7 @@
 #! /usr/bin/env python
 # -*- coding: utf-8 -*-
 #
-# Copyright (C) 2020-2023 Tiago de Paula Peixoto <tiago@skewed.de>
+# Copyright (C) 2020-2024 Tiago de Paula Peixoto <tiago@skewed.de>
 #
 # This program is free software: you can redistribute it and/or modify it under
 # the terms of the GNU Affero General Public License as published by the Free

2023-02-19 Update copyright information by Tiago de Paula Peixoto 8d005ae

diff --git a/entries/discogs_affiliation/__init__.py b/entries/discogs_affiliation/__init__.py
index c38b96d..93609c2 100644
--- a/entries/discogs_affiliation/__init__.py
+++ b/entries/discogs_affiliation/__init__.py
@@ -1,7 +1,7 @@
 #! /usr/bin/env python
 # -*- coding: utf-8 -*-
 #
-# Copyright (C) 2021 Tiago de Paula Peixoto <tiago@skewed.de>
+# Copyright (C) 2020-2023 Tiago de Paula Peixoto <tiago@skewed.de>
 #
 # This program is free software: you can redistribute it and/or modify it under
 # the terms of the GNU Affero General Public License as published by the Free

2021-06-22 Update copyright info by Tiago de Paula Peixoto db2efcc

diff --git a/entries/discogs_affiliation/__init__.py b/entries/discogs_affiliation/__init__.py
index 00532be..c38b96d 100644
--- a/entries/discogs_affiliation/__init__.py
+++ b/entries/discogs_affiliation/__init__.py
@@ -1,7 +1,7 @@
 #! /usr/bin/env python
 # -*- coding: utf-8 -*-
 #
-# Copyright (C) 2020 Tiago de Paula Peixoto <tiago@skewed.de>
+# Copyright (C) 2021 Tiago de Paula Peixoto <tiago@skewed.de>
 #
 # This program is free software: you can redistribute it and/or modify it under
 # the terms of the GNU Affero General Public License as published by the Free

2021-02-01 Un-orphan KONECT datasets by Tiago de Paula Peixoto 894eba5

KONECT has been brought back from the dead for the time being.

diff --git a/entries/discogs_affiliation/__init__.py b/entries/discogs_affiliation/__init__.py
index 9916d47..00532be 100644
--- a/entries/discogs_affiliation/__init__.py
+++ b/entries/discogs_affiliation/__init__.py
@@ -26,7 +26,6 @@ url = 'http://konect.cc/networks/discogs_affiliation'
 citation = [('J. Kunegis, "Discogs network dataset." KONECT (2016)', 'https://doi.org/10.1145/2487788.2488173')]
 icon_hash = '57342755461efe0c7a67830d'
 upstream_prefix = 'http://konect.cc/files'
-orphan = True
 files = [('download.tsv.discogs_affiliation.tar.bz2', None, 'konect')]
 
 def fetch_upstream(force=False):

2020-11-28 Update citations to use DOIs by Tiago de Paula Peixoto e878874

diff --git a/entries/discogs_affiliation/__init__.py b/entries/discogs_affiliation/__init__.py
index cb0cf52..9916d47 100644
--- a/entries/discogs_affiliation/__init__.py
+++ b/entries/discogs_affiliation/__init__.py
@@ -23,7 +23,7 @@ description = """A large bipartite network of the affiliations (contractual rela
 [^icon]: Description obtained from the [ICON](https://icon.colorado.edu) project."""
 tags = ['Economic', 'Employment', 'Unweighted', 'Multigraph']
 url = 'http://konect.cc/networks/discogs_affiliation'
-citation = [('J. Kunegis, "Discogs network dataset." KONECT (2016)', 'http://konect.cc/networks/discogs_affiliation')]
+citation = [('J. Kunegis, "Discogs network dataset." KONECT (2016)', 'https://doi.org/10.1145/2487788.2488173')]
 icon_hash = '57342755461efe0c7a67830d'
 upstream_prefix = 'http://konect.cc/files'
 orphan = True

2020-11-02 Add support for 'orphan' datasets and tag konect entries by Tiago de Paula Peixoto e330b80

diff --git a/entries/discogs_affiliation/__init__.py b/entries/discogs_affiliation/__init__.py
index e1dd828..cb0cf52 100644
--- a/entries/discogs_affiliation/__init__.py
+++ b/entries/discogs_affiliation/__init__.py
@@ -26,6 +26,7 @@ url = 'http://konect.cc/networks/discogs_affiliation'
 citation = [('J. Kunegis, "Discogs network dataset." KONECT (2016)', 'http://konect.cc/networks/discogs_affiliation')]
 icon_hash = '57342755461efe0c7a67830d'
 upstream_prefix = 'http://konect.cc/files'
+orphan = True
 files = [('download.tsv.discogs_affiliation.tar.bz2', None, 'konect')]
 
 def fetch_upstream(force=False):

2020-09-15 Change license from GPLv3 to AGPLv3 by Tiago de Paula Peixoto d754be5

diff --git a/entries/discogs_affiliation/__init__.py b/entries/discogs_affiliation/__init__.py
index 617b7dd..e1dd828 100644
--- a/entries/discogs_affiliation/__init__.py
+++ b/entries/discogs_affiliation/__init__.py
@@ -3,18 +3,18 @@
 #
 # Copyright (C) 2020 Tiago de Paula Peixoto <tiago@skewed.de>
 #
-# This program is free software: you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation, either version 3 of the License, or
-# (at your option) any later version.
+# This program is free software: you can redistribute it and/or modify it under
+# the terms of the GNU Affero General Public License as published by the Free
+# Software Foundation, either version 3 of the License, or (at your option) any
+# later version.
 #
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU General Public License for more details.
+# This program is distributed in the hope that it will be useful, but WITHOUT
+# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
+# FOR A PARTICULAR PURPOSE.  See the GNU Affero General Public License for more
+# details.
 #
-# You should have received a copy of the GNU General Public License
-# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+# You should have received a copy of the GNU Affero General Public License along
+# with this program.  If not, see <http://www.gnu.org/licenses/>.
 
 from .. import *

2020-09-13 Add explicit attribution to descriptions obtained from ICON by Tiago de Paula Peixoto 1645314

diff --git a/entries/discogs_affiliation/__init__.py b/entries/discogs_affiliation/__init__.py
index 5f29602..617b7dd 100644
--- a/entries/discogs_affiliation/__init__.py
+++ b/entries/discogs_affiliation/__init__.py
@@ -19,7 +19,8 @@
 from .. import *
 
 title = "Discogs artist-label affiliations"
-description = "A large bipartite network of the affiliations (contractual relations) among musical artists and record labels, as given in the discogs.com database. Each edge represents one release of an album by an artist with a particular label."
+description = """A large bipartite network of the affiliations (contractual relations) among musical artists and record labels, as given in the discogs.com database. Each edge represents one release of an album by an artist with a particular label.[^icon]
+[^icon]: Description obtained from the [ICON](https://icon.colorado.edu) project."""
 tags = ['Economic', 'Employment', 'Unweighted', 'Multigraph']
 url = 'http://konect.cc/networks/discogs_affiliation'
 citation = [('J. Kunegis, "Discogs network dataset." KONECT (2016)', 'http://konect.cc/networks/discogs_affiliation')]

2020-08-11 Initial commit by Tiago de Paula Peixoto cc85872

diff --git a/entries/discogs_affiliation/__init__.py b/entries/discogs_affiliation/__init__.py
new file mode 100644
index 0000000..5f29602
--- /dev/null
+++ b/entries/discogs_affiliation/__init__.py
@@ -0,0 +1,48 @@
+#! /usr/bin/env python
+# -*- coding: utf-8 -*-
+#
+# Copyright (C) 2020 Tiago de Paula Peixoto <tiago@skewed.de>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+from .. import *
+
+title = "Discogs artist-label affiliations"
+description = "A large bipartite network of the affiliations (contractual relations) among musical artists and record labels, as given in the discogs.com database. Each edge represents one release of an album by an artist with a particular label."
+tags = ['Economic', 'Employment', 'Unweighted', 'Multigraph']
+url = 'http://konect.cc/networks/discogs_affiliation'
+citation = [('J. Kunegis, "Discogs network dataset." KONECT (2016)', 'http://konect.cc/networks/discogs_affiliation')]
+icon_hash = '57342755461efe0c7a67830d'
+upstream_prefix = 'http://konect.cc/files'
+files = [('download.tsv.discogs_affiliation.tar.bz2', None, 'konect')]
+
+def fetch_upstream(force=False):
+    return fetch_upstream_files(__name__.split(".")[-1], upstream_prefix, files,
+                                force)
+
+@cache_network()
+@coerce_props()
+@annotate()
+def parse(alts=None):
+    global files
+    name = __name__.split(".")[-1]
+    for fnames, alt, fmt in files:
+        if alts is not None and alt not in alts:
+            continue
+        if isinstance(fnames, str):
+            fnames = [fnames]
+        with ExitStack() as stack:
+            fs = [stack.enter_context(open_upstream_file(name, fn, "rb")) for fn in fnames]
+            g = parse_graph(fs, fmt, directed=True)
+        yield alt, g