History of primekg
The changes to the source code of this entry in the repository, newest first, each with the differences it made to the entry's files. They are also listed on GitLab.
[entries] Remove superfluous details of the upstream distribution from the descriptions dc07aa0
The descriptions mentioned incidental details of how the upstream data are made available: the file formats (CSV, JSON, XML, GEXF, SBML, HIF, Pajek, UCINET, ...), file names, where the files were published or obtained from (Zenodo, figshare, Kaggle, the Pajek datasets page, the Internet Archive), and that some networks are only available in the gt format. These are left out; caveats about the data themselves are kept, referring to "the data" instead of "the file". The description of sp_baboons now lists its properties instead of the upstream columns, and says that events without a recipient are left out.
diff --git a/entries/primekg/__init__.py b/entries/primekg/__init__.py
index e68cd79..5819083 100644
--- a/entries/primekg/__init__.py
+++ b/entries/primekg/__init__.py
@@ -27,12 +27,12 @@ description = r"""The Precision Medicine Knowledge Graph (PrimeKG) of Chandak, H
There are four networks, the final graph and the intermediate stages of its construction, as released by the authors (version 2.1 of the data set):
-- `kg`: the final knowledge graph (from `nodes.csv` and `edges.csv`).
-- `raw`: all the relationships of the integrated resources (`kg_raw.csv`).
-- `giant`: the largest connected component of `raw` (`kg_giant.csv`).
-- `grouped`: `giant` after grouping the diseases whose names correspond to subtypes of the same disease into single nodes (`kg_grouped.csv`); this has the same relationships as the final graph.
+- `kg`: the final knowledge graph.
+- `raw`: all the relationships of the integrated resources.
+- `giant`: the largest connected component of `raw`.
+- `grouped`: `giant` after grouping the diseases whose names correspond to subtypes of the same disease into single nodes; this has the same relationships as the final graph.
-The upstream files list each relationship in both directions; each is included here once. In the intermediate networks, nodes are identified by their identifier, type, name and source. As in the authors' final graph, 63 genes appear as two nodes, under two names (e.g. an old and a current gene symbol) with the same NCBI identifier.
+The upstream data list each relationship in both directions; each is included here once. In the intermediate networks, nodes are identified by their identifier, type, name and source. As in the authors' final graph, 63 genes appear as two nodes, under two names (e.g. an old and a current gene symbol) with the same NCBI identifier.
The networks have the following properties:
[entries] Add the date when each entry was added, and remove icon_hash beff7c9
The dates are those of the commits that first added the entries' files (2020-08-11 for the entries of the initial commit); renamed entries keep the date of their first name. Changed: none
diff --git a/entries/primekg/__init__.py b/entries/primekg/__init__.py
index 84afa11..e68cd79 100644
--- a/entries/primekg/__init__.py
+++ b/entries/primekg/__init__.py
@@ -45,7 +45,7 @@ The networks have the following properties:
tags = ['Biological', 'Informational', 'Unweighted', 'Multigraph', 'Metadata']
url = ['https://doi.org/10.7910/DVN/IXA7BM', 'https://github.com/mims-harvard/PrimeKG']
citation = [('P. Chandak, K. Huang, M. Zitnik, "Building a knowledge graph to enable precision medicine." Scientific Data 10, 67 (2023)', 'https://doi.org/10.1038/s41597-023-01960-3')]
-icon_hash = None
+date_added = '2026-10-27'
upstream_license = ("CC0 1.0", "https://creativecommons.org/publicdomain/zero/1.0/")
# Harvard Dataverse doi:10.7910/DVN/IXA7BM, version 2.1, files by id:
# 6180617 nodes.tab (tab-separated version of nodes.csv), 6180616 edges.csv,
[entries] Prefix the properties in the descriptions with (Node), (Edge) or (Graph) bc3dbbd
The lists of properties at the end of the descriptions marked edge properties with "Edge" but left node properties unmarked, or grouped them in "Nodes: `a`, ...; `b`, ..." items, so the kind of a property was often unclear. Every item now starts with (Node), (Edge) or (Graph), followed by the property names, a colon and the explanation, one item per property or group of properties; qualifications (kinds of nodes, networks) follow the names in parentheses. Properties that were only named got a short explanation, and a few items that named the wrong kind of property were corrected.
diff --git a/entries/primekg/__init__.py b/entries/primekg/__init__.py
index 61f6753..84afa11 100644
--- a/entries/primekg/__init__.py
+++ b/entries/primekg/__init__.py
@@ -36,12 +36,12 @@ The upstream files list each relationship in both directions; each is included h
The networks have the following properties:
-- `node_id`: the identifier of the entity in its source (e.g. an NCBI gene ID, a DrugBank ID, a MONDO ID).
-- `node_type`: the type of the entity (`gene/protein`, `drug`, `disease`, `effect/phenotype`, `exposure`, `anatomy`, `biological_process`, `molecular_function`, `cellular_component`, `pathway`).
-- `name`: the name of the entity.
-- `source`: the source of the identifier (e.g. `NCBI`, `DrugBank`, `MONDO`, `HPO`, `REACTOME`).
-- `relation` (edges): the type of the relationship (e.g. `protein_protein`, `drug_protein`, `indication`, `disease_phenotype_positive`).
-- `display_relation` (edges): a shorter name of the type of relationship (e.g. `ppi`, `target`, `indication`)."""
+- (Node) `node_id`: the identifier of the entity in its source (e.g. an NCBI gene ID, a DrugBank ID, a MONDO ID).
+- (Node) `node_type`: the type of the entity (`gene/protein`, `drug`, `disease`, `effect/phenotype`, `exposure`, `anatomy`, `biological_process`, `molecular_function`, `cellular_component`, `pathway`).
+- (Node) `name`: the name of the entity.
+- (Node) `source`: the source of the identifier (e.g. `NCBI`, `DrugBank`, `MONDO`, `HPO`, `REACTOME`).
+- (Edge) `relation` (edges): the type of the relationship (e.g. `protein_protein`, `drug_protein`, `indication`, `disease_phenotype_positive`).
+- (Edge) `display_relation` (edges): a shorter name of the type of relationship (e.g. `ppi`, `target`, `indication`)."""
tags = ['Biological', 'Informational', 'Unweighted', 'Multigraph', 'Metadata']
url = ['https://doi.org/10.7910/DVN/IXA7BM', 'https://github.com/mims-harvard/PrimeKG']
citation = [('P. Chandak, K. Huang, M. Zitnik, "Building a knowledge graph to enable precision medicine." Scientific Data 10, 67 (2023)', 'https://doi.org/10.1038/s41597-023-01960-3')]
[entries] New entry: primekg 9ac0429
diff --git a/entries/primekg/__init__.py b/entries/primekg/__init__.py
new file mode 100644
index 0000000..61f6753
--- /dev/null
+++ b/entries/primekg/__init__.py
@@ -0,0 +1,132 @@
+#! /usr/bin/env python
+# -*- coding: utf-8 -*-
+#
+# Copyright (C) 2020-2026 Tiago de Paula Peixoto <tiago@skewed.de>
+#
+# This program is free software: you can redistribute it and/or modify it under
+# the terms of the GNU Affero General Public License as published by the Free
+# Software Foundation, either version 3 of the License, or (at your option) any
+# later version.
+#
+# This program is distributed in the hope that it will be useful, but WITHOUT
+# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
+# FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
+# details.
+#
+# You should have received a copy of the GNU Affero General Public License along
+# with this program. If not, see <http://www.gnu.org/licenses/>.
+
+from .. import *
+import numpy as np
+import pandas as pd
+import pyarrow as pa
+import pyarrow.csv
+
+title = "PrimeKG precision medicine knowledge graph (2022)"
+description = r"""The Precision Medicine Knowledge Graph (PrimeKG) of Chandak, Huang and Zitnik, which integrates 20 biomedical resources (e.g. DrugBank, DisGeNET, the Human Phenotype Ontology, Reactome, Gene Ontology, UBERON, SIDER) into a single graph centered on diseases. Nodes are biological and medical entities of ten types (genes/proteins, drugs, diseases, phenotypes, exposures, anatomical regions, biological processes, molecular functions, cellular components and pathways), and an undirected edge is a relationship between two of them, of one of 30 types (e.g. protein-protein interaction, drug indication, disease-phenotype association, drug side effect).
+
+There are four networks, the final graph and the intermediate stages of its construction, as released by the authors (version 2.1 of the data set):
+
+- `kg`: the final knowledge graph (from `nodes.csv` and `edges.csv`).
+- `raw`: all the relationships of the integrated resources (`kg_raw.csv`).
+- `giant`: the largest connected component of `raw` (`kg_giant.csv`).
+- `grouped`: `giant` after grouping the diseases whose names correspond to subtypes of the same disease into single nodes (`kg_grouped.csv`); this has the same relationships as the final graph.
+
+The upstream files list each relationship in both directions; each is included here once. In the intermediate networks, nodes are identified by their identifier, type, name and source. As in the authors' final graph, 63 genes appear as two nodes, under two names (e.g. an old and a current gene symbol) with the same NCBI identifier.
+
+The networks have the following properties:
+
+- `node_id`: the identifier of the entity in its source (e.g. an NCBI gene ID, a DrugBank ID, a MONDO ID).
+- `node_type`: the type of the entity (`gene/protein`, `drug`, `disease`, `effect/phenotype`, `exposure`, `anatomy`, `biological_process`, `molecular_function`, `cellular_component`, `pathway`).
+- `name`: the name of the entity.
+- `source`: the source of the identifier (e.g. `NCBI`, `DrugBank`, `MONDO`, `HPO`, `REACTOME`).
+- `relation` (edges): the type of the relationship (e.g. `protein_protein`, `drug_protein`, `indication`, `disease_phenotype_positive`).
+- `display_relation` (edges): a shorter name of the type of relationship (e.g. `ppi`, `target`, `indication`)."""
+tags = ['Biological', 'Informational', 'Unweighted', 'Multigraph', 'Metadata']
+url = ['https://doi.org/10.7910/DVN/IXA7BM', 'https://github.com/mims-harvard/PrimeKG']
+citation = [('P. Chandak, K. Huang, M. Zitnik, "Building a knowledge graph to enable precision medicine." Scientific Data 10, 67 (2023)', 'https://doi.org/10.1038/s41597-023-01960-3')]
+icon_hash = None
+upstream_license = ("CC0 1.0", "https://creativecommons.org/publicdomain/zero/1.0/")
+# Harvard Dataverse doi:10.7910/DVN/IXA7BM, version 2.1, files by id:
+# 6180617 nodes.tab (tab-separated version of nodes.csv), 6180616 edges.csv,
+# 6180622 kg_raw.csv, 6180625 kg_giant.csv, 6180626 kg_grouped.csv
+upstream_prefix = 'https://dataverse.harvard.edu/api/access/datafile'
+files = [(("6180617", "6180616"), "kg", None),
+ ("6180622", "raw", None),
+ ("6180625", "giant", None),
+ ("6180626", "grouped", None)]
+
+def fetch_upstream(force=False):
+ return fetch_upstream_files(__name__.split(".")[-1], upstream_prefix, files,
+ force)
+
+def read_csv(f, columns, delimiter=","):
+ "Read the columns of a CSV file with pyarrow, as dictionary-encoded strings."
+ opts = pa.csv.ConvertOptions(include_columns=columns,
+ column_types={c: pa.string() for c in columns},
+ auto_dict_encode=True)
+ return pa.csv.read_csv(f, parse_options=pa.csv.ParseOptions(delimiter=delimiter),
+ convert_options=opts)
+
+def undirected(x, y, extra):
+ """Keep each relationship once, checking that the upstream file lists it
+ in both directions."""
+ d = pd.DataFrame({"x": x, "y": y, **extra})
+ fwd = d[d["x"] < d["y"]]
+ bwd = d[d["x"] > d["y"]]
+ loops = d[d["x"] == d["y"]]
+ if len(fwd) != len(bwd):
+ print(f"warning: {len(fwd)} relationships in one direction, {len(bwd)} in the other")
+ bwd = bwd.rename(columns={"x": "y", "y": "x"})
+ fwd = pd.concat([fwd, bwd]).drop_duplicates()
+ return pd.concat([fwd, loops])
+
+@cache_network()
+@coerce_props()
+@annotate()
+def parse(alts=None):
+ name = __name__.split(".")[-1]
+ for fnames, alt, fmt in files:
+ if alts is not None and alt not in alts:
+ continue
+ g = Graph(directed=False)
+ if alt == "kg":
+ with open_upstream_file(name, fnames[0], "rb") as f:
+ nodes = pd.read_csv(f, sep="\t", dtype=str, keep_default_na=False)
+ assert (nodes["node_index"].astype(int).to_numpy() == np.arange(len(nodes))).all()
+ nodes = nodes.rename(columns={"node_name": "name", "node_source": "source"})
+ with open_upstream_file(name, fnames[1], "rb") as f:
+ t = read_csv(f, ["relation", "display_relation", "x_index", "y_index"])
+ x = t["x_index"].to_pandas().astype(int).to_numpy()
+ y = t["y_index"].to_pandas().astype(int).to_numpy()
+ extra = {c: t[c].to_pandas() for c in ["relation", "display_relation"]}
+ del t
+ else:
+ cols = ["relation", "display_relation"] + \
+ [f"{s}_{c}" for s in "xy" for c in ["id", "type", "name", "source"]]
+ with open_upstream_file(name, fnames, "rb") as f:
+ t = read_csv(f, cols)
+ keys = {}
+ for s in "xy":
+ k = pd.DataFrame({c: t[f"{s}_{c}"].to_pandas().astype(str)
+ for c in ["id", "type", "source", "name"]})
+ keys[s] = k
+ extra = {c: t[c].to_pandas() for c in ["relation", "display_relation"]}
+ del t
+ allk = pd.concat([keys["x"], keys["y"]])
+ nodes = allk.drop_duplicates().reset_index(drop=True)
+ nodes = nodes.rename(columns={"id": "node_id", "type": "node_type"})
+ idx = pd.Series(np.arange(len(nodes)),
+ index=pd.MultiIndex.from_frame(nodes[["node_id", "node_type", "source", "name"]]))
+ x = idx.loc[pd.MultiIndex.from_frame(keys["x"][["id", "type", "source", "name"]])].to_numpy()
+ y = idx.loc[pd.MultiIndex.from_frame(keys["y"][["id", "type", "source", "name"]])].to_numpy()
+ del keys, allk
+ g.add_vertex(len(nodes))
+ for c in ["node_id", "node_type", "name", "source"]:
+ g.vp[c] = g.new_vp("string", vals=nodes[c].astype(str).tolist())
+ d = undirected(x, y, extra)
+ del x, y, extra
+ g.add_edge_list(d[["x", "y"]].to_numpy())
+ for c in ["relation", "display_relation"]:
+ g.ep[c] = g.new_ep("string", vals=d[c].astype(str).tolist())
+ yield alt, g