History of clinical_trials
The changes to the source code of this entry in the repository, newest first, each with the differences it made to the entry's files. They are also listed on GitLab.
[entries] Add the date when each entry was added, and remove icon_hash beff7c9
The dates are those of the commits that first added the entries' files (2020-08-11 for the entries of the initial commit); renamed entries keep the date of their first name. Changed: none
diff --git a/entries/clinical_trials/__init__.py b/entries/clinical_trials/__init__.py
index ddf2106..2b9b7a2 100644
--- a/entries/clinical_trials/__init__.py
+++ b/entries/clinical_trials/__init__.py
@@ -63,7 +63,7 @@ url = ['https://clinicaltrials.gov/',
'https://clinicaltrials.gov/data-api/api']
citation = [('A. Tasneem, L. Aberle, H. Ananth, S. Chakraborty, K. Chiswell, B. J. McCourt, R. Pietrobon, "The Database for Aggregate Analysis of ClinicalTrials.gov (AACT) and Subsequent Regrouping by Clinical Specialty." PLoS ONE 7(3), e33677 (2012)',
'https://doi.org/10.1371/journal.pone.0033677')]
-icon_hash = None
+date_added = '2026-10-31'
upstream_license = ("ClinicalTrials.gov terms and conditions (U.S. government data, no copyright)",
"https://clinicaltrials.gov/about-site/terms-conditions")
# The download of all the studies, replaced daily (not versioned)
[entries] Prefix the properties in the descriptions with (Node), (Edge) or (Graph) bc3dbbd
The lists of properties at the end of the descriptions marked edge properties with "Edge" but left node properties unmarked, or grouped them in "Nodes: `a`, ...; `b`, ..." items, so the kind of a property was often unclear. Every item now starts with (Node), (Edge) or (Graph), followed by the property names, a colon and the explanation, one item per property or group of properties; qualifications (kinds of nodes, networks) follow the names in parentheses. Properties that were only named got a short explanation, and a few items that named the wrong kind of property were corrected.
diff --git a/entries/clinical_trials/__init__.py b/entries/clinical_trials/__init__.py
index c02bc3e..ddf2106 100644
--- a/entries/clinical_trials/__init__.py
+++ b/entries/clinical_trials/__init__.py
@@ -38,16 +38,26 @@ The registry is updated daily, and its downloads are not versioned: the networks
The networks have the following properties:
-- `is_study`: whether the node is a study.
-- `name`: the NCT number of the study (its page is `https://clinicaltrials.gov/study/<name>`), the MeSH descriptor (e.g. `Lung Neoplasms`), the name of the organization, the name of the facility, or the name of the investigator (as given, often with degrees, e.g. `Jane Doe, MD`).
-- Studies: `title` (the brief title), `study_type` (`INTERVENTIONAL`, `OBSERVATIONAL`, `EXPANDED_ACCESS`), `phases` (e.g. `PHASE2|PHASE3`), `status` (the overall recruitment status, e.g. `COMPLETED`, `RECRUITING`), `start_date`, `completion_date` and `first_posted` (Unix time, -1 if unknown; dates given as a month are taken as its first day), `enrollment` (the number of participants, actual or anticipated; -1 if unknown), `has_results`, `sponsor_class` (the class of the lead sponsor: `INDUSTRY`, `NIH`, `FED`, `OTHER_GOV`, `NETWORK`, `OTHER`, ...), `conditions` (the conditions as given, separated by `|`), `intervention_names` (the interventions as given, as `<type>: <name>`, separated by `|`).
-- MeSH descriptors (`conditions`, `interventions`): `mesh_id` (e.g. `D008175`).
-- Organizations (`sponsors`): `org_class` (the class of the organization, as above).
-- Edge `role` (`sponsors`): `lead_sponsor` or `collaborator`.
-- Sites (`sites`): `city`, `state`, `zip`, `country`, `latitude`, `longitude` (`NaN` if unknown).
-- Edge `site_status` (`sites`): the recruitment status of the site, if given.
-- Investigators (`investigators`): `affiliation` (as given).
-- Edge `role` (`investigators`): the role of the investigator: `PRINCIPAL_INVESTIGATOR`, `STUDY_DIRECTOR`, `STUDY_CHAIR` (or empty if not given)."""
+- (Node) `is_study`: whether the node is a study.
+- (Node) `name`: the NCT number of the study (its page is `https://clinicaltrials.gov/study/<name>`), the MeSH descriptor (e.g. `Lung Neoplasms`), the name of the organization, the name of the facility, or the name of the investigator (as given, often with degrees, e.g. `Jane Doe, MD`).
+- (Node) `title` (studies): the brief title.
+- (Node) `study_type` (studies): `INTERVENTIONAL`, `OBSERVATIONAL`, `EXPANDED_ACCESS`.
+- (Node) `phases` (studies): the phases (e.g. `PHASE2|PHASE3`).
+- (Node) `status` (studies): the overall recruitment status, e.g. `COMPLETED`, `RECRUITING`.
+- (Node) `start_date`, `completion_date` and `first_posted` (studies): the dates of start, completion and first posting (Unix time, -1 if unknown; dates given as a month are taken as its first day).
+- (Node) `enrollment` (studies): the number of participants, actual or anticipated; -1 if unknown.
+- (Node) `has_results` (studies): whether the study has results.
+- (Node) `sponsor_class` (studies): the class of the lead sponsor: `INDUSTRY`, `NIH`, `FED`, `OTHER_GOV`, `NETWORK`, `OTHER`, ...
+- (Node) `conditions` (studies): the conditions as given, separated by `|`.
+- (Node) `intervention_names` (studies): the interventions as given, as `<type>: <name>`, separated by `|`.
+- (Node) `mesh_id` (MeSH descriptors, `conditions`, `interventions`): the MeSH identifier (e.g. `D008175`).
+- (Node) `org_class` (organizations, `sponsors`): the class of the organization, as above.
+- (Edge) `role` (`sponsors`): `lead_sponsor` or `collaborator`.
+- (Node) `city`, `state`, `zip`, `country` (sites, `sites`): the address of the site.
+- (Node) `latitude`, `longitude` (sites, `sites`): the coordinates of the site (`NaN` if unknown).
+- (Edge) `site_status` (`sites`): the recruitment status of the site, if given.
+- (Node) `affiliation` (investigators, `investigators`): the affiliation, as given.
+- (Edge) `role` (`investigators`): the role of the investigator: `PRINCIPAL_INVESTIGATOR`, `STUDY_DIRECTOR`, `STUDY_CHAIR` (or empty if not given)."""
tags = ['Biological', 'Public Health', 'Affiliation', 'Bipartite', 'Unweighted', 'Metadata']
url = ['https://clinicaltrials.gov/',
'https://clinicaltrials.gov/data-api/api']
[entries] New entry: clinical_trials b00a520
diff --git a/entries/clinical_trials/__init__.py b/entries/clinical_trials/__init__.py
new file mode 100644
index 0000000..c02bc3e
--- /dev/null
+++ b/entries/clinical_trials/__init__.py
@@ -0,0 +1,237 @@
+#! /usr/bin/env python
+# -*- coding: utf-8 -*-
+#
+# Copyright (C) 2026 Tiago de Paula Peixoto <tiago@skewed.de>
+#
+# This program is free software: you can redistribute it and/or modify it under
+# the terms of the GNU Affero General Public License as published by the Free
+# Software Foundation, either version 3 of the License, or (at your option) any
+# later version.
+#
+# This program is distributed in the hope that it will be useful, but WITHOUT
+# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
+# FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
+# details.
+#
+# You should have received a copy of the GNU Affero General Public License along
+# with this program. If not, see <http://www.gnu.org/licenses/>.
+
+
+import json
+import math
+import zipfile
+from calendar import timegm
+
+from .. import *
+import process_entry
+
+title = "Clinical trials and their conditions, interventions, sponsors, sites and investigators (ClinicalTrials.gov)"
+description = r"""The clinical studies registered in ClinicalTrials.gov, the registry of clinical research studies of the U.S. National Library of Medicine, which covers studies in the U.S. and in more than 200 other countries (about 610,000 studies in October 2026), with the conditions they study, the interventions they test, their sponsors and collaborators, the sites (facilities) where they take place, and their investigators. Each network is an undirected bipartite network of the studies and one kind of related entity:
+
+- `conditions`: studies and the conditions (diseases) they study, as the Medical Subject Headings (MeSH) descriptors assigned to the conditions by ClinicalTrials.gov.
+- `interventions`: studies and their interventions (drugs, devices, procedures, ...), as the MeSH descriptors assigned to the interventions by ClinicalTrials.gov.
+- `sponsors`: studies and the organizations that sponsor them (the lead sponsor and the collaborators), identified by their names as given.
+- `sites`: studies and their sites (facilities, e.g. hospitals), identified by their name, city, state and country as given.
+- `investigators`: studies and their investigators, i.e. the persons responsible for the scientific conduct of the study as given by the registry ("overall officials": principal investigators, study directors and study chairs), identified by their name and affiliation exactly as given (not disambiguated: the same person with differently written names or affiliations appears as several nodes, and different persons of the same name and affiliation are merged).
+
+The registry is updated daily, and its downloads are not versioned: the networks are of the download date (the data were checked on 31 October 2026). The contact persons of the studies (central and site contacts) and the investigators of the individual sites are left out, as are the free-text descriptions, the outcomes and the results of the studies. Studies whose conditions or interventions were not mapped to MeSH descriptors have no edges in those networks (their conditions and interventions as given are in the properties `conditions` and `intervention_names`).
+
+The networks have the following properties:
+
+- `is_study`: whether the node is a study.
+- `name`: the NCT number of the study (its page is `https://clinicaltrials.gov/study/<name>`), the MeSH descriptor (e.g. `Lung Neoplasms`), the name of the organization, the name of the facility, or the name of the investigator (as given, often with degrees, e.g. `Jane Doe, MD`).
+- Studies: `title` (the brief title), `study_type` (`INTERVENTIONAL`, `OBSERVATIONAL`, `EXPANDED_ACCESS`), `phases` (e.g. `PHASE2|PHASE3`), `status` (the overall recruitment status, e.g. `COMPLETED`, `RECRUITING`), `start_date`, `completion_date` and `first_posted` (Unix time, -1 if unknown; dates given as a month are taken as its first day), `enrollment` (the number of participants, actual or anticipated; -1 if unknown), `has_results`, `sponsor_class` (the class of the lead sponsor: `INDUSTRY`, `NIH`, `FED`, `OTHER_GOV`, `NETWORK`, `OTHER`, ...), `conditions` (the conditions as given, separated by `|`), `intervention_names` (the interventions as given, as `<type>: <name>`, separated by `|`).
+- MeSH descriptors (`conditions`, `interventions`): `mesh_id` (e.g. `D008175`).
+- Organizations (`sponsors`): `org_class` (the class of the organization, as above).
+- Edge `role` (`sponsors`): `lead_sponsor` or `collaborator`.
+- Sites (`sites`): `city`, `state`, `zip`, `country`, `latitude`, `longitude` (`NaN` if unknown).
+- Edge `site_status` (`sites`): the recruitment status of the site, if given.
+- Investigators (`investigators`): `affiliation` (as given).
+- Edge `role` (`investigators`): the role of the investigator: `PRINCIPAL_INVESTIGATOR`, `STUDY_DIRECTOR`, `STUDY_CHAIR` (or empty if not given)."""
+tags = ['Biological', 'Public Health', 'Affiliation', 'Bipartite', 'Unweighted', 'Metadata']
+url = ['https://clinicaltrials.gov/',
+ 'https://clinicaltrials.gov/data-api/api']
+citation = [('A. Tasneem, L. Aberle, H. Ananth, S. Chakraborty, K. Chiswell, B. J. McCourt, R. Pietrobon, "The Database for Aggregate Analysis of ClinicalTrials.gov (AACT) and Subsequent Regrouping by Clinical Specialty." PLoS ONE 7(3), e33677 (2012)',
+ 'https://doi.org/10.1371/journal.pone.0033677')]
+icon_hash = None
+upstream_license = ("ClinicalTrials.gov terms and conditions (U.S. government data, no copyright)",
+ "https://clinicaltrials.gov/about-site/terms-conditions")
+# The download of all the studies, replaced daily (not versioned)
+_url = "https://clinicaltrials.gov/api/v2/studies/download?format=json.zip"
+_fname = "ctg-studies.json.zip"
+upstream_prefix = None
+files = [(_fname, alt, None) for alt in ("conditions", "interventions",
+ "sponsors", "sites", "investigators")]
+
+def fetch_upstream(force=False):
+ name = __name__.split(".")[-1]
+ if process_entry._check_health:
+ with open_url(_url) as r:
+ r.read(1)
+ return
+ if not force and upstream_file_exists(name, _fname):
+ return
+ with open_url(_url) as f:
+ with open_upstream_file(name, _fname, "wb") as output:
+ copy_download(f, output, label=_fname)
+
+# For checks only: if set, only this many studies are read
+_check_rows = None
+
+def studies(name):
+ "The studies (as JSON objects) of the downloaded file."
+ with open_upstream_file(name, _fname, "rb", seekable=True) as f:
+ z = zipfile.ZipFile(f)
+ for i, n in enumerate(sorted(n for n in z.namelist()
+ if n.endswith(".json"))):
+ if _check_rows is not None and i >= _check_rows:
+ break
+ with z.open(n) as m:
+ yield json.load(m)
+
+def to_time(x):
+ "Dates YYYY-MM-DD or YYYY-MM as Unix time (-1 if missing)."
+ try:
+ parts = [int(y) for y in (x or "").split("-")]
+ while len(parts) < 3:
+ parts.append(1)
+ return timegm((parts[0], parts[1], parts[2], 0, 0, 0))
+ except (ValueError, TypeError):
+ return -1
+
+def study_props(s):
+ p = s.get("protocolSection", {})
+ ident = p.get("identificationModule", {})
+ status = p.get("statusModule", {})
+ design = p.get("designModule", {})
+ sponsors = p.get("sponsorCollaboratorsModule", {})
+ conds = p.get("conditionsModule", {})
+ arms = p.get("armsInterventionsModule", {})
+ enroll = (design.get("enrollmentInfo") or {}).get("count")
+ return dict(
+ name=ident.get("nctId", ""),
+ title=ident.get("briefTitle") or "",
+ study_type=design.get("studyType") or "",
+ phases="|".join(design.get("phases") or []),
+ status=status.get("overallStatus") or "",
+ start_date=to_time((status.get("startDateStruct") or {}).get("date")),
+ completion_date=to_time((status.get("completionDateStruct") or {}).get("date")),
+ first_posted=to_time((status.get("studyFirstPostDateStruct") or {}).get("date")),
+ enrollment=enroll if isinstance(enroll, int) else -1,
+ has_results=bool(s.get("hasResults")),
+ sponsor_class=(sponsors.get("leadSponsor") or {}).get("class") or "",
+ conditions="|".join(conds.get("conditions") or []),
+ intervention_names="|".join(f"{i.get('type', '')}: {i.get('name', '')}"
+ for i in arms.get("interventions") or []))
+
+def links(s, alt):
+ """The entities linked to the study in network `alt`, as (key, props,
+ edge props)."""
+ p = s.get("protocolSection", {})
+ d = s.get("derivedSection", {})
+ if alt in ("conditions", "interventions"):
+ mod = d.get("conditionBrowseModule" if alt == "conditions"
+ else "interventionBrowseModule") or {}
+ for m in mod.get("meshes") or []:
+ yield (m.get("term", ""), dict(mesh_id=m.get("id", "")), {})
+ elif alt == "sponsors":
+ sp = p.get("sponsorCollaboratorsModule", {})
+ lead = sp.get("leadSponsor")
+ if lead and lead.get("name"):
+ yield (lead["name"], dict(org_class=lead.get("class") or ""),
+ dict(role="lead_sponsor"))
+ for c in sp.get("collaborators") or []:
+ if c.get("name"):
+ yield (c["name"], dict(org_class=c.get("class") or ""),
+ dict(role="collaborator"))
+ elif alt == "investigators":
+ # (persons by their name and affiliation, as given)
+ for o in (p.get("contactsLocationsModule") or {}).get("overallOfficials") or []:
+ key = tuple((o.get(k) or "").strip() for k in ("name", "affiliation"))
+ if not key[0]:
+ continue
+ yield (key, {}, dict(role=o.get("role") or ""))
+ else:
+ for l in (p.get("contactsLocationsModule") or {}).get("locations") or []:
+ key = tuple(l.get(k) or "" for k in ("facility", "city", "state",
+ "country"))
+ if not any(key):
+ continue
+ geo = l.get("geoPoint") or {}
+ yield (key, dict(zip=l.get("zip") or "",
+ latitude=geo.get("lat", math.nan),
+ longitude=geo.get("lon", math.nan)),
+ dict(site_status=l.get("status") or ""))
+
+_sprops = {"title": "string", "study_type": "string", "phases": "string",
+ "status": "string", "start_date": "int64_t",
+ "completion_date": "int64_t", "first_posted": "int64_t",
+ "enrollment": "int64_t", "has_results": "bool",
+ "sponsor_class": "string", "conditions": "string",
+ "intervention_names": "string"}
+_oprops = {"conditions": {"mesh_id": "string"},
+ "interventions": {"mesh_id": "string"},
+ "sponsors": {"org_class": "string"},
+ "sites": {"city": "string", "state": "string", "zip": "string",
+ "country": "string", "latitude": "double",
+ "longitude": "double"},
+ "investigators": {"affiliation": "string"}}
+_eprops = {"sponsors": {"role": "string"}, "sites": {"site_status": "string"},
+ "investigators": {"role": "string"}}
+
+@cache_network()
+@coerce_props()
+@annotate()
+def parse(alts=None):
+ name = __name__.split(".")[-1]
+ for fname, alt, fmt in files:
+ if alts is not None and alt not in alts:
+ continue
+ g = Graph(directed=False)
+ g.vp.is_study = g.new_vp("bool")
+ g.vp.name = g.new_vp("string")
+ props = dict(_sprops, **_oprops[alt])
+ for k, t in props.items():
+ if k not in g.vp:
+ g.vp[k] = g.new_vp(t)
+ eprops = _eprops.get(alt, {})
+ for k, t in eprops.items():
+ g.ep[k] = g.new_ep(t)
+ others = {}
+ edges, evals = [], []
+ for s in studies(name):
+ v = g.add_vertex()
+ g.vp.is_study[v] = True
+ for k, x in study_props(s).items():
+ g.vp[k][v] = x
+ for key, op, ep in links(s, alt):
+ u = others.get(key)
+ if u is None:
+ u = g.add_vertex()
+ others[key] = int(u)
+ if alt == "sites":
+ g.vp.name[u] = key[0]
+ g.vp.city[u], g.vp.state[u] = key[1], key[2]
+ g.vp.country[u] = key[3]
+ elif alt == "investigators":
+ g.vp.name[u], g.vp.affiliation[u] = key
+ else:
+ g.vp.name[u] = key
+ for k, x in op.items():
+ g.vp[k][u] = x
+ for k in ("start_date", "completion_date", "first_posted",
+ "enrollment"):
+ g.vp[k][u] = -1
+ u = others[key]
+ edges.append((int(v), u))
+ evals.append(ep)
+ if alt == "sites":
+ is_study = g.vp.is_study.a.astype(bool)
+ g.vp.latitude.a[is_study] = math.nan
+ g.vp.longitude.a[is_study] = math.nan
+ g.add_edge_list(edges)
+ for k in eprops:
+ p = g.ep[k]
+ for e, x in zip(g.edges(), evals):
+ p[e] = x.get(k, "")
+ yield alt, g