History of bioplex
The changes to the source code of this entry in the repository, newest first, each with the differences it made to the entry's files. They are also listed on GitLab.
[entries] New entry: bioplex 55d69e1
diff --git a/entries/bioplex/__init__.py b/entries/bioplex/__init__.py
new file mode 100644
index 0000000..72aaeb8
--- /dev/null
+++ b/entries/bioplex/__init__.py
@@ -0,0 +1,305 @@
+#! /usr/bin/env python
+# -*- coding: utf-8 -*-
+#
+# Copyright (C) 2020-2026 Tiago de Paula Peixoto <tiago@skewed.de>
+#
+# This program is free software: you can redistribute it and/or modify it under
+# the terms of the GNU Affero General Public License as published by the Free
+# Software Foundation, either version 3 of the License, or (at your option) any
+# later version.
+#
+# This program is distributed in the hope that it will be useful, but WITHOUT
+# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
+# FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
+# details.
+#
+# You should have received a copy of the GNU Affero General Public License along
+# with this program. If not, see <http://www.gnu.org/licenses/>.
+
+
+from .. import *
+from array import array
+
+title = "BioPlex human protein co-complex interactomes (2015-2022)"
+description = """Networks of protein interactions in human cell lines from the BioPlex project of the Gygi and Harper labs (Harvard Medical School). BioPlex maps the human interactome by affinity purification and mass spectrometry (AP-MS): each of thousands of human proteins (the *baits*, expressed from the ORFeome collection with an epitope tag) is purified from the cells together with the proteins bound to it (the *preys*), which are identified by mass spectrometry. The candidate bait-prey pairs are scored with the CompPASS-Plus classifier, which estimates for each pair the probabilities of a wrong protein identification, of a nonspecific (background) association, and of a genuine interaction; the published networks keep the pairs classified as interactions. Since the proteins in a purification may be bound indirectly, through other members of the same complex, the interactions are co-complex associations rather than necessarily direct (binary) contacts. Nodes are proteins, identified by the Entrez (NCBI) Gene IDs of the genes that encode them.
+
+The entry contains every network offered for download on the BioPlex site:
+
+- `1.0_293T`, `2.0_293T`, `3.0_293T`, `3.0_HCT116`: the official undirected networks of BioPlex 1.0 (2,594 baits in HEK293T cells, Huttlin et al. 2015), BioPlex 2.0 (5,891 baits in HEK293T cells, Huttlin et al. 2017), and BioPlex 3.0 (10,128 baits in HEK293T cells and 5,522 baits in HCT116 cells, Huttlin et al. 2021). An undirected edge joins two proteins found to interact, whichever was the bait.
+- `1.0_293T_directed`, `2.0_293T_directed`, `3.0_293T_directed`, `3.0_HCT116_directed`: the same networks as directed bait-prey interactions, with a directed edge from the bait to each of its interacting preys (as distributed separately by the BioPlex site; a pair of proteins that found each other as baits is joined by two opposite edges, so that the numbers of edges differ slightly from the undirected networks).
+- `2.0_293T_unfiltered`, `3.0_293T_unfiltered`, `3.0_HCT116_unfiltered`: all the candidate bait-prey pairs of BioPlex 2.0 and of BioPlex 3.0 (HEK293T and HCT116 cells) before filtering, i.e. every protein detected in the purification of every bait, with the CompPASS scores and classifier probabilities (the BioPlex site warns that these lists contain background and nonspecific interactors). A directed edge goes from the bait to each protein detected with it, so that the bait's own detection is a self-loop. Each edge is one identified protein (database entry), so that several isoforms or entries of the same gene give multiple edges between the same pair of nodes.
+- `293T_2015-12`, `293T_2016-05`, `293T_2016-08`, `293T_2016-12`, `293T_2017-04`, `293T_2017-10`: the quarterly releases of interactions of new baits in HEK293T cells (U41 grant), released before their publication in BioPlex 3.0.
+- `HCT116_2018-11`, `HCT116_2019-03`, `HCT116_2019-05`, `HCT116_2019-08`, `HCT116_2019-11`, `HCT116_2020-02`, `HCT116_2020-05`, `HCT116_2021-02`, `HCT116_2021-05`, `HCT116_2021-08`, `HCT116_2021-11`: the quarterly releases of interactions of new baits in HCT116 cells (U24 grant); the releases of August and November 2020 were not made because of the COVID-19 shutdown.
+- `U2OS_0.1`, `RPE1_0.1`: preliminary networks (version 0.1) of 2,279 baits in U2OS cells (released in August 2021) and of 1,925 baits in RPE-1 cells (released in February 2022).
+
+In the quarterly releases and the preliminary networks, a directed edge goes from the bait to each interacting prey (as classified by CompPASS-Plus), including the bait itself as a self-loop when it was detected, and, in the quarterly releases, each edge is one identified protein, so that several isoforms or entries of the same gene give multiple edges. The BioPlex site describes the quarterly releases and the preliminary networks as unpublished data, and asks users not to publish large-scale analyses of them before they are published by the authors. In the unfiltered lists and the quarterly releases, the few detected proteins without a Gene ID are identified by their UniProt entry instead (e.g. `sp|P01891|1A68_HUMAN`). Baits that were not found to interact with anything do not appear in the networks. The descriptions of the proteins in the unfiltered lists (taken from UniProt) and the proteome comparison of HEK293T and HCT116 cells (also available on the site) are not included.
+
+The networks have the following properties:
+
+- `name`: the Entrez Gene ID of the protein (or the UniProt entry of a detected protein without a Gene ID); `symbol`: the gene symbol; `is_bait`: whether the protein was used as a bait (from the lists of baits of the official networks, and otherwise whether the node is the source of edges).
+- In the official undirected networks, `uniprot`: the UniProt accession of the protein (or isoform) identified; edge `p_wrong_id`, `p_no_interaction`, `p_interaction`: the CompPASS-Plus probabilities that the pair is a wrong identification, a nonspecific (background) association, or a genuine interaction.
+- In the unfiltered lists, the quarterly releases and the preliminary networks, edge `p_wrong_id`, `p_no_interaction`, `p_interaction` as above.
+- In the unfiltered lists and the quarterly releases, edge `protein_id`: the UniProt entry of the identified protein (e.g. `sp|P05387|RLA2_HUMAN`).
+- In the unfiltered lists, the CompPASS scores of the pair (Sowa et al. 2009, Huttlin et al. 2015): edge `ave_apsm` (the average number of assembled peptide spectral matches of the prey), `nwd_score` (normalized weighted D score), `z_score`, `plate_z_score`, `entropy`, `unique_peptides`, `ratio`, `total_psms`, `ratio_total_psms`, `unique_to_total_ratio` (named `ave_apsm`, `nwdscore`, `zscore`, `plate_zscore`, `entropy`, `uPeps`, `ratio`, `total_psms`, `ratioTotalPSMs` and `UtoTratio` upstream); vertex `compass_id`: the CompPASS id of the bait's purification (-1 for proteins that were not baits).
+- In the quarterly releases and the preliminary networks, edge `plate` and `well`: the plate and well of the purification."""
+tags = ['Biological', 'Protein interactions', 'Unweighted', 'Multigraph', 'Metadata']
+url = 'https://bioplex.hms.harvard.edu/interactions.php'
+citation = [('E. L. Huttlin, L. Ting, R. J. Bruckner, F. Gebreab, M. P. Gygi, J. Szpyt, et al., "The BioPlex Network: A Systematic Exploration of the Human Interactome." Cell 162, 425-440 (2015)',
+ 'https://doi.org/10.1016/j.cell.2015.06.043'),
+ ('E. L. Huttlin, R. J. Bruckner, J. A. Paulo, J. R. Cannon, L. Ting, K. Baltier, et al., "Architecture of the human interactome defines protein communities and disease networks." Nature 545, 505-509 (2017)',
+ 'https://doi.org/10.1038/nature22366'),
+ ('E. L. Huttlin, R. J. Bruckner, J. Navarrete-Perea, J. R. Cannon, K. Baltier, F. Gebreab, et al., "Dual proteome-scale networks reveal cell-specific remodeling of the human interactome." Cell 184, 3022-3040 (2021)',
+ 'https://doi.org/10.1016/j.cell.2021.04.011')]
+icon_hash = None
+upstream_license = None
+upstream_prefix = 'https://bioplex.hms.harvard.edu/data'
+
+# Official networks: (alt, undirected list, directed list, bait list)
+official = [('1.0_293T', 'BioPlex_interactionList_v2.tsv',
+ 'BioPlex_1.0_293T_DirectedEdges.tsv', 'BioPlex_1p0_293T_baitList.tsv'),
+ ('2.0_293T', 'BioPlex_interactionList_v4a.tsv',
+ 'BioPlex_2.0_293T_DirectedEdges.tsv', 'BioPlex_2p0_293T_baitList.tsv'),
+ ('3.0_293T', 'BioPlex_293T_Network_10K_Dec_2019.tsv',
+ 'BioPlex_3.0_293T_DirectedEdges.tsv', 'BioPlex_3p0_293T_baitList.tsv'),
+ ('3.0_HCT116', 'BioPlex_HCT116_Network_5.5K_Dec_2019.tsv',
+ 'BioPlex_3.0_HCT116_DirectedEdges.tsv', 'BioPlex_3p0_HCT116_baitList.tsv')]
+unfiltered = [('2.0_293T_unfiltered', 'BaitPreyPairs_noFilters_BP2a.tsv'),
+ ('3.0_293T_unfiltered', 'BioPlex_BaitPreyPairs_noFilters_293T_10K_Dec_2019.tsv'),
+ ('3.0_HCT116_unfiltered', 'BioPlex_BaitPreyPairs_noFilters_HCT116_5.5K_Dec_2019.tsv')]
+releases = [('293T_2015-12', 'interactome_update_Dec2015.tsv'),
+ ('293T_2016-05', 'interactome_update_May2016.tsv'),
+ ('293T_2016-08', 'interactome_update_Aug2016.tsv'),
+ ('293T_2016-12', 'interactome_update_Dec2016.tsv'),
+ ('293T_2017-04', 'interactome_update_April2017.tsv'),
+ ('293T_2017-10', 'interactome_update_Nov2017.tsv'),
+ ('HCT116_2018-11', 'HCT116_data_release_Nov_2018.tsv'),
+ ('HCT116_2019-03', 'HCT116_data_release_March_2019.tsv'),
+ ('HCT116_2019-05', 'HCT116_data_release_May_2019.tsv'),
+ ('HCT116_2019-08', 'HCT116_data_release_August_2019.tsv'),
+ ('HCT116_2019-11', 'HCT116_data_release_November_2019.tsv'),
+ ('HCT116_2020-02', 'HCT116_data_release_February_2020.tsv'),
+ ('HCT116_2020-05', 'HCT116_data_release_May_2020.tsv'),
+ ('HCT116_2021-02', 'HCT116_data_release_February_2021.tsv'),
+ ('HCT116_2021-05', 'HCT116_data_release_May_2021.tsv'),
+ ('HCT116_2021-08', 'HCT116_data_release_August_2021.tsv'),
+ ('HCT116_2021-11', 'HCT116_data_release_November_2021.tsv'),
+ ('U2OS_0.1', 'BioPlex_U2OS_v0.1.tsv'),
+ ('RPE1_0.1', 'BioPlex_RPE1_v0.1.tsv')]
+
+files = [((und, bl), alt, None) for alt, und, d, bl in official]
+files += [((d, bl), alt + '_directed', None) for alt, und, d, bl in official]
+files += [(fn, alt, None) for alt, fn in unfiltered + releases]
+
+def fetch_upstream(force=False):
+ return fetch_upstream_files(__name__.split(".")[-1], upstream_prefix, files,
+ force)
+
+def _rows(f):
+ """The rows of a tab-separated file, with the quotes around fields
+ removed (the files have no quotes or tabs within fields)."""
+ for l in f:
+ l = l.rstrip("\r\n")
+ if not l.strip():
+ continue
+ yield [x[1:-1] if len(x) >= 2 and x[0] == '"' and x[-1] == '"' else x
+ for x in l.split("\t")]
+
+# Columns of the unfiltered lists: (upstream name, property name, type)
+compass_cols = [('ave_apsm', 'ave_apsm', 'd'), ('nwdscore', 'nwd_score', 'd'),
+ ('zscore', 'z_score', 'd'), ('plate_zscore', 'plate_z_score', 'd'),
+ ('entropy', 'entropy', 'd'), ('uPeps', 'unique_peptides', 'l'),
+ ('ratio', 'ratio', 'd'), ('total_psms', 'total_psms', 'l'),
+ ('ratioTotalPSMs', 'ratio_total_psms', 'd'),
+ ('UtoTratio', 'unique_to_total_ratio', 'd'),
+ ('pWrongID', 'p_wrong_id', 'd'), ('pNoInt', 'p_no_interaction', 'd'),
+ ('pInt', 'p_interaction', 'd')]
+
+class _Builder:
+ """Accumulates the vertices and edges of a large network in compact
+ arrays."""
+ def __init__(self):
+ self.idx = {}
+ self.symbol = []
+ self.src = array('l')
+ self.tgt = array('l')
+
+ def vertex(self, name, symbol):
+ v = self.idx.get(name)
+ if v is None:
+ v = self.idx[name] = len(self.symbol)
+ self.symbol.append(symbol)
+ elif not self.symbol[v]:
+ self.symbol[v] = symbol
+ return v
+
+ def graph(self, baits):
+ g = Graph(directed=True)
+ g.add_vertex(len(self.symbol))
+ g.add_edge_list(numpy.array([numpy.frombuffer(self.src, dtype="int64"),
+ numpy.frombuffer(self.tgt, dtype="int64")]).T)
+ g.vp.name = g.new_vp("string", vals=list(self.idx.keys()))
+ g.vp.symbol = g.new_vp("string", vals=self.symbol)
+ is_bait = g.new_vp("bool")
+ for v in baits:
+ is_bait.a[v] = True
+ g.vp.is_bait = is_bait
+ return g
+
+def _gene(gene_id, protein_id):
+ if gene_id in ("", "NA"):
+ assert protein_id
+ return protein_id
+ return gene_id
+
+@cache_network()
+@coerce_props(vforce={'name': 'string', 'symbol': 'string', 'uniprot': 'string'},
+ eforce={'protein_id': 'string', 'plate': 'string',
+ 'well': 'string'})
+@annotate()
+def parse(alts=None):
+ name = __name__.split(".")[-1]
+
+ def read_baits(fname):
+ with open_upstream_file(name, fname, "r") as f:
+ rows = list(_rows(f))
+ assert rows[0][:2] == ["Symbol", "GeneID"], rows[0]
+ return {r[1]: r[0] for r in rows[1:]}
+
+ for alt, und, dfname, bl in official:
+ if alts is None or alt in alts:
+ baits = read_baits(bl)
+ with open_upstream_file(name, und, "r") as f:
+ rows = list(_rows(f))
+ assert len(rows[0]) == 9 and rows[0][0].replace(" ", "") == "GeneA", rows[0]
+ rows = rows[1:]
+ assert all(len(r) == 9 for r in rows)
+ g = Graph(directed=False)
+ p_wrong = g.new_ep("double")
+ p_no = g.new_ep("double")
+ p_int = g.new_ep("double")
+ g.vp.name = g.add_edge_list(((r[0], r[1], float(r[6]), float(r[7]),
+ float(r[8])) for r in rows),
+ hashed=True, hash_type="string",
+ eprops=[p_wrong, p_no, p_int])
+ vidx = {g.vp.name[v]: v for v in g.vertices()}
+ symbol = g.new_vp("string")
+ uniprot = g.new_vp("string")
+ for r in rows:
+ for gid, up, sym in [(r[0], r[2], r[4]), (r[1], r[3], r[5])]:
+ v = vidx[gid]
+ symbol[v] = sym
+ uniprot[v] = up
+ is_bait = g.new_vp("bool")
+ for gid in baits:
+ if gid in vidx:
+ is_bait[vidx[gid]] = True
+ g.vp.symbol = symbol
+ g.vp.uniprot = uniprot
+ g.vp.is_bait = is_bait
+ g.ep.p_wrong_id = p_wrong
+ g.ep.p_no_interaction = p_no
+ g.ep.p_interaction = p_int
+ yield alt, g
+
+ if alts is None or alt + "_directed" in alts:
+ baits = read_baits(bl)
+ with open_upstream_file(name, dfname, "r") as f:
+ rows = list(_rows(f))
+ assert rows[0] == ["Bait GeneID", "Bait Symbol", "Prey GeneID",
+ "Prey Symbol"], rows[0]
+ rows = rows[1:]
+ assert all(len(r) == 4 for r in rows)
+ g = Graph(directed=True)
+ g.vp.name = g.add_edge_list(((r[0], r[2]) for r in rows),
+ hashed=True, hash_type="string")
+ vidx = {g.vp.name[v]: v for v in g.vertices()}
+ symbol = g.new_vp("string")
+ for r in rows:
+ symbol[vidx[r[0]]] = r[1]
+ symbol[vidx[r[2]]] = r[3]
+ is_bait = g.new_vp("bool")
+ for gid in baits:
+ if gid in vidx:
+ is_bait[vidx[gid]] = True
+ g.vp.symbol = symbol
+ g.vp.is_bait = is_bait
+ yield alt + "_directed", g
+
+ for alt, fname in unfiltered:
+ if alts is not None and alt not in alts:
+ continue
+ b = _Builder()
+ cols = {k: array(t) for k, p, t in compass_cols}
+ protein_id = []
+ compass_id = {}
+ with open_upstream_file(name, fname, "r") as f:
+ rows = _rows(f)
+ hdr = next(rows)
+ if hdr[-1] == "":
+ hdr = hdr[:-1]
+ assert hdr[:7] == ["CompPASS_ID", "bait_symbol", "bait_geneid",
+ "db_protein_id", "symbol", "prot_description",
+ "gene_id"], hdr
+ ix = {k: i for i, k in enumerate(hdr)}
+ for r in rows:
+ assert len(r) in (len(hdr), len(hdr) + 1), r
+ assert r[2] not in ("", "NA"), r
+ s = b.vertex(r[2], r[1])
+ assert compass_id.setdefault(s, r[0]) == r[0], r
+ t = b.vertex(_gene(r[6], r[3]), r[4])
+ b.src.append(s)
+ b.tgt.append(t)
+ protein_id.append(r[3])
+ for k, p, typ in compass_cols:
+ x = r[ix[k]]
+ cols[k].append(float(x) if typ == 'd' else int(float(x)))
+ g = b.graph(compass_id.keys())
+ cid = g.new_vp("int64_t", val=-1)
+ for v, c in compass_id.items():
+ cid[v] = int(c)
+ g.vp.compass_id = cid
+ g.ep.protein_id = g.new_ep("string", vals=protein_id)
+ del protein_id
+ for k, p, typ in compass_cols:
+ g.ep[p] = g.new_ep("double" if typ == 'd' else "int64_t",
+ vals=numpy.frombuffer(cols[k],
+ dtype="float64" if typ == 'd' else "int64"))
+ yield alt, g
+
+ for alt, fname in releases:
+ if alts is not None and alt not in alts:
+ continue
+ b = _Builder()
+ plate, well, protein_id, pw, pni, pint = [], [], [], [], [], []
+ with open_upstream_file(name, fname, "r") as f:
+ rows = list(_rows(f))
+ hdr = rows[0]
+ ix = {k: i for i, k in enumerate(hdr)}
+ assert {"plate_num", "well_num", "symbol", "gene_id", "bait_symbol",
+ "bait_geneid", "pWrongID", "pNoInt", "pInt"} <= set(hdr), hdr
+ has_pid = "db_protein_id" in ix
+ baits = set()
+ for r in rows[1:]:
+ assert len(r) == len(hdr), r
+ assert r[ix["bait_geneid"]] not in ("", "NA"), r
+ s = b.vertex(r[ix["bait_geneid"]], r[ix["bait_symbol"]])
+ baits.add(s)
+ pid = r[ix["db_protein_id"]] if has_pid else ""
+ t = b.vertex(_gene(r[ix["gene_id"]], pid), r[ix["symbol"]])
+ b.src.append(s)
+ b.tgt.append(t)
+ plate.append(r[ix["plate_num"]])
+ well.append(r[ix["well_num"]])
+ protein_id.append(pid)
+ pw.append(float(r[ix["pWrongID"]]))
+ pni.append(float(r[ix["pNoInt"]]))
+ pint.append(float(r[ix["pInt"]]))
+ g = b.graph(baits)
+ g.ep.plate = g.new_ep("string", vals=plate)
+ g.ep.well = g.new_ep("string", vals=well)
+ if has_pid:
+ g.ep.protein_id = g.new_ep("string", vals=protein_id)
+ g.ep.p_wrong_id = g.new_ep("double", vals=pw)
+ g.ep.p_no_interaction = g.new_ep("double", vals=pni)
+ g.ep.p_interaction = g.new_ep("double", vals=pint)
+ yield alt, g