logo Netzschleuder network catalog, repository, and centrifuge

History of fly_banc

The changes to the source code of this entry in the repository, newest first, each with the differences it made to the entry's files. They are also listed on GitLab.

[entries] Add the date when each entry was added, and remove icon_hash by Tiago de Paula Peixoto beff7c9
The dates are those of the commits that first added the entries' files
(2020-08-11 for the entries of the initial commit); renamed entries keep
the date of their first name.

Changed: none
diff --git a/entries/fly_banc/__init__.py b/entries/fly_banc/__init__.py
index f5953bf..ceaa833 100644
--- a/entries/fly_banc/__init__.py
+++ b/entries/fly_banc/__init__.py
@@ -54,7 +54,7 @@ The networks have the following properties:
 tags = ['Biological', 'Connectome', 'Weighted', 'Multigraph', 'Metadata']
 url = 'https://doi.org/10.7910/DVN/7WTH1N'
 citation = [('A. S. Bates, J. S. Phelps, M. Kim, H. H. Yang et al., "Distributed control circuits across a brain-and-cord connectome." Nature 656, 957-970 (2026)', 'https://doi.org/10.1038/s41586-026-10735-w')]
-icon_hash = None
+date_added = '2026-10-27'
 upstream_license = ("CC BY 4.0", "https://creativecommons.org/licenses/by/4.0/")
 # Harvard Dataverse doi:10.7910/DVN/7WTH1N, files by id, for the dataset
 # versions 3.0 (current), 2.0/2.1 and 1.0 (only the files that differ)
[entries] Prefix the properties in the descriptions with (Node), (Edge) or (Graph) by Tiago de Paula Peixoto bc3dbbd
The lists of properties at the end of the descriptions marked edge
properties with "Edge" but left node properties unmarked, or grouped
them in "Nodes: `a`, ...; `b`, ..." items, so the kind of a property
was often unclear. Every item now starts with (Node), (Edge) or (Graph),
followed by the property names, a colon and the explanation, one item
per property or group of properties; qualifications (kinds of nodes,
networks) follow the names in parentheses. Properties that were only
named got a short explanation, and a few items that named the wrong
kind of property were corrected.
diff --git a/entries/fly_banc/__init__.py b/entries/fly_banc/__init__.py
index 4c62bef..f5953bf 100644
--- a/entries/fly_banc/__init__.py
+++ b/entries/fly_banc/__init__.py
@@ -36,21 +36,21 @@ The earlier versions of the Dataverse dataset, all of the same materialization v
 
 The networks have the following properties:
 
-- `banc_id`: the neuron's identifier (root ID) in materialization v888 of the BANC data set.
-- `side`: the side of the body of the neuron (`left`, `right`).
-- `region`: the part of the nervous system where the neuron is (`central_brain`, `optic_lobe`, `ventral_nerve_cord`); empty if not assigned.
-- `flow`: whether the neuron is `intrinsic`, `afferent` or `efferent`; empty if not assigned.
-- `super_class`, `cell_class`, `cell_sub_class`, `cell_type`: the four levels of the authors' cell type hierarchy (e.g. `central_brain_intrinsic`, `antennal_lobe_projection_neuron`, `multiglomerular_projection_neuron`, `M_lvPNm40`); empty if not assigned.
-- `fafb_cell_type`: the matched cell type in the FlyWire (FAFB) brain connectome (see the `flywire` entry); empty if not assigned.
-- `hemilineage`: the developmental hemilineage of the neuron; empty if not assigned.
-- `nerve`: the nerve through which the neuron enters or exits the nervous system; empty if not assigned.
-- `neuromere`: the developmental neuromere (segment) of the neuron, for neurons of the ventral nerve cord; empty if not assigned.
-- `neurotransmitter`: the neurotransmitter predicted for the neuron from its synapses; empty if not assigned.
-- `neurotransmitter_score`: the confidence of the prediction (NaN if not assigned).
-- `sexually_dimorphic`: whether the neuron's type is `isomorphic`, `dimorphic` or `female-specific`, from comparison with male connectomes; empty if not assigned.
-- `weight` (edges): the number of synapses from $i$ to $j$ (in the given compartments, in the `*_split` networks).
-- `pre_compartment`, `post_compartment` (edges, only in the `*_split` networks): the compartments of neurons $i$ and $j$ where the synapses are located (`axon`, `dendrite`, `primary.dendrite`, `primary.neurite`, `unknown`).
-- `neurotransmitter`, `neurotransmitter_score` (edges, only in the `*_split` networks): the neurotransmitter predicted for the presynaptic neuron $i$ in the compartment table, and the confidence of the prediction."""
+- (Node) `banc_id`: the neuron's identifier (root ID) in materialization v888 of the BANC data set.
+- (Node) `side`: the side of the body of the neuron (`left`, `right`).
+- (Node) `region`: the part of the nervous system where the neuron is (`central_brain`, `optic_lobe`, `ventral_nerve_cord`); empty if not assigned.
+- (Node) `flow`: whether the neuron is `intrinsic`, `afferent` or `efferent`; empty if not assigned.
+- (Node) `super_class`, `cell_class`, `cell_sub_class`, `cell_type`: the four levels of the authors' cell type hierarchy (e.g. `central_brain_intrinsic`, `antennal_lobe_projection_neuron`, `multiglomerular_projection_neuron`, `M_lvPNm40`); empty if not assigned.
+- (Node) `fafb_cell_type`: the matched cell type in the FlyWire (FAFB) brain connectome (see the `flywire` entry); empty if not assigned.
+- (Node) `hemilineage`: the developmental hemilineage of the neuron; empty if not assigned.
+- (Node) `nerve`: the nerve through which the neuron enters or exits the nervous system; empty if not assigned.
+- (Node) `neuromere`: the developmental neuromere (segment) of the neuron, for neurons of the ventral nerve cord; empty if not assigned.
+- (Node) `neurotransmitter`: the neurotransmitter predicted for the neuron from its synapses; empty if not assigned.
+- (Node) `neurotransmitter_score`: the confidence of the prediction (NaN if not assigned).
+- (Node) `sexually_dimorphic`: whether the neuron's type is `isomorphic`, `dimorphic` or `female-specific`, from comparison with male connectomes; empty if not assigned.
+- (Edge) `weight`: the number of synapses from $i$ to $j$ (in the given compartments, in the `*_split` networks).
+- (Edge) `pre_compartment`, `post_compartment` (only in the `*_split` networks): the compartments of neurons $i$ and $j$ where the synapses are located (`axon`, `dendrite`, `primary.dendrite`, `primary.neurite`, `unknown`).
+- (Edge) `neurotransmitter`, `neurotransmitter_score` (only in the `*_split` networks): the neurotransmitter predicted for the presynaptic neuron $i$ in the compartment table, and the confidence of the prediction."""
 tags = ['Biological', 'Connectome', 'Weighted', 'Multigraph', 'Metadata']
 url = 'https://doi.org/10.7910/DVN/7WTH1N'
 citation = [('A. S. Bates, J. S. Phelps, M. Kim, H. H. Yang et al., "Distributed control circuits across a brain-and-cord connectome." Nature 656, 957-970 (2026)', 'https://doi.org/10.1038/s41586-026-10735-w')]
[entries] New entry: fly_banc by Tiago de Paula Peixoto 856f899
diff --git a/entries/fly_banc/__init__.py b/entries/fly_banc/__init__.py
new file mode 100644
index 0000000..4c62bef
--- /dev/null
+++ b/entries/fly_banc/__init__.py
@@ -0,0 +1,139 @@
+#! /usr/bin/env python
+# -*- coding: utf-8 -*-
+#
+# Copyright (C) 2020-2026 Tiago de Paula Peixoto <tiago@skewed.de>
+#
+# This program is free software: you can redistribute it and/or modify it under
+# the terms of the GNU Affero General Public License as published by the Free
+# Software Foundation, either version 3 of the License, or (at your option) any
+# later version.
+#
+# This program is distributed in the hope that it will be useful, but WITHOUT
+# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
+# FOR A PARTICULAR PURPOSE.  See the GNU Affero General Public License for more
+# details.
+#
+# You should have received a copy of the GNU Affero General Public License along
+# with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+from .. import *
+import numpy as np
+import pandas as pd
+import pyarrow.feather
+
+title = "BANC fly brain and nerve cord (2026)"
+description = r"""The connectome of the entire central nervous system of an adult female fruit fly *Drosophila melanogaster*, the brain together with the ventral nerve cord (VNC, the fly's analog of the spinal cord), from the "Brain And Nerve Cord" (BANC) data set. It was reconstructed by Bates, Phelps, Kim, Yang and collaborators from serial-section transmission electron microscopy images, with automated segmentation and synapse detection followed by extensive manual proofreading. Nodes are neurons, and a directed edge $(i, j)$ indicates chemical synapses from neuron $i$ (presynaptic) to neuron $j$ (postsynaptic). This is materialization v888 of the data set, as released by the authors with the published paper.
+
+The connections were detected with two synapse detection models, both included here: the "v3" model (synapses of at least 10 voxels), which the authors recommend for new work, and the earlier "v2" model (synapses of at least 5 voxels), which the analyses of the paper use and which finds about 14% fewer connected pairs. Autapses (synapses of a neuron onto itself) were excluded by the authors from all tables except the v2 neuron-to-neuron table of the current release, which contains them (as self-loops, 140,032 in `v2_all`) although its documentation says otherwise; they are kept as given. Following the authors (and the convention of FlyWire Codex), pairs of neurons are considered connected if they have at least five synapses. The networks of the current release (Harvard Dataverse version 3.0, July 2026) are:
+
+- `v3_min5`, `v2_min5`: only the pairs of neurons with at least five synapses: these are 1,861,418 of the 12,923,260 connected pairs (23.1 million of 40.7 million synapses) with the v3 model, and 1,562,742 of the 11,176,571 connected pairs (19.3 million of 34.4 million synapses) with the v2 model.
+- `v3_all`, `v2_all`: all connections, without threshold on the number of synapses.
+- `v3_split`, `v2_split`: all connections, split according to the compartments of the two neurons where the synapses are located (the axon, the dendrite, the primary dendrite, the primary neurite, or unknown, from the authors' split of each neuron by flow centrality), so that the same pair of neurons may be connected by several parallel edges, one per pair of compartments. For neurons truncated at the boundary of the volume, the authors set the compartment on the side facing the nervous system (axon for afferent neurons, dendrite for efferent ones).
+
+All these networks have the same nodes: the 150,802 segments marked as proofread, leaving out those classified as glia, trachea or not a neuron (neurons without connections are isolated nodes).
+
+The earlier versions of the Dataverse dataset, all of the same materialization v888, are included as the networks with the prefixes `dv2_` (versions 2.0 and 2.1, June 2026, which share their files) and `dv1_` (version 1.0, May 2026), with the same six networks each. Their neuron annotations differ from those of the current release (e.g. 74 neurons were not yet marked as proofread, so that they have 150,728 nodes, and versions 2.0 and 2.1 lack the side of most neurons); their connection tables are the same as those of the current release, except for version 1.0, whose v2 tables were later replaced: its `dv1_v2_all` has no autapses, and its compartment table `dv1_v2_split` contains only 15.9 million synapses, about half of those of the other v2 tables.
+
+The networks have the following properties:
+
+- `banc_id`: the neuron's identifier (root ID) in materialization v888 of the BANC data set.
+- `side`: the side of the body of the neuron (`left`, `right`).
+- `region`: the part of the nervous system where the neuron is (`central_brain`, `optic_lobe`, `ventral_nerve_cord`); empty if not assigned.
+- `flow`: whether the neuron is `intrinsic`, `afferent` or `efferent`; empty if not assigned.
+- `super_class`, `cell_class`, `cell_sub_class`, `cell_type`: the four levels of the authors' cell type hierarchy (e.g. `central_brain_intrinsic`, `antennal_lobe_projection_neuron`, `multiglomerular_projection_neuron`, `M_lvPNm40`); empty if not assigned.
+- `fafb_cell_type`: the matched cell type in the FlyWire (FAFB) brain connectome (see the `flywire` entry); empty if not assigned.
+- `hemilineage`: the developmental hemilineage of the neuron; empty if not assigned.
+- `nerve`: the nerve through which the neuron enters or exits the nervous system; empty if not assigned.
+- `neuromere`: the developmental neuromere (segment) of the neuron, for neurons of the ventral nerve cord; empty if not assigned.
+- `neurotransmitter`: the neurotransmitter predicted for the neuron from its synapses; empty if not assigned.
+- `neurotransmitter_score`: the confidence of the prediction (NaN if not assigned).
+- `sexually_dimorphic`: whether the neuron's type is `isomorphic`, `dimorphic` or `female-specific`, from comparison with male connectomes; empty if not assigned.
+- `weight` (edges): the number of synapses from $i$ to $j$ (in the given compartments, in the `*_split` networks).
+- `pre_compartment`, `post_compartment` (edges, only in the `*_split` networks): the compartments of neurons $i$ and $j$ where the synapses are located (`axon`, `dendrite`, `primary.dendrite`, `primary.neurite`, `unknown`).
+- `neurotransmitter`, `neurotransmitter_score` (edges, only in the `*_split` networks): the neurotransmitter predicted for the presynaptic neuron $i$ in the compartment table, and the confidence of the prediction."""
+tags = ['Biological', 'Connectome', 'Weighted', 'Multigraph', 'Metadata']
+url = 'https://doi.org/10.7910/DVN/7WTH1N'
+citation = [('A. S. Bates, J. S. Phelps, M. Kim, H. H. Yang et al., "Distributed control circuits across a brain-and-cord connectome." Nature 656, 957-970 (2026)', 'https://doi.org/10.1038/s41586-026-10735-w')]
+icon_hash = None
+upstream_license = ("CC BY 4.0", "https://creativecommons.org/licenses/by/4.0/")
+# Harvard Dataverse doi:10.7910/DVN/7WTH1N, files by id, for the dataset
+# versions 3.0 (current), 2.0/2.1 and 1.0 (only the files that differ)
+upstream_prefix = 'https://dataverse.harvard.edu/api/access/datafile'
+_versions = {
+    # prefix: (banc_888_meta.feather, {table: banc_888_edgelist_<table>.feather})
+    "": ('14033740', {"v3": '13918810', "v2": '13992792',
+                      "v3_split": '13992837', "v2_split": '13992835'}),
+    "dv2_": ('13992790', {"v3": '13918810', "v2": '13992792',
+                          "v3_split": '13992837', "v2_split": '13992835'}),
+    "dv1_": ('13923343', {"v3": '13918810', "v2": '13916462',
+                          "v3_split": '13918812', "v2_split": '13916463'}),
+}
+files = []
+for _pfx, (_meta, _tables) in _versions.items():
+    for _model in ["v3", "v2"]:
+        for _thres in ["min5", "all"]:
+            files.append(((_tables[_model], _meta), f"{_pfx}{_model}_{_thres}", None))
+    for _model in ["v3", "v2"]:
+        files.append(((_tables[f"{_model}_split"], _meta), f"{_pfx}{_model}_split", None))
+
+# minimum number of synapses between two neurons for them to be connected
+min_synapses = {"min5": 5, "all": 0}
+
+def fetch_upstream(force=False):
+    return fetch_upstream_files(__name__.split(".")[-1], upstream_prefix, files,
+                                force)
+
+vprops = ["side", "region", "flow", "super_class", "cell_class",
+          "cell_sub_class", "cell_type", "fafb_cell_type", "hemilineage",
+          "nerve", "neuromere", ("neurotransmitter_predicted", "neurotransmitter"),
+          "sexually_dimorphic"]
+
+@cache_network()
+@coerce_props()
+@annotate()
+def parse(alts=None):
+    name = __name__.split(".")[-1]
+    for (fedges, fmeta), alt, fmt in files:
+        if alts is not None and alt not in alts:
+            continue
+        with open_upstream_file(name, fmeta, "rb", seekable=True) as f:
+            meta = pd.read_feather(f)
+        meta = meta[(meta["proofread"] == "TRUE") &
+                    ~meta["super_class"].isin(["glia", "trachea", "not_a_neuron"])]
+        meta = meta.reset_index(drop=True)
+
+        g = Graph()
+        g.add_vertex(len(meta))
+        g.vp.banc_id = g.new_vp("int64_t", vals=meta["banc_888_id"].astype("int64").to_numpy())
+        for p in vprops:
+            col, pname = p if isinstance(p, tuple) else (p, p)
+            vals = meta[col].fillna("").astype(str)
+            g.vp[pname] = g.new_vp("string", vals=vals.tolist())
+        g.vp.neurotransmitter_score = \
+            g.new_vp("double", vals=meta["neurotransmitter_score"].astype(float).to_numpy())
+
+        split = alt.endswith("_split")
+        cols = ["pre", "post", "count"]
+        if split:
+            cols += ["pre_label", "post_label", "pre_conf_nt", "pre_conf_nt_p"]
+        with open_upstream_file(name, fedges, "rb", seekable=True) as f:
+            edges = pyarrow.feather.read_table(f, columns=cols).to_pandas(strings_to_categorical=split)
+        idx = pd.Series(np.arange(len(meta)), index=meta["banc_888_id"].to_numpy())
+        minw = 0 if split else min_synapses[alt.split("_")[-1]]
+        edges = edges[(edges["count"] >= minw) &
+                      edges["pre"].isin(idx.index) & edges["post"].isin(idx.index)]
+        if split:
+            edges = edges.sort_values(["pre", "post", "pre_label", "post_label"])
+        s = idx.loc[edges["pre"].to_numpy()].to_numpy()
+        t = idx.loc[edges["post"].to_numpy()].to_numpy()
+        g.add_edge_list(np.column_stack((s, t)))
+        g.ep.weight = g.new_ep("int", vals=edges["count"].to_numpy())
+        if split:
+            for col, p in [("pre_label", "pre_compartment"),
+                           ("post_label", "post_compartment"),
+                           ("pre_conf_nt", "neurotransmitter")]:
+                g.ep[p] = g.new_ep("string", vals=edges[col].astype(object).fillna("").tolist())
+            g.ep.neurotransmitter_score = \
+                g.new_ep("double", vals=edges["pre_conf_nt_p"].astype(float).to_numpy())
+        del edges, s, t
+        yield alt, g