History of splotopen
The changes to the source code of this entry in the repository, newest first, each with the differences it made to the entry's files. They are also listed on GitLab.
[entries] Add the date when each entry was added, and remove icon_hash beff7c9
The dates are those of the commits that first added the entries' files (2020-08-11 for the entries of the initial commit); renamed entries keep the date of their first name. Changed: none
diff --git a/entries/splotopen/__init__.py b/entries/splotopen/__init__.py
index d283f6a..fd347d7 100644
--- a/entries/splotopen/__init__.py
+++ b/entries/splotopen/__init__.py
@@ -67,7 +67,7 @@ tags = ['Biological', 'Ecological', 'Weighted', 'Bipartite', 'Multigraph', 'Meta
url = ['https://idata.idiv.de/ddm/Data/ShowData/3474?version=76', 'https://doi.org/10.25829/idiv.3474-40-3292']
citation = [('F. M. Sabatini, J. Lenoir, T. Hattab, E. A. Arnst, M. Chytrý, J. Dengler, P. De Ruffray, S. M. Hennekens, U. Jandt, F. Jansen, et al., "sPlotOpen – An environmentally balanced, open-access, global dataset of vegetation plots." Global Ecology and Biogeography 30, 1740-1764 (2021)',
'https://doi.org/10.1111/geb.13346')]
-icon_hash = None
+date_added = '2026-10-31'
upstream_license = ('Creative Commons licenses of the source datasets (CC BY, CC BY-SA or CC BY-NC)',
'https://doi.org/10.1111/geb.13346')
# Version 76 (version id 5806) of dataset 3474 of the iDiv data repository;
[entries] Prefix the properties in the descriptions with (Node), (Edge) or (Graph) bc3dbbd
The lists of properties at the end of the descriptions marked edge properties with "Edge" but left node properties unmarked, or grouped them in "Nodes: `a`, ...; `b`, ..." items, so the kind of a property was often unclear. Every item now starts with (Node), (Edge) or (Graph), followed by the property names, a colon and the explanation, one item per property or group of properties; qualifications (kinds of nodes, networks) follow the names in parentheses. Properties that were only named got a short explanation, and a few items that named the wrong kind of property were corrected.
diff --git a/entries/splotopen/__init__.py b/entries/splotopen/__init__.py
index 8090bf9..d283f6a 100644
--- a/entries/splotopen/__init__.py
+++ b/entries/splotopen/__init__.py
@@ -32,37 +32,37 @@ Nodes are plots and species (with names resolved to a common taxonomic backbone)
The network has the following properties:
-- `is_plot`: whether the node is a plot (otherwise it is a species).
-- `name`: the identifier of the plot (`PlotObservationID`), or the name of the species.
-- `givd_id`, `dataset`: the GIVD code and the name of the source dataset of the plot.
-- `continent`, `country`, `biome`: the continent, country and biome (as defined by sPlot) of the plot.
-- `date`: the date of recording of the plot (as Unix time; `-1` if unknown).
-- `latitude`, `longitude`, `location_uncertainty`: the coordinates of the plot (WGS84) and their uncertainty in meters.
-- `releve_area`: the area of the plot, in square meters.
-- `plant_recorded`: the group of plants recorded in the plot (e.g. `All vascular plants`).
-- `elevation`, `aspect`, `slope`: the elevation (m), aspect and slope (degrees) of the plot.
-- `is_forest`: whether the plot is a forest, as estimated from the growth forms of the species (empty if unknown).
-- `esy`: the habitat of the plot in the EUNIS classification (European plots).
-- `naturalness`: `1` for natural and `2` for semi-natural vegetation (NaN if unknown).
-- `forest`, `shrubland`, `grassland`, `wetland`, `sparse_vegetation`: the vegetation type of the plot as given by the data contributors (`TRUE`, `FALSE`, or empty if unknown).
-- `cover_total`, `cover_tree_layer`, `cover_shrub_layer`, `cover_herb_layer`, `cover_moss_layer`, `cover_lichen_layer`, `cover_algae_layer`, `cover_litter_layer`, `cover_bare_rocks`, `cover_cryptogams`, `cover_bare_soil`: the cover (in %) of the vegetation and of its layers, and of litter, rocks and bare soil (NaN if not given).
-- `height_trees_highest`, `height_trees_lowest`, `height_shrubs_highest`, `height_shrubs_lowest` (m), `height_herbs_average`, `height_herbs_lowest`, `height_herbs_highest` (cm): the heights of the vegetation layers (NaN if not given).
-- `soilclim_pc1`, `soilclim_pc2`: the coordinates of the plot on the first two principal components of its climate and soil conditions.
-- `resample_1`, `resample_2`, `resample_3`, `resample_1_consensus`: whether the plot belongs to each of the three resampled sets (and to the first one after replacements).
-- `species_richness`: the number of species in the plot (-1 for species).
-- `traitcoverage_cover`, `traitcoverage_pa`: the proportion of the relative cover of the species of the plot for which the traits are available, and the proportion of the species with trait information.
-- `<trait>_cwm`, `<trait>_cwv`: the community-weighted mean and variance in the plot of 18 plant functional traits (from the gap-filled data of the TRY database, as computed by sPlot; the values are apparently on a logarithmic scale, since they can be negative; NaN if not given, e.g. the variances of plots with a single species with traits): `leafarea` (leaf area, mm²), `stemdens` (stem specific density, g/cm³), `sla` (specific leaf area, m²/kg), `leafc_perdrymass`, `leafn`, `leafp` (leaf carbon, nitrogen and phosphorus content per dry mass, mg/g), `plantheight` (plant height, m), `seedmass` (seed dry mass, mg), `seed_length` (mm), `ldmc` (leaf dry matter content, g/g), `leafnperarea` (leaf nitrogen per area, g/m²), `leafnpratio` (leaf N/P ratio), `leaf_delta_15n` (leaf δ15N, per mil), `seed_num_rep_unit` (seed number per reproduction unit), `leaffreshmass` (leaf fresh mass, g), `stem_cond_dens` (stem conduit density, mm⁻²), `disp_unit_leng` (dispersal unit length, mm), `wood_vessel_length` (wood vessel element length, μm).
-- `db_bibtexkey`: the BibTeX key of the reference of the source dataset of the plot (in the graph property `references`).
-- `releve_author`, `releve_coauthors`: the names of the authors or surveyors of the plot, as given in the data.
-- `biblioreference`, `bibtexkey`: the bibliographic reference of the publication in which the plot was described, and its BibTeX key (in `references`).
-- `nr_table_in_publ`, `nr_releve_in_table`: the table of the publication containing the plot, and the identifier of the plot in it.
-- `original_nr_in_database`, `original_plotid`, `original_subplotid`: the identifiers of the plot (and subplot, for nested plots) in the source database.
-- `project`, `remarks`: the project in which the plot was recorded, and remarks.
-- `guid`: the unique identifier given by Turboveg when the plot was first stored.
-- Edge `relative_cover`: the abundance of the species in the plot, normalized to sum to one in each plot.
-- Edge `original_abundance`, `abundance_scale`: the abundance of the species as in the original data, and its scale (`CoverPerc`: cover percentage, `pa`: presence or absence, `x_BA`: basal area, `x_IC`: individual count, `x_SC`: stem count, `x_IV`: relative importance, `x_RF`: relative frequency; the values `x_PF` and `x`, also found in the data, are not documented).
-- Edge `original_species`: the name of the species in the original data, before the taxonomic resolution (one of them is garbled in the data, with invalid characters, which are replaced by `�`).
-- Graph `references`: the references of the source datasets and of the publications of the plots, in BibTeX format (`sPlotOpen_references.bib`)."""
+- (Node) `is_plot`: whether the node is a plot (otherwise it is a species).
+- (Node) `name`: the identifier of the plot (`PlotObservationID`), or the name of the species.
+- (Node) `givd_id`, `dataset`: the GIVD code and the name of the source dataset of the plot.
+- (Node) `continent`, `country`, `biome`: the continent, country and biome (as defined by sPlot) of the plot.
+- (Node) `date`: the date of recording of the plot (as Unix time; `-1` if unknown).
+- (Node) `latitude`, `longitude`, `location_uncertainty`: the coordinates of the plot (WGS84) and their uncertainty in meters.
+- (Node) `releve_area`: the area of the plot, in square meters.
+- (Node) `plant_recorded`: the group of plants recorded in the plot (e.g. `All vascular plants`).
+- (Node) `elevation`, `aspect`, `slope`: the elevation (m), aspect and slope (degrees) of the plot.
+- (Node) `is_forest`: whether the plot is a forest, as estimated from the growth forms of the species (empty if unknown).
+- (Node) `esy`: the habitat of the plot in the EUNIS classification (European plots).
+- (Node) `naturalness`: `1` for natural and `2` for semi-natural vegetation (NaN if unknown).
+- (Node) `forest`, `shrubland`, `grassland`, `wetland`, `sparse_vegetation`: the vegetation type of the plot as given by the data contributors (`TRUE`, `FALSE`, or empty if unknown).
+- (Node) `cover_total`, `cover_tree_layer`, `cover_shrub_layer`, `cover_herb_layer`, `cover_moss_layer`, `cover_lichen_layer`, `cover_algae_layer`, `cover_litter_layer`, `cover_bare_rocks`, `cover_cryptogams`, `cover_bare_soil`: the cover (in %) of the vegetation and of its layers, and of litter, rocks and bare soil (NaN if not given).
+- (Node) `height_trees_highest`, `height_trees_lowest`, `height_shrubs_highest`, `height_shrubs_lowest` (m), `height_herbs_average`, `height_herbs_lowest`, `height_herbs_highest` (cm): the heights of the vegetation layers (NaN if not given).
+- (Node) `soilclim_pc1`, `soilclim_pc2`: the coordinates of the plot on the first two principal components of its climate and soil conditions.
+- (Node) `resample_1`, `resample_2`, `resample_3`, `resample_1_consensus`: whether the plot belongs to each of the three resampled sets (and to the first one after replacements).
+- (Node) `species_richness`: the number of species in the plot (-1 for species).
+- (Node) `traitcoverage_cover`, `traitcoverage_pa`: the proportion of the relative cover of the species of the plot for which the traits are available, and the proportion of the species with trait information.
+- (Node) `<trait>_cwm`, `<trait>_cwv`: the community-weighted mean and variance in the plot of 18 plant functional traits (from the gap-filled data of the TRY database, as computed by sPlot; the values are apparently on a logarithmic scale, since they can be negative; NaN if not given, e.g. the variances of plots with a single species with traits): `leafarea` (leaf area, mm²), `stemdens` (stem specific density, g/cm³), `sla` (specific leaf area, m²/kg), `leafc_perdrymass`, `leafn`, `leafp` (leaf carbon, nitrogen and phosphorus content per dry mass, mg/g), `plantheight` (plant height, m), `seedmass` (seed dry mass, mg), `seed_length` (mm), `ldmc` (leaf dry matter content, g/g), `leafnperarea` (leaf nitrogen per area, g/m²), `leafnpratio` (leaf N/P ratio), `leaf_delta_15n` (leaf δ15N, per mil), `seed_num_rep_unit` (seed number per reproduction unit), `leaffreshmass` (leaf fresh mass, g), `stem_cond_dens` (stem conduit density, mm⁻²), `disp_unit_leng` (dispersal unit length, mm), `wood_vessel_length` (wood vessel element length, μm).
+- (Node) `db_bibtexkey`: the BibTeX key of the reference of the source dataset of the plot (in the graph property `references`).
+- (Node) `releve_author`, `releve_coauthors`: the names of the authors or surveyors of the plot, as given in the data.
+- (Node) `biblioreference`, `bibtexkey`: the bibliographic reference of the publication in which the plot was described, and its BibTeX key (in `references`).
+- (Node) `nr_table_in_publ`, `nr_releve_in_table`: the table of the publication containing the plot, and the identifier of the plot in it.
+- (Node) `original_nr_in_database`, `original_plotid`, `original_subplotid`: the identifiers of the plot (and subplot, for nested plots) in the source database.
+- (Node) `project`, `remarks`: the project in which the plot was recorded, and remarks.
+- (Node) `guid`: the unique identifier given by Turboveg when the plot was first stored.
+- (Edge) `relative_cover`: the abundance of the species in the plot, normalized to sum to one in each plot.
+- (Edge) `original_abundance`, `abundance_scale`: the abundance of the species as in the original data, and its scale (`CoverPerc`: cover percentage, `pa`: presence or absence, `x_BA`: basal area, `x_IC`: individual count, `x_SC`: stem count, `x_IV`: relative importance, `x_RF`: relative frequency; the values `x_PF` and `x`, also found in the data, are not documented).
+- (Edge) `original_species`: the name of the species in the original data, before the taxonomic resolution (one of them is garbled in the data, with invalid characters, which are replaced by `�`).
+- (Graph) `references`: the references of the source datasets and of the publications of the plots, in BibTeX format (`sPlotOpen_references.bib`)."""
tags = ['Biological', 'Ecological', 'Weighted', 'Bipartite', 'Multigraph', 'Metadata']
url = ['https://idata.idiv.de/ddm/Data/ShowData/3474?version=76', 'https://doi.org/10.25829/idiv.3474-40-3292']
citation = [('F. M. Sabatini, J. Lenoir, T. Hattab, E. A. Arnst, M. Chytrý, J. Dengler, P. De Ruffray, S. M. Hennekens, U. Jandt, F. Jansen, et al., "sPlotOpen – An environmentally balanced, open-access, global dataset of vegetation plots." Global Ecology and Biogeography 30, 1740-1764 (2021)',
[entries] New entry: splotopen 350bc1d
diff --git a/entries/splotopen/__init__.py b/entries/splotopen/__init__.py
new file mode 100644
index 0000000..8090bf9
--- /dev/null
+++ b/entries/splotopen/__init__.py
@@ -0,0 +1,205 @@
+#! /usr/bin/env python
+# -*- coding: utf-8 -*-
+#
+# Copyright (C) 2026 Tiago de Paula Peixoto <tiago@skewed.de>
+#
+# This program is free software: you can redistribute it and/or modify it under
+# the terms of the GNU Affero General Public License as published by the Free
+# Software Foundation, either version 3 of the License, or (at your option) any
+# later version.
+#
+# This program is distributed in the hope that it will be useful, but WITHOUT
+# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
+# FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
+# details.
+#
+# You should have received a copy of the GNU Affero General Public License along
+# with this program. If not, see <http://www.gnu.org/licenses/>.
+
+import csv
+import datetime as dt
+import io
+import math
+import sys
+import zipfile
+
+from .. import *
+
+title = "sPlotOpen global vegetation plots (1888-2015)"
+description = r"""Bipartite network of vegetation plots and the vascular plant species recorded in them, from sPlotOpen, an open-access, environmentally balanced subset of sPlot, the global database of vegetation plots. The plots come from 105 source datasets (vegetation-plot databases of the Global Index of Vegetation-Plot Databases, GIVD) and cover all continents and biomes; they were recorded between 1888 and 2015 by many surveyors, who listed the plant species present in a plot of a given area and estimated their abundance (usually their cover). The plots were selected so as to balance the environmental conditions (climate and soil) of the world, in three partially overlapping resampled sets of about 50,000 plots each (marked by vertex properties), which are meant to be used as replicates in global analyses.
+
+Nodes are plots and species (with names resolved to a common taxonomic backbone), and an edge connects a plot to each species recorded in it, with the abundance of the species in the plot; a species can be listed several times in a plot (e.g. in several vegetation layers, or under several original names resolved to the same species), which gives parallel edges. The plots also have the community-weighted means and variances of plant functional traits computed by sPlot (`sPlotOpen_CWM_CWV.txt`), and their bibliographic metadata (`sPlotOpen_metadata.txt`: the references of the source datasets and of the publications of the plots, the names of the surveyors, the original identifiers), with the references in BibTeX format (`sPlotOpen_references.bib`) as a graph property. The data are those of version 76 (2023-03-07) of the dataset in the iDiv data repository.
+
+The network has the following properties:
+
+- `is_plot`: whether the node is a plot (otherwise it is a species).
+- `name`: the identifier of the plot (`PlotObservationID`), or the name of the species.
+- `givd_id`, `dataset`: the GIVD code and the name of the source dataset of the plot.
+- `continent`, `country`, `biome`: the continent, country and biome (as defined by sPlot) of the plot.
+- `date`: the date of recording of the plot (as Unix time; `-1` if unknown).
+- `latitude`, `longitude`, `location_uncertainty`: the coordinates of the plot (WGS84) and their uncertainty in meters.
+- `releve_area`: the area of the plot, in square meters.
+- `plant_recorded`: the group of plants recorded in the plot (e.g. `All vascular plants`).
+- `elevation`, `aspect`, `slope`: the elevation (m), aspect and slope (degrees) of the plot.
+- `is_forest`: whether the plot is a forest, as estimated from the growth forms of the species (empty if unknown).
+- `esy`: the habitat of the plot in the EUNIS classification (European plots).
+- `naturalness`: `1` for natural and `2` for semi-natural vegetation (NaN if unknown).
+- `forest`, `shrubland`, `grassland`, `wetland`, `sparse_vegetation`: the vegetation type of the plot as given by the data contributors (`TRUE`, `FALSE`, or empty if unknown).
+- `cover_total`, `cover_tree_layer`, `cover_shrub_layer`, `cover_herb_layer`, `cover_moss_layer`, `cover_lichen_layer`, `cover_algae_layer`, `cover_litter_layer`, `cover_bare_rocks`, `cover_cryptogams`, `cover_bare_soil`: the cover (in %) of the vegetation and of its layers, and of litter, rocks and bare soil (NaN if not given).
+- `height_trees_highest`, `height_trees_lowest`, `height_shrubs_highest`, `height_shrubs_lowest` (m), `height_herbs_average`, `height_herbs_lowest`, `height_herbs_highest` (cm): the heights of the vegetation layers (NaN if not given).
+- `soilclim_pc1`, `soilclim_pc2`: the coordinates of the plot on the first two principal components of its climate and soil conditions.
+- `resample_1`, `resample_2`, `resample_3`, `resample_1_consensus`: whether the plot belongs to each of the three resampled sets (and to the first one after replacements).
+- `species_richness`: the number of species in the plot (-1 for species).
+- `traitcoverage_cover`, `traitcoverage_pa`: the proportion of the relative cover of the species of the plot for which the traits are available, and the proportion of the species with trait information.
+- `<trait>_cwm`, `<trait>_cwv`: the community-weighted mean and variance in the plot of 18 plant functional traits (from the gap-filled data of the TRY database, as computed by sPlot; the values are apparently on a logarithmic scale, since they can be negative; NaN if not given, e.g. the variances of plots with a single species with traits): `leafarea` (leaf area, mm²), `stemdens` (stem specific density, g/cm³), `sla` (specific leaf area, m²/kg), `leafc_perdrymass`, `leafn`, `leafp` (leaf carbon, nitrogen and phosphorus content per dry mass, mg/g), `plantheight` (plant height, m), `seedmass` (seed dry mass, mg), `seed_length` (mm), `ldmc` (leaf dry matter content, g/g), `leafnperarea` (leaf nitrogen per area, g/m²), `leafnpratio` (leaf N/P ratio), `leaf_delta_15n` (leaf δ15N, per mil), `seed_num_rep_unit` (seed number per reproduction unit), `leaffreshmass` (leaf fresh mass, g), `stem_cond_dens` (stem conduit density, mm⁻²), `disp_unit_leng` (dispersal unit length, mm), `wood_vessel_length` (wood vessel element length, μm).
+- `db_bibtexkey`: the BibTeX key of the reference of the source dataset of the plot (in the graph property `references`).
+- `releve_author`, `releve_coauthors`: the names of the authors or surveyors of the plot, as given in the data.
+- `biblioreference`, `bibtexkey`: the bibliographic reference of the publication in which the plot was described, and its BibTeX key (in `references`).
+- `nr_table_in_publ`, `nr_releve_in_table`: the table of the publication containing the plot, and the identifier of the plot in it.
+- `original_nr_in_database`, `original_plotid`, `original_subplotid`: the identifiers of the plot (and subplot, for nested plots) in the source database.
+- `project`, `remarks`: the project in which the plot was recorded, and remarks.
+- `guid`: the unique identifier given by Turboveg when the plot was first stored.
+- Edge `relative_cover`: the abundance of the species in the plot, normalized to sum to one in each plot.
+- Edge `original_abundance`, `abundance_scale`: the abundance of the species as in the original data, and its scale (`CoverPerc`: cover percentage, `pa`: presence or absence, `x_BA`: basal area, `x_IC`: individual count, `x_SC`: stem count, `x_IV`: relative importance, `x_RF`: relative frequency; the values `x_PF` and `x`, also found in the data, are not documented).
+- Edge `original_species`: the name of the species in the original data, before the taxonomic resolution (one of them is garbled in the data, with invalid characters, which are replaced by `�`).
+- Graph `references`: the references of the source datasets and of the publications of the plots, in BibTeX format (`sPlotOpen_references.bib`)."""
+tags = ['Biological', 'Ecological', 'Weighted', 'Bipartite', 'Multigraph', 'Metadata']
+url = ['https://idata.idiv.de/ddm/Data/ShowData/3474?version=76', 'https://doi.org/10.25829/idiv.3474-40-3292']
+citation = [('F. M. Sabatini, J. Lenoir, T. Hattab, E. A. Arnst, M. Chytrý, J. Dengler, P. De Ruffray, S. M. Hennekens, U. Jandt, F. Jansen, et al., "sPlotOpen – An environmentally balanced, open-access, global dataset of vegetation plots." Global Ecology and Biogeography 30, 1740-1764 (2021)',
+ 'https://doi.org/10.1111/geb.13346')]
+icon_hash = None
+upstream_license = ('Creative Commons licenses of the source datasets (CC BY, CC BY-SA or CC BY-NC)',
+ 'https://doi.org/10.1111/geb.13346')
+# Version 76 (version id 5806) of dataset 3474 of the iDiv data repository;
+# the zip file is generated at each download (its member names end with
+# numbers in parentheses, which may vary), so the members are found by prefix
+upstream_prefix = 'https://idata.idiv.de/ddm/Data/DownloadZip'
+ZIP = '3474?version=5806'
+files = [(ZIP, None, None)]
+
+def fetch_upstream(force=False):
+ return fetch_upstream_files(__name__.split(".")[-1], upstream_prefix, files,
+ force)
+
+def member(z, prefix):
+ ms = [n for n in z.namelist() if n.startswith(prefix)]
+ assert len(ms) == 1, (prefix, ms)
+ return ms[0]
+
+def to_float(x):
+ try:
+ return float(x)
+ except ValueError:
+ return math.nan
+
+def to_time(x):
+ try:
+ return int(dt.datetime.strptime(x, "%Y-%m-%d").replace(
+ tzinfo=dt.timezone.utc).timestamp())
+ except ValueError:
+ return -1
+
+# for checks: read only this many rows of species in plots (None: all)
+_check_rows = None
+
+HEADER_STRINGS = ["GIVD_ID", "Dataset", "Continent", "Country", "Biome",
+ "Plant_recorded", "is_forest", "ESY", "Forest", "Shrubland",
+ "Grassland", "Wetland", "Sparse_vegetation"]
+HEADER_BOOLS = ["Resample_1", "Resample_2", "Resample_3", "Resample_1_consensus"]
+
+@cache_network()
+@coerce_props()
+@annotate()
+def parse(alts=None):
+ name = __name__.split(".")[-1]
+ with open_upstream_file(name, ZIP, "rb", seekable=True) as fz:
+ z = zipfile.ZipFile(fz)
+ with z.open(member(z, "sPlotOpen_header")) as fb:
+ reader = csv.reader(io.TextIOWrapper(fb, encoding="utf8", newline=""),
+ delimiter="\t")
+ hcols = next(reader)
+ plots = [row for row in reader]
+ with z.open(member(z, "sPlotOpen_DT")) as fb:
+ # one line has a garbled original species name (invalid UTF-8,
+ # with a null byte), whose invalid bytes are replaced
+ f = io.TextIOWrapper(fb, encoding="utf8", errors="replace", newline="")
+ reader = csv.reader((l.replace("\0", "") for l in f), delimiter="\t")
+ dcols = next(reader)
+ assert dcols == ["PlotObservationID", "Species", "Original_species",
+ "Original_abundance", "Abundance_scale",
+ "Relative_cover"], dcols
+ rows = [row for i, row in zip(range(_check_rows or sys.maxsize), reader)]
+ # the community-weighted means and variances of the traits, and the
+ # bibliographic metadata of the plots
+ tables = {}
+ for prefix in ["sPlotOpen_CWM_CWV", "sPlotOpen_metadata"]:
+ with z.open(member(z, prefix)) as fb:
+ reader = csv.reader(io.TextIOWrapper(fb, encoding="utf8", newline=""),
+ delimiter="\t")
+ h = next(reader)
+ assert h[0] == "PlotObservationID", h
+ t = {row[0]: row for row in reader}
+ assert all(len(row) == len(h) for row in t.values())
+ tables[prefix] = (h, t)
+ # the references in BibTeX format (of the keys of the metadata)
+ with z.open(member(z, "sPlotOpen_references")) as fb:
+ bib = fb.read().decode("utf8", errors="replace")
+
+ for row in plots + rows:
+ assert len(row) in (len(hcols), len(dcols)), row
+ c = {x: i for i, x in enumerate(hcols)}
+ nplots = len(plots)
+ pidx = {row[0]: i for i, row in enumerate(plots)}
+ assert len(pidx) == nplots
+ sidx = {}
+ edges = []
+ for row in rows:
+ s = sidx.setdefault(row[1], nplots + len(sidx))
+ edges.append((pidx[row[0]], s))
+
+ g = Graph(directed=False)
+ g.add_vertex(nplots + len(sidx))
+ g.add_edge_list(numpy.array(edges, dtype="int64").reshape(-1, 2))
+ g.vp.is_plot = g.new_vp("bool", vals=[True] * nplots + [False] * len(sidx))
+ species = sorted(sidx, key=lambda s: sidx[s])
+ g.vp.name = g.new_vp("string", vals=[row[0] for row in plots] + species)
+ def na(x):
+ return "" if x == "NA" else x
+ ns = [""] * len(sidx)
+ nn = [math.nan] * len(sidx)
+ for col in hcols[1:]:
+ p = col.lower()
+ if col in HEADER_STRINGS:
+ g.vp[p] = g.new_vp("string", vals=[na(row[c[col]]) for row in plots] + ns)
+ elif col in HEADER_BOOLS:
+ g.vp[p] = g.new_vp("bool", vals=[row[c[col]] == "TRUE" for row in plots] +
+ [False] * len(sidx))
+ elif col == "Date_of_recording":
+ g.vp.date = g.new_vp("int64_t", vals=[to_time(row[c[col]]) for row in plots] +
+ [-1] * len(sidx))
+ else:
+ g.vp[p] = g.new_vp("double", vals=[to_float(row[c[col]]) for row in plots] + nn)
+ pids = [row[0] for row in plots]
+ h, t = tables["sPlotOpen_CWM_CWV"]
+ assert set(t) == set(pids)
+ for i, col in enumerate(h[1:], 1):
+ p = col.lower()
+ if col == "Species_richness":
+ g.vp[p] = g.new_vp("int", vals=[int(t[x][i]) for x in pids] +
+ [-1] * len(sidx))
+ else:
+ g.vp[p] = g.new_vp("double", vals=[to_float(t[x][i]) for x in pids] + nn)
+ h, t = tables["sPlotOpen_metadata"]
+ assert set(t) == set(pids)
+ for i, col in enumerate(h[1:], 1):
+ if col == "GIVD_ID":
+ # (the same as in the header)
+ assert all(t[x][i] == row[c["GIVD_ID"]] for x, row in zip(pids, plots))
+ continue
+ g.vp[col.lower()] = g.new_vp("string", vals=[na(t[x][i]) for x in pids] + ns)
+ g.gp.references = g.new_gp("string", val=bib)
+ g.ep.relative_cover = g.new_ep("double", vals=[to_float(r[5]) for r in rows])
+ g.ep.original_abundance = g.new_ep("double", vals=[to_float(r[3]) for r in rows])
+ g.ep.abundance_scale = g.new_ep("string", vals=[r[4] for r in rows])
+ g.ep.original_species = g.new_ep("string", vals=[r[2] for r in rows])
+ yield None, g