History of wikipedia_link
The changes to the source code of this entry in the repository, newest first, each with the differences it made to the entry's files. They are also listed on GitLab.
[entries] Add the date when each entry was added, and remove icon_hash beff7c9
The dates are those of the commits that first added the entries' files (2020-08-11 for the entries of the initial commit); renamed entries keep the date of their first name. Changed: none
diff --git a/entries/wikipedia_link/__init__.py b/entries/wikipedia_link/__init__.py
index ad0b31e..6d218a8 100644
--- a/entries/wikipedia_link/__init__.py
+++ b/entries/wikipedia_link/__init__.py
@@ -33,7 +33,7 @@ The networks have the following properties:
tags = ['Informational', 'Web graph', 'Unweighted', 'Metadata']
url = 'https://dumps.wikimedia.org/'
citation = [('"Wikimedia Foundation. Wikimedia downloads."', 'https://dumps.wikimedia.org/')]
-icon_hash = '57f2d9bd8cf714f319c39129'
+date_added = '2020-08-11'
upstream_license = ("Creative Commons Attribution-ShareAlike 4.0 (CC BY-SA 4.0) and GFDL",
"https://dumps.wikimedia.org/legal.html")
[entries] Prefix the properties in the descriptions with (Node), (Edge) or (Graph) bc3dbbd
The lists of properties at the end of the descriptions marked edge properties with "Edge" but left node properties unmarked, or grouped them in "Nodes: `a`, ...; `b`, ..." items, so the kind of a property was often unclear. Every item now starts with (Node), (Edge) or (Graph), followed by the property names, a colon and the explanation, one item per property or group of properties; qualifications (kinds of nodes, networks) follow the names in parentheses. Properties that were only named got a short explanation, and a few items that named the wrong kind of property were corrected.
diff --git a/entries/wikipedia_link/__init__.py b/entries/wikipedia_link/__init__.py
index 0b23ec5..ad0b31e 100644
--- a/entries/wikipedia_link/__init__.py
+++ b/entries/wikipedia_link/__init__.py
@@ -27,9 +27,9 @@ These networks replace an earlier version extracted from the dumps of 2016, whic
The networks have the following properties:
-- `name`: the page title.
-- `is_redirect`: whether the page is a redirect.
-- `length`: the length of the page text in bytes."""
+- (Node) `name`: the page title.
+- (Node) `is_redirect`: whether the page is a redirect.
+- (Node) `length`: the length of the page text in bytes."""
tags = ['Informational', 'Web graph', 'Unweighted', 'Metadata']
url = 'https://dumps.wikimedia.org/'
citation = [('"Wikimedia Foundation. Wikimedia downloads."', 'https://dumps.wikimedia.org/')]
[entries] Remove unneeded code from the entries 2c69f19
Unneeded "global files" statements, unused imports and variables, f-strings without placeholders, duplicate keys in foodweb_little_rock (keeping the order and the values used), and the second definition of get_lines() in twitter_higgs. Nothing changes in the networks or descriptions. Changed: none
diff --git a/entries/wikipedia_link/__init__.py b/entries/wikipedia_link/__init__.py
index bc20b9a..0b23ec5 100644
--- a/entries/wikipedia_link/__init__.py
+++ b/entries/wikipedia_link/__init__.py
@@ -94,7 +94,6 @@ def fetch_upstream(force=False):
@coerce_props(vforce={"name": "string"})
@annotate()
def parse(alts=None):
- global files
name = __name__.split(".")[-1]
for fnames, alt, fmt in files:
if alts is not None and alt not in alts:
[entries] Remove the mentions of KONECT from the descriptions of edit_wikibooks, edit_wikinews, edit_wikipedia, edit_wikiquote, edit_wiktionary, linux, prosper, reactome, unicodelang, wiki_categories, wiki_link_dyn, wikipedia_link and wordnet 2216557
These entries were originally taken from KONECT and now download their data from the original sources, so their descriptions no longer refer to KONECT.
diff --git a/entries/wikipedia_link/__init__.py b/entries/wikipedia_link/__init__.py
index 3f2d952..bc20b9a 100644
--- a/entries/wikipedia_link/__init__.py
+++ b/entries/wikipedia_link/__init__.py
@@ -23,7 +23,7 @@ description = """Networks of hyperlinks between the articles of 168 language edi
For the Simple English Wikipedia, the evolving network of link additions and removals, reconstructed from the full edit history of the same dumps (without the links generated by templates), is in the entry `wiki_link_dyn`.
-These networks replace an earlier version extracted by KONECT from the dumps of 2016, which had no node names and omitted isolated pages.
+These networks replace an earlier version extracted from the dumps of 2016, which had no node names and omitted isolated pages.
The networks have the following properties:
[entries] Merge duplicate entries and add cross-references between related ones 9bc6033
- Removed duplicates, merged into the entries holding the same data: dblp_coauthor_snap (soc_net_comms), cattle and bison (dom; bison's breeding data move there, as do the wolves' age and sex), football_tsevans (football, with both conference assignments), eu_procurements_alt (eu_procurements), foursquare_friendships (foursquare_global). - software_dependencies: colt and scolt are one network; ambassador: the duplicate 2001 network is dropped and the three same-edge networks combined; dom: the dataset count corrected (436). - macaque_neural: Young's 71-area network added (young71; the 47-area one is now cocomac47), both from archived copies. - collins_yeast, celegans_interactomes: from archived copies of the dead CCSB site (byte-identical files). - Cross-references between entries holding versions of the same data (food webs, Wikipedia, Slashdot, physicians, SNAP/Mislove, CCSB). - ICON ids of duplicate records added; macaque_neural's corrected. - prosper, dbpedia_all: string properties given in edge-index order (needs graph-tool with the fixed vals ordering).
diff --git a/entries/wikipedia_link/__init__.py b/entries/wikipedia_link/__init__.py
index e7acd62..3f2d952 100644
--- a/entries/wikipedia_link/__init__.py
+++ b/entries/wikipedia_link/__init__.py
@@ -21,6 +21,8 @@ from .. import *
title = "Wikipedia links (2026)"
description = """Networks of hyperlinks between the articles of 168 language editions of Wikipedia, named by their language code (e.g. `en`, `de`, `simple` for the Simple English Wikipedia). They were extracted from the Wikimedia database dumps of 1 October 2026 (tables `page`, `pagelinks` and `linktarget`). Nodes are the pages in the main (article) namespace, including redirect pages, and a directed edge $(i,j)$ means that page $i$ contains a link to page $j$ (multiple links between the same pair of pages count once). The links are those recorded by MediaWiki for the current version of each page, which include the links generated by templates. Links to pages that do not exist ("red links") and to pages in other namespaces are omitted. A redirect page has a single outgoing edge, to the page it redirects to, and links to a redirect are not resolved to its target. Pages without any links are included as isolated nodes.
+For the Simple English Wikipedia, the evolving network of link additions and removals, reconstructed from the full edit history of the same dumps (without the links generated by templates), is in the entry `wiki_link_dyn`.
+
These networks replace an earlier version extracted by KONECT from the dumps of 2016, which had no node names and omitted isolated pages.
The networks have the following properties:
[entries] Rebuild the Wikimedia and Stack Overflow entries from the original dumps b65b158
- edit_wikibooks, edit_wikinews, edit_wikiquote, edit_wiktionary, wikipedia_link, wiki_categories, wiki_link_dyn: from the Wikimedia dumps of 2026-10-01 (newer than KONECT's extractions), with page titles, user names (including unregistered editors, marked `anonymous`), timestamps and namespaces; the sister projects gain the language editions added since. - stackoverflow: from the Stack Exchange data dump of December 2011 (the one KONECT used), with KONECT's swapped sides corrected and user and question properties.
diff --git a/entries/wikipedia_link/__init__.py b/entries/wikipedia_link/__init__.py
index 709d584..e7acd62 100644
--- a/entries/wikipedia_link/__init__.py
+++ b/entries/wikipedia_link/__init__.py
@@ -18,16 +18,30 @@
from .. import *
-title = "Wikipedia links (2016)"
-description = """Networks of hyperlinks among articles on Wikipedia, for all available languages. A directed edge (i,j) indicates that article i hyperlinks to j.[^icon]
-[^icon]: Description obtained from the [ICON](https://icon.colorado.edu) project."""
-tags = ['Informational', 'Web graph', 'Unweighted']
-url = 'http://konect.cc/networks/wikipedia_link_en'
-citation = [('J. Kunegis, "Wikipedia links." KONECT - The Koblenz Network Collection (2016).', 'https://doi.org/10.1145/2487788.2488173')]
+title = "Wikipedia links (2026)"
+description = """Networks of hyperlinks between the articles of 168 language editions of Wikipedia, named by their language code (e.g. `en`, `de`, `simple` for the Simple English Wikipedia). They were extracted from the Wikimedia database dumps of 1 October 2026 (tables `page`, `pagelinks` and `linktarget`). Nodes are the pages in the main (article) namespace, including redirect pages, and a directed edge $(i,j)$ means that page $i$ contains a link to page $j$ (multiple links between the same pair of pages count once). The links are those recorded by MediaWiki for the current version of each page, which include the links generated by templates. Links to pages that do not exist ("red links") and to pages in other namespaces are omitted. A redirect page has a single outgoing edge, to the page it redirects to, and links to a redirect are not resolved to its target. Pages without any links are included as isolated nodes.
+
+These networks replace an earlier version extracted by KONECT from the dumps of 2016, which had no node names and omitted isolated pages.
+
+The networks have the following properties:
+
+- `name`: the page title.
+- `is_redirect`: whether the page is a redirect.
+- `length`: the length of the page text in bytes."""
+tags = ['Informational', 'Web graph', 'Unweighted', 'Metadata']
+url = 'https://dumps.wikimedia.org/'
+citation = [('"Wikimedia Foundation. Wikimedia downloads."', 'https://dumps.wikimedia.org/')]
icon_hash = '57f2d9bd8cf714f319c39129'
-upstream_prefix = 'http://konect.cc/files'
-files = [(f'download.tsv.wikipedia_link_{la}.tar.bz2', f"{la}", 'konect') for la
- in ["af", "als", "am", "an", "ar", "arz", "as", "ast", "az", "ba",
+upstream_license = ("Creative Commons Attribution-ShareAlike 4.0 (CC BY-SA 4.0) and GFDL",
+ "https://dumps.wikimedia.org/legal.html")
+
+# Fixed dump date (see entries/edit_wikipedia)
+DUMP_DATE = "20261001"
+# Too large to process on an ordinary machine (see --skip-huge)
+huge = True
+upstream_prefix = 'https://dumps.wikimedia.org'
+
+languages = ["af", "als", "am", "an", "ar", "arz", "as", "ast", "az", "ba",
"bar", "bat_smg", "bcl", "be", "be_x_old", "bg", "bh", "bn", "bpy",
"br", "bs", "bug", "ca", "cdo", "ce", "ceb", "ckb", "co", "crh", "cs",
"csb", "cv", "cy", "da", "de", "diq", "dv", "el", "eml", "en", "eo",
@@ -43,14 +57,39 @@ files = [(f'download.tsv.wikipedia_link_{la}.tar.bz2', f"{la}", 'konect') for la
"simple", "sk", "sl", "so", "sq", "sr", "su", "sv", "sw", "ta", "te",
"tg", "th", "tk", "tl", "tr", "tt", "uk", "ur", "uz", "vec", "vi",
"vls", "vo", "wa", "war", "xal", "xmf", "yi", "yo", "zh",
- "zh_min_nan", "zh_yue"]]
+ "zh_min_nan", "zh_yue"]
+files = [(sql_files(f"{la}wiki", ["page", "pagelinks", "linktarget"], DUMP_DATE), la, None)
+ for la in languages]
+
+def parse_links(f_page, f_pagelinks, f_linktarget):
+ page_ids, redirect, length, titles, index = read_pages(f_page)
+ lt_ids, lt_vals = read_linktargets(f_linktarget, index)
+ del index
+ pidx = numpy.argsort(page_ids)
+ page_keys = page_ids[pidx]
+ edges = []
+ for rows in sql_int_rows(f_pagelinks, 3):
+ rows = rows[rows[:, 1] == 0] # links from the main namespace
+ s = lookup(page_keys, pidx, rows[:, 0])
+ t = lookup(lt_ids, lt_vals, rows[:, 2])
+ ok = (s >= 0) & (t >= 0)
+ edges.append(numpy.column_stack((s[ok], t[ok])).astype("int32"))
+ g = Graph(directed=True)
+ g.add_vertex(len(titles))
+ for e in edges:
+ g.add_edge_list(e)
+ del edges
+ g.vp.name = g.new_vp("string", vals=titles)
+ g.vp.is_redirect = g.new_vp("bool", vals=redirect)
+ g.vp.length = g.new_vp("int64_t", vals=length)
+ return g
def fetch_upstream(force=False):
return fetch_upstream_files(__name__.split(".")[-1], upstream_prefix, files,
force)
@cache_network()
-@coerce_props()
+@coerce_props(vforce={"name": "string"})
@annotate()
def parse(alts=None):
global files
@@ -58,9 +97,8 @@ def parse(alts=None):
for fnames, alt, fmt in files:
if alts is not None and alt not in alts:
continue
- if isinstance(fnames, str):
- fnames = [fnames]
with ExitStack() as stack:
- fs = [stack.enter_context(open_upstream_file(name, fn, "rb")) for fn in fnames]
- g = parse_graph(fs, fmt, directed=True)
+ fs = [stack.enter_context(open_upstream_file(name, fn, "rb"))
+ for fn in fnames]
+ g = parse_links(*fs)
yield alt, g
Update copyright information 73a4f0d
diff --git a/entries/wikipedia_link/__init__.py b/entries/wikipedia_link/__init__.py
index fb952a7..709d584 100644
--- a/entries/wikipedia_link/__init__.py
+++ b/entries/wikipedia_link/__init__.py
@@ -1,7 +1,7 @@
#! /usr/bin/env python
# -*- coding: utf-8 -*-
#
-# Copyright (C) 2020-2024 Tiago de Paula Peixoto <tiago@skewed.de>
+# Copyright (C) 2020-2026 Tiago de Paula Peixoto <tiago@skewed.de>
#
# This program is free software: you can redistribute it and/or modify it under
# the terms of the GNU Affero General Public License as published by the Free
Update copyright information 3d288ea
diff --git a/entries/wikipedia_link/__init__.py b/entries/wikipedia_link/__init__.py
index cf3733e..fb952a7 100644
--- a/entries/wikipedia_link/__init__.py
+++ b/entries/wikipedia_link/__init__.py
@@ -1,7 +1,7 @@
#! /usr/bin/env python
# -*- coding: utf-8 -*-
#
-# Copyright (C) 2020-2023 Tiago de Paula Peixoto <tiago@skewed.de>
+# Copyright (C) 2020-2024 Tiago de Paula Peixoto <tiago@skewed.de>
#
# This program is free software: you can redistribute it and/or modify it under
# the terms of the GNU Affero General Public License as published by the Free
Update copyright information 8d005ae
diff --git a/entries/wikipedia_link/__init__.py b/entries/wikipedia_link/__init__.py
index 1fb7f04..cf3733e 100644
--- a/entries/wikipedia_link/__init__.py
+++ b/entries/wikipedia_link/__init__.py
@@ -1,7 +1,7 @@
#! /usr/bin/env python
# -*- coding: utf-8 -*-
#
-# Copyright (C) 2021 Tiago de Paula Peixoto <tiago@skewed.de>
+# Copyright (C) 2020-2023 Tiago de Paula Peixoto <tiago@skewed.de>
#
# This program is free software: you can redistribute it and/or modify it under
# the terms of the GNU Affero General Public License as published by the Free
Update copyright info db2efcc
diff --git a/entries/wikipedia_link/__init__.py b/entries/wikipedia_link/__init__.py
index c4c2818..1fb7f04 100644
--- a/entries/wikipedia_link/__init__.py
+++ b/entries/wikipedia_link/__init__.py
@@ -1,7 +1,7 @@
#! /usr/bin/env python
# -*- coding: utf-8 -*-
#
-# Copyright (C) 2020 Tiago de Paula Peixoto <tiago@skewed.de>
+# Copyright (C) 2021 Tiago de Paula Peixoto <tiago@skewed.de>
#
# This program is free software: you can redistribute it and/or modify it under
# the terms of the GNU Affero General Public License as published by the Free
Un-orphan KONECT datasets 894eba5
KONECT has been brought back from the dead for the time being.
diff --git a/entries/wikipedia_link/__init__.py b/entries/wikipedia_link/__init__.py
index 1fb6577..c4c2818 100644
--- a/entries/wikipedia_link/__init__.py
+++ b/entries/wikipedia_link/__init__.py
@@ -26,7 +26,6 @@ url = 'http://konect.cc/networks/wikipedia_link_en'
citation = [('J. Kunegis, "Wikipedia links." KONECT - The Koblenz Network Collection (2016).', 'https://doi.org/10.1145/2487788.2488173')]
icon_hash = '57f2d9bd8cf714f319c39129'
upstream_prefix = 'http://konect.cc/files'
-orphan = True
files = [(f'download.tsv.wikipedia_link_{la}.tar.bz2', f"{la}", 'konect') for la
in ["af", "als", "am", "an", "ar", "arz", "as", "ast", "az", "ba",
"bar", "bat_smg", "bcl", "be", "be_x_old", "bg", "bh", "bn", "bpy",
Update citations to use DOIs e878874
diff --git a/entries/wikipedia_link/__init__.py b/entries/wikipedia_link/__init__.py
index 1ff013d..1fb6577 100644
--- a/entries/wikipedia_link/__init__.py
+++ b/entries/wikipedia_link/__init__.py
@@ -23,7 +23,7 @@ description = """Networks of hyperlinks among articles on Wikipedia, for all ava
[^icon]: Description obtained from the [ICON](https://icon.colorado.edu) project."""
tags = ['Informational', 'Web graph', 'Unweighted']
url = 'http://konect.cc/networks/wikipedia_link_en'
-citation = [('J. Kunegis, "Wikipedia links." KONECT - The Koblenz Network Collection (2016).', 'http://konect.cc/')]
+citation = [('J. Kunegis, "Wikipedia links." KONECT - The Koblenz Network Collection (2016).', 'https://doi.org/10.1145/2487788.2488173')]
icon_hash = '57f2d9bd8cf714f319c39129'
upstream_prefix = 'http://konect.cc/files'
orphan = True
Add support for 'orphan' datasets and tag konect entries e330b80
diff --git a/entries/wikipedia_link/__init__.py b/entries/wikipedia_link/__init__.py
index 1c1698b..1ff013d 100644
--- a/entries/wikipedia_link/__init__.py
+++ b/entries/wikipedia_link/__init__.py
@@ -26,6 +26,7 @@ url = 'http://konect.cc/networks/wikipedia_link_en'
citation = [('J. Kunegis, "Wikipedia links." KONECT - The Koblenz Network Collection (2016).', 'http://konect.cc/')]
icon_hash = '57f2d9bd8cf714f319c39129'
upstream_prefix = 'http://konect.cc/files'
+orphan = True
files = [(f'download.tsv.wikipedia_link_{la}.tar.bz2', f"{la}", 'konect') for la
in ["af", "als", "am", "an", "ar", "arz", "as", "ast", "az", "ba",
"bar", "bat_smg", "bcl", "be", "be_x_old", "bg", "bh", "bn", "bpy",
Change license from GPLv3 to AGPLv3 d754be5
diff --git a/entries/wikipedia_link/__init__.py b/entries/wikipedia_link/__init__.py
index dae7425..1c1698b 100644
--- a/entries/wikipedia_link/__init__.py
+++ b/entries/wikipedia_link/__init__.py
@@ -3,18 +3,18 @@
#
# Copyright (C) 2020 Tiago de Paula Peixoto <tiago@skewed.de>
#
-# This program is free software: you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation, either version 3 of the License, or
-# (at your option) any later version.
+# This program is free software: you can redistribute it and/or modify it under
+# the terms of the GNU Affero General Public License as published by the Free
+# Software Foundation, either version 3 of the License, or (at your option) any
+# later version.
#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
+# This program is distributed in the hope that it will be useful, but WITHOUT
+# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
+# FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
+# details.
#
-# You should have received a copy of the GNU General Public License
-# along with this program. If not, see <http://www.gnu.org/licenses/>.
+# You should have received a copy of the GNU Affero General Public License along
+# with this program. If not, see <http://www.gnu.org/licenses/>.
from .. import *
Add explicit attribution to descriptions obtained from ICON 1645314
diff --git a/entries/wikipedia_link/__init__.py b/entries/wikipedia_link/__init__.py
index fccbb8a..dae7425 100644
--- a/entries/wikipedia_link/__init__.py
+++ b/entries/wikipedia_link/__init__.py
@@ -19,7 +19,8 @@
from .. import *
title = "Wikipedia links (2016)"
-description = "Networks of hyperlinks among articles on Wikipedia, for all available languages. A directed edge (i,j) indicates that article i hyperlinks to j."
+description = """Networks of hyperlinks among articles on Wikipedia, for all available languages. A directed edge (i,j) indicates that article i hyperlinks to j.[^icon]
+[^icon]: Description obtained from the [ICON](https://icon.colorado.edu) project."""
tags = ['Informational', 'Web graph', 'Unweighted']
url = 'http://konect.cc/networks/wikipedia_link_en'
citation = [('J. Kunegis, "Wikipedia links." KONECT - The Koblenz Network Collection (2016).', 'http://konect.cc/')]
Initial commit cc85872
diff --git a/entries/wikipedia_link/__init__.py b/entries/wikipedia_link/__init__.py
new file mode 100644
index 0000000..fccbb8a
--- /dev/null
+++ b/entries/wikipedia_link/__init__.py
@@ -0,0 +1,65 @@
+#! /usr/bin/env python
+# -*- coding: utf-8 -*-
+#
+# Copyright (C) 2020 Tiago de Paula Peixoto <tiago@skewed.de>
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+from .. import *
+
+title = "Wikipedia links (2016)"
+description = "Networks of hyperlinks among articles on Wikipedia, for all available languages. A directed edge (i,j) indicates that article i hyperlinks to j."
+tags = ['Informational', 'Web graph', 'Unweighted']
+url = 'http://konect.cc/networks/wikipedia_link_en'
+citation = [('J. Kunegis, "Wikipedia links." KONECT - The Koblenz Network Collection (2016).', 'http://konect.cc/')]
+icon_hash = '57f2d9bd8cf714f319c39129'
+upstream_prefix = 'http://konect.cc/files'
+files = [(f'download.tsv.wikipedia_link_{la}.tar.bz2', f"{la}", 'konect') for la
+ in ["af", "als", "am", "an", "ar", "arz", "as", "ast", "az", "ba",
+ "bar", "bat_smg", "bcl", "be", "be_x_old", "bg", "bh", "bn", "bpy",
+ "br", "bs", "bug", "ca", "cdo", "ce", "ceb", "ckb", "co", "crh", "cs",
+ "csb", "cv", "cy", "da", "de", "diq", "dv", "el", "eml", "en", "eo",
+ "es", "et", "eu", "fa", "fi", "fiu_vro", "fo", "fr", "fy", "ga",
+ "gag", "gan", "gd", "gl", "glk", "gu", "gv", "hak", "he", "hi", "hif",
+ "hr", "hsb", "ht", "hu", "hy", "ia", "id", "ilo", "io", "is", "it",
+ "ja", "jv", "ka", "kk", "kn", "ko", "ku", "kv", "ky", "la", "lb",
+ "lez", "li", "lmo", "lo", "lt", "lv", "map_bms", "mg", "mhr", "mi",
+ "min", "mk", "ml", "mn", "mr", "ms", "my", "myv", "mzn", "nah", "nap",
+ "nds", "nds_nl", "ne", "new", "nl", "nn", "no", "nso", "oc", "or",
+ "os", "pa", "pam", "pl", "pms", "pnb", "ps", "pt", "qu", "ro",
+ "roa_tara", "ru", "rue", "sa", "sah", "scn", "sco", "se", "sh", "si",
+ "simple", "sk", "sl", "so", "sq", "sr", "su", "sv", "sw", "ta", "te",
+ "tg", "th", "tk", "tl", "tr", "tt", "uk", "ur", "uz", "vec", "vi",
+ "vls", "vo", "wa", "war", "xal", "xmf", "yi", "yo", "zh",
+ "zh_min_nan", "zh_yue"]]
+
+def fetch_upstream(force=False):
+ return fetch_upstream_files(__name__.split(".")[-1], upstream_prefix, files,
+ force)
+
+@cache_network()
+@coerce_props()
+@annotate()
+def parse(alts=None):
+ global files
+ name = __name__.split(".")[-1]
+ for fnames, alt, fmt in files:
+ if alts is not None and alt not in alts:
+ continue
+ if isinstance(fnames, str):
+ fnames = [fnames]
+ with ExitStack() as stack:
+ fs = [stack.enter_context(open_upstream_file(name, fn, "rb")) for fn in fnames]
+ g = parse_graph(fs, fmt, directed=True)
+ yield alt, g