logo Netzschleuder network catalog, repository, and centrifuge

History of wiki_norms

The changes to the source code of this entry in the repository, newest first, each with the differences it made to the entry's files. They are also listed on GitLab.

[entries] Add the date when each entry was added, and remove icon_hash by Tiago de Paula Peixoto beff7c9
The dates are those of the commits that first added the entries' files
(2020-08-11 for the entries of the initial commit); renamed entries keep
the date of their first name.

Changed: none
diff --git a/entries/wiki_norms/__init__.py b/entries/wiki_norms/__init__.py
index 55b7897..beb39d6 100644
--- a/entries/wiki_norms/__init__.py
+++ b/entries/wiki_norms/__init__.py
@@ -41,7 +41,7 @@ The source data also include a column "Final In Degree", which is zero for all p
 tags = ['Informational', 'Web graph', 'Unweighted', 'Metadata']
 url = 'https://sites.santafe.edu/~simon/styled-9/styled-10/'
 citation = [('B. Heaberlin, S. DeDeo, "The Evolution of Wikipedia\'s Norm Network." Future Internet 8(2), 14 (2016)', 'https://doi.org/10.3390/fi8020014')]
-icon_hash = '56a85c8e26855e083a2f795b'
+date_added = '2026-10-25'
 upstream_license = None
 upstream_prefix = 'https://sites.santafe.edu/~simon'
 files = [(('heaberlin_dedeo_norm_network.zip:links.csv',
[entries] Prefix the properties in the descriptions with (Node), (Edge) or (Graph) by Tiago de Paula Peixoto bc3dbbd
The lists of properties at the end of the descriptions marked edge
properties with "Edge" but left node properties unmarked, or grouped
them in "Nodes: `a`, ...; `b`, ..." items, so the kind of a property
was often unclear. Every item now starts with (Node), (Edge) or (Graph),
followed by the property names, a colon and the explanation, one item
per property or group of properties; qualifications (kinds of nodes,
networks) follow the names in parentheses. Properties that were only
named got a short explanation, and a few items that named the wrong
kind of property were corrected.
diff --git a/entries/wiki_norms/__init__.py b/entries/wiki_norms/__init__.py
index 2dd511c..55b7897 100644
--- a/entries/wiki_norms/__init__.py
+++ b/entries/wiki_norms/__init__.py
@@ -23,7 +23,17 @@ description = """The hyperlink network among the pages of the English Wikipedia
 
 The network has the following properties:
 
-- Nodes: `name`, the title of the page (without the "Wikipedia:" prefix); `creation_date`, the date of creation of the page (UTC); `type`, the classification of the page by the editors (Policy, Guideline, Essay, Proposal, Humor, Convention, Culture, Basic, Information, or Other); `community`, the Louvain community of the page in the giant component, numbered as in the paper (-1 for pages outside the giant component); `ec`, an estimate of the eigenvector centrality of the page, computed as its PageRank (with random jump probability 0.15); `pageviews`, the fraction of the page views of all pages of the network received by the page in July 2015 (from the StatsGrok service); `edits` and `editors`, the number of edits of the page and of distinct editors; `talk_edits` and `talk_editors`, the number of edits of the associated talk page and of distinct editors of it (the two columns are identical in the source data, so the latter likely repeats the former); `size`, the size of the page in bytes; `words`, the number of words of the page used in the topic model; `topics`, the topic distribution of the page in a latent Dirichlet allocation (LDA) model with 20 topics, ordered as in the paper (empty for the ten pages without text).
+- (Node) `name`: the title of the page (without the "Wikipedia:" prefix).
+- (Node) `creation_date`: the date of creation of the page (UTC).
+- (Node) `type`: the classification of the page by the editors (Policy, Guideline, Essay, Proposal, Humor, Convention, Culture, Basic, Information, or Other).
+- (Node) `community`: the Louvain community of the page in the giant component, numbered as in the paper (-1 for pages outside the giant component).
+- (Node) `ec`: an estimate of the eigenvector centrality of the page, computed as its PageRank (with random jump probability 0.15).
+- (Node) `pageviews`: the fraction of the page views of all pages of the network received by the page in July 2015 (from the StatsGrok service).
+- (Node) `edits` and `editors`: the number of edits of the page and of distinct editors.
+- (Node) `talk_edits` and `talk_editors`: the number of edits of the associated talk page and of distinct editors of it (the two columns are identical in the source data, so the latter likely repeats the former).
+- (Node) `size`: the size of the page in bytes.
+- (Node) `words`: the number of words of the page used in the topic model.
+- (Node) `topics`: the topic distribution of the page in a latent Dirichlet allocation (LDA) model with 20 topics, ordered as in the paper (empty for the ten pages without text).
 
 The source data also include a column "Final In Degree", which is zero for all pages and was left out.[^icon]
 
[entries] Remove unneeded code from the entries by Tiago de Paula Peixoto 2c69f19
Unneeded "global files" statements, unused imports and variables,
f-strings without placeholders, duplicate keys in foodweb_little_rock
(keeping the order and the values used), and the second definition of
get_lines() in twitter_higgs. Nothing changes in the networks or
descriptions.

Changed: none
diff --git a/entries/wiki_norms/__init__.py b/entries/wiki_norms/__init__.py
index 356f39b..2dd511c 100644
--- a/entries/wiki_norms/__init__.py
+++ b/entries/wiki_norms/__init__.py
@@ -17,7 +17,6 @@
 # with this program.  If not, see <http://www.gnu.org/licenses/>.
 
 from .. import *
-import io
 
 title = "Wikipedia norm network (2015)"
 description = """The hyperlink network among the pages of the English Wikipedia that describe its social norms, i.e. its policies, guidelines, essays and proposals, from B. Heaberlin and S. DeDeo. Nodes represent pages in the "Wikipedia:" namespace, and a directed edge from page $i$ to page $j$ means that the text of page $i$ contains at least one hyperlink to page $j$. The pages were collected by following all links within the namespace, starting from the page "Assume good faith", and include only pages that existed on 20 August 2015. Pages that are only lists, or that describe projects, outreach initiatives or noticeboards, were removed, and redirects (e.g. "WP:AGF") were merged with the pages they point to. Links from navigation boxes, infoboxes and other template-generated content were discarded, as well as self-links, so that the remaining links reflect deliberate connections between norms; multiple links between the same pair of pages are counted once. After this filtering, 95 pages have no links.
@@ -47,7 +46,6 @@ def fetch_upstream(force=False):
 @coerce_props()
 @annotate()
 def parse(alts=None):
-    global files
     name = __name__.split(".")[-1]
     for fnames, alt, fmt in files:
         if alts is not None and alt not in alts:
[entries] New entry: wiki_norms by Tiago de Paula Peixoto 26fc422
diff --git a/entries/wiki_norms/__init__.py b/entries/wiki_norms/__init__.py
new file mode 100644
index 0000000..356f39b
--- /dev/null
+++ b/entries/wiki_norms/__init__.py
@@ -0,0 +1,93 @@
+#! /usr/bin/env python
+# -*- coding: utf-8 -*-
+#
+# Copyright (C) 2020-2026 Tiago de Paula Peixoto <tiago@skewed.de>
+#
+# This program is free software: you can redistribute it and/or modify it under
+# the terms of the GNU Affero General Public License as published by the Free
+# Software Foundation, either version 3 of the License, or (at your option) any
+# later version.
+#
+# This program is distributed in the hope that it will be useful, but WITHOUT
+# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
+# FOR A PARTICULAR PURPOSE.  See the GNU Affero General Public License for more
+# details.
+#
+# You should have received a copy of the GNU Affero General Public License along
+# with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+from .. import *
+import io
+
+title = "Wikipedia norm network (2015)"
+description = """The hyperlink network among the pages of the English Wikipedia that describe its social norms, i.e. its policies, guidelines, essays and proposals, from B. Heaberlin and S. DeDeo. Nodes represent pages in the "Wikipedia:" namespace, and a directed edge from page $i$ to page $j$ means that the text of page $i$ contains at least one hyperlink to page $j$. The pages were collected by following all links within the namespace, starting from the page "Assume good faith", and include only pages that existed on 20 August 2015. Pages that are only lists, or that describe projects, outreach initiatives or noticeboards, were removed, and redirects (e.g. "WP:AGF") were merged with the pages they point to. Links from navigation boxes, infoboxes and other template-generated content were discarded, as well as self-links, so that the remaining links reflect deliberate connections between norms; multiple links between the same pair of pages are counted once. After this filtering, 95 pages have no links.
+
+The network has the following properties:
+
+- Nodes: `name`, the title of the page (without the "Wikipedia:" prefix); `creation_date`, the date of creation of the page (UTC); `type`, the classification of the page by the editors (Policy, Guideline, Essay, Proposal, Humor, Convention, Culture, Basic, Information, or Other); `community`, the Louvain community of the page in the giant component, numbered as in the paper (-1 for pages outside the giant component); `ec`, an estimate of the eigenvector centrality of the page, computed as its PageRank (with random jump probability 0.15); `pageviews`, the fraction of the page views of all pages of the network received by the page in July 2015 (from the StatsGrok service); `edits` and `editors`, the number of edits of the page and of distinct editors; `talk_edits` and `talk_editors`, the number of edits of the associated talk page and of distinct editors of it (the two columns are identical in the source data, so the latter likely repeats the former); `size`, the size of the page in bytes; `words`, the number of words of the page used in the topic model; `topics`, the topic distribution of the page in a latent Dirichlet allocation (LDA) model with 20 topics, ordered as in the paper (empty for the ten pages without text).
+
+The source data also include a column "Final In Degree", which is zero for all pages and was left out.[^icon]
+
+[^icon]: Description obtained in part from the [ICON](https://icon.colorado.edu) project."""
+tags = ['Informational', 'Web graph', 'Unweighted', 'Metadata']
+url = 'https://sites.santafe.edu/~simon/styled-9/styled-10/'
+citation = [('B. Heaberlin, S. DeDeo, "The Evolution of Wikipedia\'s Norm Network." Future Internet 8(2), 14 (2016)', 'https://doi.org/10.3390/fi8020014')]
+icon_hash = '56a85c8e26855e083a2f795b'
+upstream_license = None
+upstream_prefix = 'https://sites.santafe.edu/~simon'
+files = [(('heaberlin_dedeo_norm_network.zip:links.csv',
+           'heaberlin_dedeo_norm_network.zip:nodes.csv',
+           'heaberlin_dedeo_norm_network.zip:topics.csv'), None, None)]
+
+def fetch_upstream(force=False):
+    return fetch_upstream_files(__name__.split(".")[-1], upstream_prefix, files,
+                                force)
+
+@cache_network()
+@coerce_props()
+@annotate()
+def parse(alts=None):
+    global files
+    name = __name__.split(".")[-1]
+    for fnames, alt, fmt in files:
+        if alts is not None and alt not in alts:
+            continue
+        g = Graph(directed=True)
+        with open_upstream_file(name, fnames[1], "r") as f:
+            rows = list(csv.reader(f, delimiter="\t", quoting=csv.QUOTE_NONE))
+        header, rows = rows[0], rows[1:]
+        assert header[0] == "Page ID" and len(header) == 13
+        ids = [int(r[0]) for r in rows]
+        assert sorted(ids) == list(range(len(ids)))
+        g.add_vertex(len(ids))
+        props = [(1, "name", "string", str),
+                 (2, "creation_date", "string", str),
+                 (3, "type", "string", str),
+                 (4, "community", "int", int),
+                 (6, "ec", "double", float),
+                 (7, "pageviews", "double", float),
+                 (8, "edits", "int", int),
+                 (9, "editors", "int", int),
+                 (10, "talk_edits", "int", int),
+                 (11, "talk_editors", "int", int),
+                 (12, "size", "int", int)]
+        for col, pname, ptype, conv in props:
+            p = g.vp[pname] = g.new_vp(ptype)
+            for i, r in zip(ids, rows):
+                p[i] = conv(r[col])
+
+        g.vp.words = g.new_vp("int")
+        g.vp.topics = g.new_vp("vector<double>")
+        with open_upstream_file(name, fnames[2], "r") as f:
+            rows = list(csv.reader(f, delimiter="\t", quoting=csv.QUOTE_NONE))
+        assert rows[0][:2] == ["Page ID", "Number of Words"]
+        for r in rows[1:]:
+            v = g.vertex(int(r[0]))
+            g.vp.words[v] = round(float(r[1]))
+            g.vp.topics[v] = [float(x) for x in r[2:]]
+
+        with open_upstream_file(name, fnames[0], "r") as f:
+            rows = list(csv.reader(f))
+        assert rows[0] == ["Source", "Target"]
+        g.add_edge_list([(int(s), int(t)) for s, t in rows[1:]])
+        yield alt, g