heterophilous_graphs — Heterophilous node classification benchmarks (roman-empire, amazon-ratings, questions, minesweeper, 2023)

Description

Four of the networks assembled by Platonov et al. as benchmarks for node classification on graphs in which connected nodes tend to have different labels ("heterophily"), as distributed in their repository:

roman_empire: the words of the English Wikipedia article "Roman Empire" (from the dump of 1 March 2022), one of the longest articles of Wikipedia. Each node is one occurrence of a word in the text, and two words are connected if they follow each other in the text, or if they are connected in the dependency tree of their sentence (as parsed by spaCy). The network is thus a chain with syntactic shortcuts. Each word is labeled with its syntactic role (as given by spaCy): the 17 most frequent roles are separate classes, and all the others form an 18th class.
amazon_ratings: products of Amazon (books, music CDs, DVDs and VHS tapes), from the Amazon product co-purchasing metadata of SNAP (2006), connected if they are frequently bought together. The network is the largest connected component of the 5-core of the co-purchasing network. Each product is labeled with its average rating by reviewers, grouped into five classes.
questions: users of the question-answering website Yandex Q who are interested in the topic "medicine", connected if one of them answered a question of the other between September 2021 and August 2022. Each user is labeled by whether they remained active (their account was neither deleted nor blocked) at the end of that period.
minesweeper: a synthetic network, inspired by the game Minesweeper, and not derived from any real data: a regular 100 x 100 grid, in which each cell is connected to its (up to eight) neighbors, horizontally, vertically and diagonally. A random 20% of the cells are "mines", which is the label to predict. The node features of the benchmark give the number of mines among the neighbors of each cell, but only for a random half of the cells (for the other half it is marked as unknown), and they are kept as a property.

The edges are undirected. For the three real networks, the data do not include the words, products and users themselves (the text of the words, the product identifiers and the user names are not given), and the labels are given only as class indices: their meaning (e.g. which syntactic role, or which rating range, a class stands for) is not given. The node features of these benchmarks (averages of fastText embeddings of the words, or of the descriptions of products and users, with 300 dimensions) are left out, since they are derived data that would make the network files much larger; the only exception is the indicator of the users of questions without a description, kept as a property. The train, validation and test splits of the benchmarks are also left out. The fifth network of the benchmark, tolokers, is the entry tolokers.

The node indices are those of the source data. The networks have the following properties:

label: the class of the node, an index from 0 (the syntactic role for roman_empire, from 0 to 17, where 17 groups the less frequent roles; the rating class for amazon_ratings, from 0 to 4; for questions, 1 if the user was no longer active and 0 otherwise; for minesweeper, 1 if the cell is a mine and 0 otherwise).
has_description (questions only): whether the user has a profile description.
row, column (minesweeper only): the position of the cell in the grid, from 0 to 99.
neighbor_mines (minesweeper only): the number of mines among the neighbors of the cell, as given by the node features (from 0 to 4, or 5 for five or more), or -1 if it is unknown.

Tags

Informational Language Economic Commerce Social Online Unweighted Metadata

Citation

O. Platonov, D. Kuznedelev, M. Diskin, A. Babenko, L. Prokhorenkova, "A critical look at the evaluation of GNNs under heterophily: Are we really making progress?" International Conference on Learning Representations (ICLR) (2023) 10.48550/arXiv.2302.11640 Sci-Hub

Upstream URL OK

https://github.com/yandex-research/heterophilous-graphs

Upstream license

MIT, https://github.com/yandex-research/heterophilous-graphs/blob/a431395582e929d88271309716bea4fe24ce6318/LICENSE

Networks

Tip: click on a column header to sort the list, or hover over it for a description.

Column legend

$\left<k\right>$: Mean degree
$\sigma_k$: Standard deviation of the degrees
$\lambda_h$: Largest eigenvalue of the Hashimoto (non-backtracking) matrix
$\tau$: Random walk mixing time
$r$: Degree assortativity
$c$: Global clustering coefficient
$\oslash$: (Pseudo-) diameter
$S$: Relative size of the largest component
NPs: Node properties
EPs: Edge properties

Name	Nodes	Edges	$\left<k\right>$	$\sigma_k$	$\lambda_h$	$\tau$	$r$	$c$	$\oslash$	$S$	Kind	Mode	NPs	gt	GraphML	GML	csv
roman_empire	22,662	32,927	2.91	1.03	2.80	9191502.76	-0.03	0.29	6,824	1.00	Undirected	Unipartite	label	69 KiB	148 KiB	145 KiB	167 KiB
amazon_ratings	24,492	93,050	7.60	5.97	17.64	9551.28	-0.09	0.32	46	1.00	Undirected	Unipartite	label	181 KiB	432 KiB	390 KiB	428 KiB
questions	48,921	153,540	6.28	27.11	91.84	121.48	-0.15	0.02	16	1.00	Undirected	Unipartite	label has_description	326 KiB	674 KiB	668 KiB	675 KiB
minesweeper	10,000	39,402	7.88	0.59	6.99	2658.97	0.39	0.43	99	1.00	Undirected	Unipartite	label row column neighbor_mines	38 KiB	134 KiB	118 KiB	129 KiB