History of transfermarkt
The changes to the source code of this entry in the repository, newest first, each with the differences it made to the entry's files. They are also listed on GitLab.
[entries] Add the date when each entry was added, and remove icon_hash beff7c9
The dates are those of the commits that first added the entries' files (2020-08-11 for the entries of the initial commit); renamed entries keep the date of their first name. Changed: none
diff --git a/entries/transfermarkt/__init__.py b/entries/transfermarkt/__init__.py
index ad321e4..d19c6bb 100644
--- a/entries/transfermarkt/__init__.py
+++ b/entries/transfermarkt/__init__.py
@@ -91,7 +91,7 @@ url = ['https://github.com/dcaribou/transfermarkt-datasets',
'https://www.kaggle.com/datasets/davidcariboo/player-scores']
citation = [('David (dcaribou), "Football Data from Transfermarkt." Kaggle, dataset, version 679 (2026)', 'https://www.kaggle.com/datasets/davidcariboo/player-scores/versions/679')]
upstream_license = ('Creative Commons CC0 1.0 Universal', 'https://creativecommons.org/publicdomain/zero/1.0/')
-icon_hash = None
+date_added = '2026-10-31'
# Version 679 (2026-09-05) of the Kaggle copy of the dataset, the last one
# (updates were paused in July 2026; data current to 2026-07-06)
upstream_prefix = 'kaggle://davidcariboo/player-scores/679'
[entries] Prefix the properties in the descriptions with (Node), (Edge) or (Graph) bc3dbbd
The lists of properties at the end of the descriptions marked edge properties with "Edge" but left node properties unmarked, or grouped them in "Nodes: `a`, ...; `b`, ..." items, so the kind of a property was often unclear. Every item now starts with (Node), (Edge) or (Graph), followed by the property names, a colon and the explanation, one item per property or group of properties; qualifications (kinds of nodes, networks) follow the names in parentheses. Properties that were only named got a short explanation, and a few items that named the wrong kind of property were corrected.
diff --git a/entries/transfermarkt/__init__.py b/entries/transfermarkt/__init__.py
index ad58b44..ad321e4 100644
--- a/entries/transfermarkt/__init__.py
+++ b/entries/transfermarkt/__init__.py
@@ -33,14 +33,59 @@ The game events (goals, cards, substitutions) and the histories of market values
The networks have the following properties:
-- Nodes of `transfers` and `games`: `club_id`, the id of the club (or national team) in Transfermarkt; `name`, its name; `domestic_competition`, the id of its domestic league, for the clubs of the covered leagues (e.g. `GB1` for the Premier League, `L1` for the Bundesliga), empty otherwise. In `games`, also `national_team`, whether the team is a national team.
-- Edges of `transfers`: `player_id` and `player_name`, the id and name of the player; `time`, the date of the transfer (Unix time); `season`, the season of the transfer (e.g. `24/25`); `fee`, the transfer fee (euros; NaN if not given; 0 for free transfers); `market_value`, the market value of the player at the time (euros; NaN if not given).
-- Edges of `games`: `game_id`, the id of the game in Transfermarkt; `time`, the date of the game (Unix time); `competition_id`, the competition (e.g. `GB1`, `CL` for the UEFA Champions League, `FIWC` for the World Cup); `competition_type`, its type (`domestic_league`, `domestic_cup`, `international_cup`, `national_team_competition`, `other`; empty for 1,214 games); `season`, the year in which the season started; `round`, the round or matchday; `home_goals` and `away_goals`, the goals scored by the home and away teams; `attendance`, the number of spectators (-1 if not given); `stadium`; `referee`; `home_manager` and `away_manager`, the managers of the two teams; `home_formation` and `away_formation`, their formations (e.g. `4-2-3-1`). Missing values are empty strings or -1.
-- Nodes of `teammates`: `player_id`, the id of the player in Transfermarkt; `name`; `has_profile`, whether the dataset has a profile of the player, from which the following properties come (empty strings, -1 or NaN otherwise): `date_of_birth` (YYYY-MM-DD); `country_of_birth`, `city_of_birth` and `country_of_citizenship`; `position` (e.g. `Attack`) and `sub_position` (e.g. `Centre-Forward`); `foot`, the preferred foot; `height` (cm); `current_club_id` and `current_club_name`, the club of the player in his last season in the data; `market_value` and `highest_market_value`, his latest and highest market values (euros).
-- Edges of `teammates`: `games`, the number of games in which both players were listed for the same club; `games_played`, the number of those in which both played (according to the appearances table); `first_time` and `last_time`, the dates of the first and the last of these games (Unix time).
-- Nodes of `player_club` and `player_game`: `is_player`, whether the node is a player (the other nodes are clubs or games); `player_id`, the id of the player (-1 for the other nodes; the profiles of the players are in `teammates`); `name`, the name of the player, or of the club (in `player_club`; empty for a few clubs without a name in the data; empty for the games). In `player_club`, also `club_id`, the id of the club (-1 for players), and `national_team`, whether it is a national team. In `player_game`, also `game_id`, the id of the game (-1 for players), `time`, its date (Unix time), `competition_id` and `season` (as in `games`; empty for players).
-- Edges of `player_club`: `games`, the number of games in which the player was listed for the club; `games_played`, the number of those in which he played; `first_time` and `last_time`, the dates of the first and the last of these games (Unix time).
-- Edges of `player_game`: `club_id`, the club for which the player was listed; `time`, the date of the game (Unix time); from the lineups (empty strings if the player is not in the lineup of the game), `lineup`, `starting_lineup` or `substitutes`, `position`, the position (e.g. `Centre-Back`), `number`, the shirt number, and `captain`, whether he was the captain; `played`, whether the player has an appearance in the game; from the appearances (-1 if not available), `minutes_played`, `goals`, `assists`, `yellow_cards` and `red_cards`."""
+- (Node) `club_id` (`transfers` and `games`): the id of the club (or national team) in Transfermarkt.
+- (Node) `name` (`transfers` and `games`): its name.
+- (Node) `domestic_competition` (`transfers` and `games`): the id of its domestic league, for the clubs of the covered leagues (e.g. `GB1` for the Premier League, `L1` for the Bundesliga), empty otherwise.
+- (Node) `national_team` (`games`): whether the team is a national team.
+- (Edge) `player_id`, `player_name` (`transfers`): the id and name of the player.
+- (Edge) `time` (`transfers`): the date of the transfer (Unix time).
+- (Edge) `season` (`transfers`): the season of the transfer (e.g. `24/25`).
+- (Edge) `fee` (`transfers`): the transfer fee (euros; NaN if not given; 0 for free transfers).
+- (Edge) `market_value` (`transfers`): the market value of the player at the time (euros; NaN if not given).
+- (Edge) `game_id` (`games`): the id of the game in Transfermarkt.
+- (Edge) `time` (`games`): the date of the game (Unix time).
+- (Edge) `competition_id` (`games`): the competition (e.g. `GB1`, `CL` for the UEFA Champions League, `FIWC` for the World Cup).
+- (Edge) `competition_type` (`games`): its type (`domestic_league`, `domestic_cup`, `international_cup`, `national_team_competition`, `other`; empty for 1,214 games).
+- (Edge) `season` (`games`): the year in which the season started.
+- (Edge) `round` (`games`): the round or matchday.
+- (Edge) `home_goals`, `away_goals` (`games`): the goals scored by the home and away teams.
+- (Edge) `attendance` (`games`): the number of spectators (-1 if not given).
+- (Edge) `stadium`, `referee` (`games`): the stadium and the referee.
+- (Edge) `home_manager`, `away_manager` (`games`): the managers of the two teams.
+- (Edge) `home_formation`, `away_formation` (`games`): their formations (e.g. `4-2-3-1`).
+- (Edge) Missing values of the edges of `games` are empty strings or -1.
+- (Node) `player_id` (`teammates`): the id of the player in Transfermarkt.
+- (Node) `name` (`teammates`): the name of the player.
+- (Node) `has_profile` (`teammates`): whether the dataset has a profile of the player, from which the following properties come (empty strings, -1 or NaN otherwise).
+- (Node) `date_of_birth` (`teammates`): the date of birth (YYYY-MM-DD).
+- (Node) `country_of_birth`, `city_of_birth`, `country_of_citizenship` (`teammates`): the country and city of birth, and the country of citizenship.
+- (Node) `position`, `sub_position` (`teammates`): the position (e.g. `Attack`) and sub-position (e.g. `Centre-Forward`).
+- (Node) `foot` (`teammates`): the preferred foot.
+- (Node) `height` (`teammates`): the height (cm).
+- (Node) `current_club_id`, `current_club_name` (`teammates`): the club of the player in his last season in the data.
+- (Node) `market_value`, `highest_market_value` (`teammates`): his latest and highest market values (euros).
+- (Edge) `games` (`teammates`): the number of games in which both players were listed for the same club.
+- (Edge) `games_played` (`teammates`): the number of those in which both played (according to the appearances table).
+- (Edge) `first_time`, `last_time` (`teammates`): the dates of the first and the last of these games (Unix time).
+- (Node) `is_player` (`player_club` and `player_game`): whether the node is a player (the other nodes are clubs or games).
+- (Node) `player_id` (`player_club` and `player_game`): the id of the player (-1 for the other nodes; the profiles of the players are in `teammates`).
+- (Node) `name` (`player_club` and `player_game`): the name of the player, or of the club (in `player_club`; empty for a few clubs without a name in the data; empty for the games).
+- (Node) `club_id` (`player_club`): the id of the club (-1 for players).
+- (Node) `national_team` (`player_club`): whether it is a national team.
+- (Node) `game_id` (`player_game`): the id of the game (-1 for players).
+- (Node) `time` (`player_game`): its date (Unix time).
+- (Node) `competition_id`, `season` (`player_game`): as in `games` (empty for players).
+- (Edge) `games` (`player_club`): the number of games in which the player was listed for the club.
+- (Edge) `games_played` (`player_club`): the number of those in which he played.
+- (Edge) `first_time`, `last_time` (`player_club`): the dates of the first and the last of these games (Unix time).
+- (Edge) `club_id` (`player_game`): the club for which the player was listed.
+- (Edge) `time` (`player_game`): the date of the game (Unix time).
+- (Edge) `lineup` (`player_game`): from the lineups, `starting_lineup` or `substitutes` (empty if the player is not in the lineup of the game).
+- (Edge) `position` (`player_game`): from the lineups, the position (e.g. `Centre-Back`; empty if the player is not in the lineup of the game).
+- (Edge) `number` (`player_game`): from the lineups, the shirt number (empty if the player is not in the lineup of the game).
+- (Edge) `captain` (`player_game`): from the lineups, whether he was the captain.
+- (Edge) `played` (`player_game`): whether the player has an appearance in the game.
+- (Edge) `minutes_played`, `goals`, `assists`, `yellow_cards`, `red_cards` (`player_game`): from the appearances (-1 if not available)."""
tags = ['Social', 'Sports', 'Economic', 'Multigraph', 'Bipartite', 'Weighted', 'Timestamps', 'Metadata']
url = ['https://github.com/dcaribou/transfermarkt-datasets',
'https://www.kaggle.com/datasets/davidcariboo/player-scores']
[entries] New entry: transfermarkt e44eef7
diff --git a/entries/transfermarkt/__init__.py b/entries/transfermarkt/__init__.py
new file mode 100644
index 0000000..ad58b44
--- /dev/null
+++ b/entries/transfermarkt/__init__.py
@@ -0,0 +1,352 @@
+#! /usr/bin/env python
+# -*- coding: utf-8 -*-
+#
+# Copyright (C) 2026 Tiago de Paula Peixoto <tiago@skewed.de>
+#
+# This program is free software: you can redistribute it and/or modify it under
+# the terms of the GNU Affero General Public License as published by the Free
+# Software Foundation, either version 3 of the License, or (at your option) any
+# later version.
+#
+# This program is distributed in the hope that it will be useful, but WITHOUT
+# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
+# FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
+# details.
+#
+# You should have received a copy of the GNU Affero General Public License along
+# with this program. If not, see <http://www.gnu.org/licenses/>.
+
+from .. import *
+
+title = "Transfermarkt football transfers, matches, teammates and appearances (2026)"
+description = """Networks of football (soccer) clubs and players, from the "transfermarkt-datasets" of David (dcaribou), a dataset scraped from the football website Transfermarkt and cleaned into tables of competitions, clubs, players, games, lineups, appearances and transfers. It covers the games of 70 competitions (the first divisions of the main European leagues, their national cups, the European club competitions, and the main national-team tournaments, e.g. the World Cup), mostly from the 2012/13 season on (a few national-team tournaments since 2006), and the transfers of the players of the clubs of these leagues (since 1993). The dataset was updated weekly until its updates were paused in July 2026; the networks are of the last version (version 679 of the Kaggle copy, September 5, 2026), which is current to July 6, 2026.
+
+There are five networks:
+
+- `transfers`: a directed multigraph of clubs, with an edge from the club a player left to the club he joined, for each of the 175,165 transfers in the data (including loans and returns from loans, and moves to and from youth or reserve teams, which are clubs of their own, and to "Without Club" or "Retired"). Clubs outside the covered leagues are included as far as they appear in the transfers. 8 transfers are from a club to itself (self-loops). Some transfers are dated in the future (519 after July 6, 2026, up to 2030), as given in the data. A club can appear under several names (e.g. after renaming); the latest one is used.
+- `games`: a directed multigraph of clubs and national teams, with an edge from the home team to the away team for each of the 88,958 games.
+- `teammates`: an undirected network of players, with an edge between two players listed for the same club in the same game (in the starting lineup or among the substitutes, from the lineups table, or as having played in the game, from the appearances table), weighted by the number of such games. The lineups are available for 81,197 games, and the appearances for 73,426 games (only for the players of clubs in the covered domestic leagues). The players without a profile in the dataset (most of those of clubs outside the covered leagues) are included with their names only; the players with a profile but without any game in the lineups or appearances are included as isolated nodes.
+- `player_club`: a bipartite network of players and clubs (or national teams), with an edge between a player and each club for which he was listed in a game (in the lineups or the appearances, as in `teammates`), weighted by the number of such games (115,515 players, 3,156 clubs and 222,975 edges). Only the players listed in games are included.
+- `player_game`: a bipartite network of players and games, with an edge for each player listed in a game (in the starting lineup or among the substitutes, or as having played in it), with the details of the lineup and of the appearance (3,316,321 edges, of which 1,894,350 with an appearance).
+
+The game events (goals, cards, substitutions) and the histories of market values of the players are not included.
+
+The networks have the following properties:
+
+- Nodes of `transfers` and `games`: `club_id`, the id of the club (or national team) in Transfermarkt; `name`, its name; `domestic_competition`, the id of its domestic league, for the clubs of the covered leagues (e.g. `GB1` for the Premier League, `L1` for the Bundesliga), empty otherwise. In `games`, also `national_team`, whether the team is a national team.
+- Edges of `transfers`: `player_id` and `player_name`, the id and name of the player; `time`, the date of the transfer (Unix time); `season`, the season of the transfer (e.g. `24/25`); `fee`, the transfer fee (euros; NaN if not given; 0 for free transfers); `market_value`, the market value of the player at the time (euros; NaN if not given).
+- Edges of `games`: `game_id`, the id of the game in Transfermarkt; `time`, the date of the game (Unix time); `competition_id`, the competition (e.g. `GB1`, `CL` for the UEFA Champions League, `FIWC` for the World Cup); `competition_type`, its type (`domestic_league`, `domestic_cup`, `international_cup`, `national_team_competition`, `other`; empty for 1,214 games); `season`, the year in which the season started; `round`, the round or matchday; `home_goals` and `away_goals`, the goals scored by the home and away teams; `attendance`, the number of spectators (-1 if not given); `stadium`; `referee`; `home_manager` and `away_manager`, the managers of the two teams; `home_formation` and `away_formation`, their formations (e.g. `4-2-3-1`). Missing values are empty strings or -1.
+- Nodes of `teammates`: `player_id`, the id of the player in Transfermarkt; `name`; `has_profile`, whether the dataset has a profile of the player, from which the following properties come (empty strings, -1 or NaN otherwise): `date_of_birth` (YYYY-MM-DD); `country_of_birth`, `city_of_birth` and `country_of_citizenship`; `position` (e.g. `Attack`) and `sub_position` (e.g. `Centre-Forward`); `foot`, the preferred foot; `height` (cm); `current_club_id` and `current_club_name`, the club of the player in his last season in the data; `market_value` and `highest_market_value`, his latest and highest market values (euros).
+- Edges of `teammates`: `games`, the number of games in which both players were listed for the same club; `games_played`, the number of those in which both played (according to the appearances table); `first_time` and `last_time`, the dates of the first and the last of these games (Unix time).
+- Nodes of `player_club` and `player_game`: `is_player`, whether the node is a player (the other nodes are clubs or games); `player_id`, the id of the player (-1 for the other nodes; the profiles of the players are in `teammates`); `name`, the name of the player, or of the club (in `player_club`; empty for a few clubs without a name in the data; empty for the games). In `player_club`, also `club_id`, the id of the club (-1 for players), and `national_team`, whether it is a national team. In `player_game`, also `game_id`, the id of the game (-1 for players), `time`, its date (Unix time), `competition_id` and `season` (as in `games`; empty for players).
+- Edges of `player_club`: `games`, the number of games in which the player was listed for the club; `games_played`, the number of those in which he played; `first_time` and `last_time`, the dates of the first and the last of these games (Unix time).
+- Edges of `player_game`: `club_id`, the club for which the player was listed; `time`, the date of the game (Unix time); from the lineups (empty strings if the player is not in the lineup of the game), `lineup`, `starting_lineup` or `substitutes`, `position`, the position (e.g. `Centre-Back`), `number`, the shirt number, and `captain`, whether he was the captain; `played`, whether the player has an appearance in the game; from the appearances (-1 if not available), `minutes_played`, `goals`, `assists`, `yellow_cards` and `red_cards`."""
+tags = ['Social', 'Sports', 'Economic', 'Multigraph', 'Bipartite', 'Weighted', 'Timestamps', 'Metadata']
+url = ['https://github.com/dcaribou/transfermarkt-datasets',
+ 'https://www.kaggle.com/datasets/davidcariboo/player-scores']
+citation = [('David (dcaribou), "Football Data from Transfermarkt." Kaggle, dataset, version 679 (2026)', 'https://www.kaggle.com/datasets/davidcariboo/player-scores/versions/679')]
+upstream_license = ('Creative Commons CC0 1.0 Universal', 'https://creativecommons.org/publicdomain/zero/1.0/')
+icon_hash = None
+# Version 679 (2026-09-05) of the Kaggle copy of the dataset, the last one
+# (updates were paused in July 2026; data current to 2026-07-06)
+upstream_prefix = 'kaggle://davidcariboo/player-scores/679'
+files = [(("player-scores.zip:transfers.csv",
+ "player-scores.zip:clubs.csv"), "transfers", None),
+ (("player-scores.zip:games.csv",
+ "player-scores.zip:clubs.csv",
+ "player-scores.zip:national_teams.csv"), "games", None),
+ (("player-scores.zip:game_lineups.csv",
+ "player-scores.zip:appearances.csv",
+ "player-scores.zip:players.csv"), "teammates", None),
+ (("player-scores.zip:game_lineups.csv",
+ "player-scores.zip:appearances.csv",
+ "player-scores.zip:games.csv",
+ "player-scores.zip:clubs.csv",
+ "player-scores.zip:national_teams.csv"), "player_club", None),
+ (("player-scores.zip:game_lineups.csv",
+ "player-scores.zip:appearances.csv",
+ "player-scores.zip:games.csv"), "player_game", None)]
+
+def fetch_upstream(force=False):
+ return fetch_upstream_files(__name__.split(".")[-1], upstream_prefix, files,
+ force)
+
+def read_csv(name, fname, **kwargs):
+ import polars as pl
+ with open_upstream_file(name, fname, "rb") as f:
+ return pl.read_csv(f.read(), infer_schema_length=0, **kwargs)
+
+def unix_time(col):
+ "Dates (YYYY-MM-DD) as Unix time (-1 where missing)."
+ import polars as pl
+ return (pl.col(col).str.slice(0, 10).str.to_date(strict=False)
+ .cast(pl.Datetime("ms")).dt.epoch("s").fill_null(-1))
+
+def to_float(col):
+ import polars as pl
+ return pl.col(col).cast(pl.Float64, strict=False).fill_null(numpy.nan)
+
+def to_int(col):
+ import polars as pl
+ return pl.col(col).cast(pl.Float64, strict=False).fill_null(-1).cast(pl.Int64)
+
+def club_graph(directed, ids, names, comps):
+ "Graph with the given clubs as nodes, with their properties."
+ g = Graph(directed=directed)
+ g.add_vertex(len(ids))
+ g.vp.club_id = g.new_vp("int64_t", vals=[int(i) for i in ids])
+ g.vp.name = g.new_vp("string", vals=[names.get(i) or "" for i in ids])
+ g.vp.domestic_competition = g.new_vp("string", vals=[comps.get(i) or ""
+ for i in ids])
+ return g
+
+def player_rows(name, fnames):
+ """The players listed for a club in a game (in the lineups, or in the
+ appearances), one row per game and player, with the details of both
+ tables (null where a table has no row)."""
+ import polars as pl
+ ids = ["game_id", "player_id"]
+ lu = (read_csv(name, fnames[0],
+ columns=ids + ["club_id", "date", "player_name", "type",
+ "position", "number", "team_captain"])
+ .unique(ids, keep="first", maintain_order=True))
+ ap = (read_csv(name, fnames[1],
+ columns=ids + ["player_club_id", "date", "player_name",
+ "yellow_cards", "red_cards", "goals",
+ "assists", "minutes_played"])
+ .rename({"player_club_id": "club_id"})
+ .unique(ids, keep="first", maintain_order=True))
+ rows = (lu.join(ap, on=ids, how="full", coalesce=True, suffix="_ap")
+ .with_columns(pl.col("club_id").fill_null(pl.col("club_id_ap")),
+ pl.col("date").fill_null(pl.col("date_ap")),
+ pl.col("player_name").fill_null(pl.col("player_name_ap")),
+ pl.col("minutes_played").is_not_null().alias("played"))
+ .with_columns(pl.col(["game_id", "player_id",
+ "club_id"]).cast(pl.Int64),
+ unix_time("date").alias("time"))
+ .sort("game_id", "player_id"))
+ return rows
+
+def bipartite_graph(name, alt, fnames):
+ "The bipartite networks of players and clubs, or players and games."
+ import polars as pl
+ rows = player_rows(name, fnames)
+ pnames = {}
+ for p, n in zip(rows["player_id"], rows["player_name"]):
+ if n:
+ pnames[p] = n
+ players = numpy.unique(rows["player_id"].to_numpy())
+ key = "club_id" if alt == "player_club" else "game_id"
+ others = numpy.unique(rows[key].to_numpy())
+ g = Graph(directed=False)
+ g.add_vertex(len(players) + len(others))
+ rows = rows.with_columns(pl.col("player_id")
+ .replace_strict(players, numpy.arange(len(players)))
+ .alias("u"),
+ pl.col(key)
+ .replace_strict(others, numpy.arange(len(players),
+ len(players) + len(others)))
+ .alias("v"))
+ g.vp.is_player = g.new_vp("bool", vals=[True] * len(players) +
+ [False] * len(others))
+ g.vp.player_id = g.new_vp("int64_t", vals=list(players) + [-1] * len(others))
+ pnames = [pnames.get(p, "") for p in players]
+ if alt == "player_club":
+ ga = read_csv(name, fnames[2])
+ clubs = read_csv(name, fnames[3])
+ nat = read_csv(name, fnames[4])
+ cnames = {}
+ for df, col in [(clubs, "club_id"), (nat, "national_team_id")]:
+ for c, n in zip(df[col], df["name"]):
+ cnames[int(c)] = n
+ for r in ga.sort("date", "game_id").iter_rows(named=True):
+ for side in ["home", "away"]:
+ c = int(r[f"{side}_club_id"])
+ if c not in cnames and r[f"{side}_club_name"]:
+ cnames[c] = r[f"{side}_club_name"]
+ nat_ids = set(int(c) for c in nat["national_team_id"])
+ g.vp.club_id = g.new_vp("int64_t", vals=[-1] * len(players) + list(others))
+ g.vp.name = g.new_vp("string", vals=pnames + [cnames.get(c, "")
+ for c in others])
+ g.vp.national_team = g.new_vp("bool", vals=[False] * len(players) +
+ [c in nat_ids for c in others])
+ pairs = (rows.group_by("u", "v")
+ .agg(pl.len().alias("games"),
+ pl.col("played").sum().alias("games_played"),
+ pl.col("time").min().alias("first_time"),
+ pl.col("time").max().alias("last_time"))
+ .sort("u", "v"))
+ g.add_edge_list(pairs.select("u", "v").to_numpy())
+ for k in ["games", "games_played", "first_time", "last_time"]:
+ g.ep[k] = g.new_ep("int64_t", vals=pairs[k])
+ else:
+ ga = read_csv(name, fnames[2]).with_columns(pl.col("game_id").cast(pl.Int64),
+ unix_time("date").alias("time"))
+ ginfo = {r["game_id"]: r for r in ga.iter_rows(named=True)}
+ g.vp.game_id = g.new_vp("int64_t", vals=[-1] * len(players) + list(others))
+ g.vp.name = g.new_vp("string", vals=pnames + [""] * len(others))
+ g.vp.time = g.new_vp("int64_t", vals=[-1] * len(players) +
+ [ginfo[i]["time"] if i in ginfo else -1
+ for i in others])
+ for k in ["competition_id", "season"]:
+ g.vp[k] = g.new_vp("string", vals=[""] * len(players) +
+ [(ginfo[i][k] or "") if i in ginfo else ""
+ for i in others])
+ g.add_edge_list(rows.select("u", "v").to_numpy())
+ g.ep.club_id = g.new_ep("int64_t", vals=rows["club_id"].fill_null(-1))
+ g.ep.time = g.new_ep("int64_t", vals=rows["time"])
+ for k, c in [("lineup", "type"), ("position", "position"),
+ ("number", "number")]:
+ g.ep[k] = g.new_ep("string", vals=rows[c].fill_null(""))
+ g.ep.captain = g.new_ep("bool", vals=rows["team_captain"].fill_null("0") == "1")
+ g.ep.played = g.new_ep("bool", vals=rows["played"])
+ for k in ["minutes_played", "goals", "assists", "yellow_cards",
+ "red_cards"]:
+ g.ep[k] = g.new_ep("int64_t", vals=rows.select(to_int(k))[k])
+ return g
+
+@cache_network()
+@coerce_props()
+@annotate()
+def parse(alts=None):
+ import polars as pl
+ name = __name__.split(".")[-1]
+ for fnames, alt, fmt in files:
+ if alts is not None and alt not in alts:
+ continue
+ if alt == "transfers":
+ t = read_csv(name, fnames[0])
+ clubs = read_csv(name, fnames[1])
+ comps = dict(zip(clubs["club_id"], clubs["domestic_competition_id"]))
+ # a club can appear under several names (e.g. renamed clubs):
+ # the latest one is used
+ names = {}
+ for r in t.sort("transfer_date").iter_rows(named=True):
+ names[r["from_club_id"]] = r["from_club_name"]
+ names[r["to_club_id"]] = r["to_club_name"]
+ for r in clubs.iter_rows(named=True):
+ names.setdefault(r["club_id"], r["name"])
+ ids = sorted(names, key=int)
+ idx = {c: i for i, c in enumerate(ids)}
+ g = club_graph(True, ids, names, comps)
+ g.add_edge_list(numpy.array([[idx[a] for a in t["from_club_id"]],
+ [idx[b] for b in t["to_club_id"]]]).T)
+ t = t.select(to_int("player_id").alias("player_id"),
+ pl.col("player_name").fill_null(""),
+ unix_time("transfer_date").alias("time"),
+ pl.col("transfer_season").fill_null(""),
+ to_float("transfer_fee").alias("fee"),
+ to_float("market_value_in_eur").alias("market_value"))
+ g.ep.player_id = g.new_ep("int64_t", vals=t["player_id"])
+ g.ep.player_name = g.new_ep("string", vals=t["player_name"])
+ g.ep.time = g.new_ep("int64_t", vals=t["time"])
+ g.ep.season = g.new_ep("string", vals=t["transfer_season"])
+ g.ep.fee = g.new_ep("double", vals=t["fee"])
+ g.ep.market_value = g.new_ep("double", vals=t["market_value"])
+ elif alt == "games":
+ ga = read_csv(name, fnames[0]).sort("date", "game_id")
+ clubs = read_csv(name, fnames[1])
+ nat = read_csv(name, fnames[2])
+ comps = dict(zip(clubs["club_id"], clubs["domestic_competition_id"]))
+ names = {}
+ for r in ga.iter_rows(named=True):
+ for side in ["home", "away"]:
+ if r[f"{side}_club_name"]:
+ names[r[f"{side}_club_id"]] = r[f"{side}_club_name"]
+ else:
+ names.setdefault(r[f"{side}_club_id"], None)
+ for df, col in [(clubs, "club_id"), (nat, "national_team_id")]:
+ for c, n in zip(df[col], df["name"]):
+ if names.get(c, 0) is None:
+ names[c] = n
+ nat_ids = set(nat["national_team_id"])
+ ids = sorted(names, key=int)
+ idx = {c: i for i, c in enumerate(ids)}
+ g = club_graph(True, ids, names, comps)
+ g.vp.national_team = g.new_vp("bool", vals=[i in nat_ids for i in ids])
+ g.add_edge_list(numpy.array([[idx[a] for a in ga["home_club_id"]],
+ [idx[b] for b in ga["away_club_id"]]]).T)
+ g.ep.game_id = g.new_ep("int64_t", vals=ga["game_id"].cast(pl.Int64))
+ g.ep.time = g.new_ep("int64_t", vals=ga.select(unix_time("date"))["date"])
+ for k in ["competition_id", "competition_type", "season", "round",
+ "stadium", "referee"]:
+ g.ep[k] = g.new_ep("string", vals=ga[k].fill_null(""))
+ for k, c in [("home_goals", "home_club_goals"),
+ ("away_goals", "away_club_goals"),
+ ("attendance", "attendance")]:
+ g.ep[k] = g.new_ep("int64_t", vals=ga.select(to_int(c))[c])
+ for k, c in [("home_manager", "home_club_manager_name"),
+ ("away_manager", "away_club_manager_name"),
+ ("home_formation", "home_club_formation"),
+ ("away_formation", "away_club_formation")]:
+ g.ep[k] = g.new_ep("string", vals=ga[c].fill_null(""))
+ elif alt in ("player_club", "player_game"):
+ g = bipartite_graph(name, alt, fnames)
+ else:
+ cols = ["game_id", "club_id", "player_id", "date"]
+ lu = read_csv(name, fnames[0], columns=cols + ["player_name"])
+ ap = read_csv(name, fnames[1],
+ columns=["game_id", "player_club_id", "player_id",
+ "date", "player_name"])
+ ap = ap.rename({"player_club_id": "club_id"})
+ players = read_csv(name, fnames[2])
+ # the players listed for the same club in the same game (in the
+ # starting lineup or on the bench), and those who played
+ rows = (pl.concat([lu.select(cols).with_columns(played=pl.lit(False)),
+ ap.select(cols).with_columns(played=pl.lit(True))])
+ .with_columns(pl.col(["game_id", "club_id",
+ "player_id"]).cast(pl.Int64),
+ unix_time("date").alias("time"))
+ .group_by("game_id", "club_id", "player_id")
+ .agg(pl.col("played").any(), pl.col("time").max()))
+ pnames = {}
+ for df in [lu, ap]:
+ for p, n in zip(df["player_id"].cast(pl.Int64), df["player_name"]):
+ if n:
+ pnames[p] = n
+ del lu, ap
+ prof = {int(r["player_id"]): r for r in players.iter_rows(named=True)}
+ ids = numpy.unique(numpy.concatenate([rows["player_id"].to_numpy(),
+ numpy.array(list(prof),
+ dtype="int64")]))
+ rows = rows.with_columns(pl.col("player_id")
+ .replace_strict(ids, numpy.arange(len(ids)))
+ .alias("v"))
+ rows = rows.select("game_id", "club_id", "v", "played", "time")
+ pairs = (rows.join(rows, on=["game_id", "club_id"], suffix="_2")
+ .filter(pl.col("v") < pl.col("v_2"))
+ .group_by("v", "v_2")
+ .agg(pl.len().alias("games"),
+ (pl.col("played") & pl.col("played_2")).sum()
+ .alias("games_played"),
+ pl.col("time").min().alias("first_time"),
+ pl.col("time").max().alias("last_time"))
+ .sort("v", "v_2"))
+ del rows
+ g = Graph(directed=False)
+ g.add_vertex(len(ids))
+ g.add_edge_list(pairs.select("v", "v_2").to_numpy())
+ for k in ["games", "games_played", "first_time", "last_time"]:
+ g.ep[k] = g.new_ep("int64_t", vals=pairs[k])
+ del pairs
+ g.vp.player_id = g.new_vp("int64_t", vals=ids)
+ g.vp.name = g.new_vp("string", vals=[prof[i]["name"] if i in prof
+ else pnames.get(i, "")
+ for i in ids])
+ g.vp.has_profile = g.new_vp("bool", vals=[i in prof for i in ids])
+ for k in ["date_of_birth", "country_of_birth", "city_of_birth",
+ "country_of_citizenship", "position", "sub_position",
+ "foot", "current_club_name"]:
+ g.vp[k] = g.new_vp("string", vals=[((prof[i][k] or "").strip()
+ if i in prof else "")
+ for i in ids])
+ g.vp.current_club_id = g.new_vp("int64_t", vals=[int(prof[i]["current_club_id"] or -1)
+ if i in prof else -1
+ for i in ids])
+ g.vp.date_of_birth = g.new_vp("string", vals=[x[:10] for x in g.vp.date_of_birth])
+ for k, c in [("height", "height_in_cm"),
+ ("market_value", "market_value_in_eur"),
+ ("highest_market_value", "highest_market_value_in_eur")]:
+ g.vp[k] = g.new_vp("double", vals=[float(prof[i][c]) if i in prof and prof[i][c]
+ else numpy.nan for i in ids])
+ yield alt, g