logo Netzschleuder network catalog, repository, and centrifuge

History of enron_emails_cmu

The changes to the source code of this entry in the repository, newest first, each with the differences it made to the entry's files. They are also listed on GitLab.

[entries] Add the date when each entry was added, and remove icon_hash by Tiago de Paula Peixoto beff7c9
The dates are those of the commits that first added the entries' files
(2020-08-11 for the entries of the initial commit); renamed entries keep
the date of their first name.

Changed: none
diff --git a/entries/enron_emails_cmu/__init__.py b/entries/enron_emails_cmu/__init__.py
index ab54402..d8a98df 100644
--- a/entries/enron_emails_cmu/__init__.py
+++ b/entries/enron_emails_cmu/__init__.py
@@ -35,7 +35,7 @@ The network has the following properties:
 tags = ['Social', 'Communication', 'Unweighted', 'Timestamps', 'Metadata']
 url = 'https://www.cs.cmu.edu/~enron/'
 citation = [('B. Klimt, Y. Yang, "The Enron Corpus: A New Dataset for Email Classification Research." Machine Learning: ECML 2004, Lecture Notes in Computer Science 3201, 217-226 (2004)', 'https://doi.org/10.1007/978-3-540-30115-8_22')]
-icon_hash = None
+date_added = '2026-10-25'
 upstream_license = None
 upstream_prefix = 'https://www.cs.cmu.edu/~enron'
 files = [('enron_mail_20150507.tar.gz', None, None)]
[entries] Prefix the properties in the descriptions with (Node), (Edge) or (Graph) by Tiago de Paula Peixoto bc3dbbd
The lists of properties at the end of the descriptions marked edge
properties with "Edge" but left node properties unmarked, or grouped
them in "Nodes: `a`, ...; `b`, ..." items, so the kind of a property
was often unclear. Every item now starts with (Node), (Edge) or (Graph),
followed by the property names, a colon and the explanation, one item
per property or group of properties; qualifications (kinds of nodes,
networks) follow the names in parentheses. Properties that were only
named got a short explanation, and a few items that named the wrong
kind of property were corrected.
diff --git a/entries/enron_emails_cmu/__init__.py b/entries/enron_emails_cmu/__init__.py
index 82c14bb..ab54402 100644
--- a/entries/enron_emails_cmu/__init__.py
+++ b/entries/enron_emails_cmu/__init__.py
@@ -28,10 +28,10 @@ description = """A bipartite network of email addresses and the emails in the En
 
 The network has the following properties:
 
-- `is_email`, whether the node is an email (otherwise an email address).
-- Address nodes: `name`, the email address (in lower case).
-- Email nodes: `time`, the date of the email from its Date header field, in Unix time (seconds since 1970-01-01 UTC; -1 if missing or invalid).
-- Edges: `field`, the header field in which the address appears: `from` (the sender), `to`, `cc` or `bcc`. If an address appears in more than one field of the same email, only the first of these is used, in this order. In this corpus every address in a Bcc field is also in the Cc field of the same message, so no edge has the value `bcc`."""
+- (Node) `is_email`: whether the node is an email (otherwise an email address).
+- (Node) `name` (addresses): the email address (in lower case).
+- (Node) `time` (emails): the date of the email from its Date header field, in Unix time (seconds since 1970-01-01 UTC; -1 if missing or invalid).
+- (Edge) `field`: the header field in which the address appears: `from` (the sender), `to`, `cc` or `bcc`. If an address appears in more than one field of the same email, only the first of these is used, in this order. In this corpus every address in a Bcc field is also in the Cc field of the same message, so no edge has the value `bcc`."""
 tags = ['Social', 'Communication', 'Unweighted', 'Timestamps', 'Metadata']
 url = 'https://www.cs.cmu.edu/~enron/'
 citation = [('B. Klimt, Y. Yang, "The Enron Corpus: A New Dataset for Email Classification Research." Machine Learning: ECML 2004, Lecture Notes in Computer Science 3201, 217-226 (2004)', 'https://doi.org/10.1007/978-3-540-30115-8_22')]
[entries] Remove unneeded code from the entries by Tiago de Paula Peixoto 2c69f19
Unneeded "global files" statements, unused imports and variables,
f-strings without placeholders, duplicate keys in foodweb_little_rock
(keeping the order and the values used), and the second definition of
get_lines() in twitter_higgs. Nothing changes in the networks or
descriptions.

Changed: none
diff --git a/entries/enron_emails_cmu/__init__.py b/entries/enron_emails_cmu/__init__.py
index 7f0c908..82c14bb 100644
--- a/entries/enron_emails_cmu/__init__.py
+++ b/entries/enron_emails_cmu/__init__.py
@@ -58,7 +58,6 @@ def parse_addresses(vals):
 @coerce_props()
 @annotate()
 def parse(alts=None):
-    global files
     name = __name__.split(".")[-1]
     hparser = email.parser.BytesParser(policy=email.policy.compat32)
     for fname, alt, fmt in files:
[entries] New entry: enron_emails_cmu by Tiago de Paula Peixoto 19288e7
diff --git a/entries/enron_emails_cmu/__init__.py b/entries/enron_emails_cmu/__init__.py
new file mode 100644
index 0000000..7f0c908
--- /dev/null
+++ b/entries/enron_emails_cmu/__init__.py
@@ -0,0 +1,128 @@
+#! /usr/bin/env python
+# -*- coding: utf-8 -*-
+#
+# Copyright (C) 2020-2026 Tiago de Paula Peixoto <tiago@skewed.de>
+#
+# This program is free software: you can redistribute it and/or modify it under
+# the terms of the GNU Affero General Public License as published by the Free
+# Software Foundation, either version 3 of the License, or (at your option) any
+# later version.
+#
+# This program is distributed in the hope that it will be useful, but WITHOUT
+# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
+# FOR A PARTICULAR PURPOSE.  See the GNU Affero General Public License for more
+# details.
+#
+# You should have received a copy of the GNU Affero General Public License along
+# with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+from .. import *
+import tarfile
+import hashlib
+import email.parser
+import email.policy
+import email.utils
+
+title = "Enron email corpus (1998-2002)"
+description = """A bipartite network of email addresses and the emails in the Enron email corpus, as distributed by William Cohen at Carnegie Mellon University (version of May 7, 2015). The corpus contains about 0.5 million messages from the mailboxes of about 150 Enron employees, mostly senior management, organized into folders. The messages were made public by the US Federal Energy Regulatory Commission during its investigation of the company, and were prepared by the CALO project, with integrity problems corrected at SRI; attachments are not included, and some messages were deleted at the request of affected employees. Invalid recipient addresses were converted to the form user@enron.com where possible, and to no_address@enron.com when no recipient was specified. Since the same email is often stored in several folders and mailboxes, the messages were deduplicated: messages with identical date, sender, recipients, subject and body are considered the same email. Each email is connected to its sender and to all the addresses in its To, Cc and Bcc header fields, which include addresses outside Enron. Almost all emails are dated between 1999 and 2002; 483 have dates before 1998 or after 2002 (292 of them in 1980), which are mostly invalid. Messages without any sender or recipient address are not included.
+
+The network has the following properties:
+
+- `is_email`, whether the node is an email (otherwise an email address).
+- Address nodes: `name`, the email address (in lower case).
+- Email nodes: `time`, the date of the email from its Date header field, in Unix time (seconds since 1970-01-01 UTC; -1 if missing or invalid).
+- Edges: `field`, the header field in which the address appears: `from` (the sender), `to`, `cc` or `bcc`. If an address appears in more than one field of the same email, only the first of these is used, in this order. In this corpus every address in a Bcc field is also in the Cc field of the same message, so no edge has the value `bcc`."""
+tags = ['Social', 'Communication', 'Unweighted', 'Timestamps', 'Metadata']
+url = 'https://www.cs.cmu.edu/~enron/'
+citation = [('B. Klimt, Y. Yang, "The Enron Corpus: A New Dataset for Email Classification Research." Machine Learning: ECML 2004, Lecture Notes in Computer Science 3201, 217-226 (2004)', 'https://doi.org/10.1007/978-3-540-30115-8_22')]
+icon_hash = None
+upstream_license = None
+upstream_prefix = 'https://www.cs.cmu.edu/~enron'
+files = [('enron_mail_20150507.tar.gz', None, None)]
+
+def fetch_upstream(force=False):
+    return fetch_upstream_files(__name__.split(".")[-1], upstream_prefix, files,
+                                force)
+
+FIELDS = ["from", "to", "cc", "bcc"]
+
+def parse_addresses(vals):
+    addrs = []
+    for name, addr in email.utils.getaddresses([str(v) for v in vals]):
+        addr = addr.strip().strip("<>'\"").lower()
+        if addr != "":
+            addrs.append(addr)
+    return addrs
+
+@cache_network()
+@coerce_props()
+@annotate()
+def parse(alts=None):
+    global files
+    name = __name__.split(".")[-1]
+    hparser = email.parser.BytesParser(policy=email.policy.compat32)
+    for fname, alt, fmt in files:
+        if alts is not None and alt not in alts:
+            continue
+        seen = set()
+        n_files = 0
+        addr_idx = {}
+        times = []
+        egroups = []
+        emembers = []
+        efields = []
+        with open_upstream_file(name, fname, "rb") as f:
+            with tarfile.open(fileobj=f, mode="r|") as tf:
+                for m in tf:
+                    if not m.isfile():
+                        continue
+                    n_files += 1
+                    data = tf.extractfile(m).read()
+                    msg = hparser.parsebytes(data, headersonly=True)
+                    body = msg.get_payload()
+                    if isinstance(body, str):
+                        body = body.encode("utf8", "surrogateescape")
+                    else:
+                        body = b""
+                    addrs = [parse_addresses(msg.get_all(k, []))
+                             for k in FIELDS]
+                    date = msg.get("Date", "")
+                    key = hashlib.sha1(repr((date, addrs,
+                                             str(msg.get("Subject", "")))).encode("utf8", "surrogateescape")
+                                       + body).digest()
+                    if key in seen:
+                        continue
+                    seen.add(key)
+                    if not any(addrs):
+                        continue        # no sender or recipient
+                    try:
+                        t = int(email.utils.parsedate_to_datetime(str(date)).timestamp())
+                    except (TypeError, ValueError, IndexError, OverflowError):
+                        t = -1
+                    i = len(times)
+                    times.append(t)
+                    done = set()
+                    for k, field in enumerate(addrs):
+                        for a in field:
+                            if a in done:
+                                continue
+                            done.add(a)
+                            egroups.append(i)
+                            emembers.append(addr_idx.setdefault(a, len(addr_idx)))
+                            efields.append(k)
+        N = len(addr_idx)
+        g = Graph(directed=False)
+        g.add_vertex(N + len(times))
+        g.vp.is_email = g.new_vp("bool")
+        g.vp.is_email.a[N:] = True
+        g.vp.name = g.new_vp("string")
+        for a, v in addr_idx.items():
+            g.vp.name[v] = a
+        g.vp.time = g.new_vp("int64_t", val=-1)
+        g.vp.time.a[N:] = times
+        g.add_edge_list(numpy.column_stack((numpy.array(emembers, dtype="int64"),
+                                            N + numpy.array(egroups, dtype="int64"))))
+        g.ep.field = g.new_ep("string")
+        for e in g.edges():
+            g.ep.field[e] = FIELDS[efields[g.edge_index[e]]]
+        yield alt, g