Internal security incident identified and resolved at the Wikimedia Foundation

Love padlocks
A security incident with the Wikimedia Foundation’s Mailman mailing list system was identified and addressed today. Photo by Petar Milošević, freely licensed under CC by-SA 4.0.
On November 12, the Wikimedia Operations team identified a security incident on the Wikimedia Foundation’s Mailman mailing list system that resulted in the breach of four staff email accounts. We immediately investigated the incident, addressed the underlying vulnerabilities, and took steps to remedy the situation.
To our knowledge, the affected accounts have now been secured, and the security incident has been resolved. As part of our commitment to transparency, we are sharing an overview of this incident and how we responded.
How did this happen?
An account with legitimate access to the server hosting our mailing list system obtained passwords from configuration files. A number of those passwords were then tested against staff email accounts and matched in four cases.
What has been done to fix it?
We immediately locked the four affected staff accounts, changed affected passwords, and applied additional security measures. We also locked the account believed to have been behind the breach and have terminated all future access from that account to internal systems. At this time, we have no evidence of other production services being impacted. Out of an abundance of caution, we are in the process of regenerating all passwords stored by our mailing list system. If you use your Mailman password for other accounts, we recommend that you change your password for those accounts.
The Wikimedia Foundation takes the privacy of staff and users very seriously. We will continue to monitor our systems and implement additional security measures to prevent this from happening again.
Mark Bergsma, Director of Technical Operations*
Michelle Paulson, Legal Director*
Wikimedia Foundation

*We would like to thank the various teams, including Ops, Performance, Communications, Legal, Office IT, and Community Advocacy, that worked together throughout the day to expeditiously investigate and resolve this issue.

Archive notice: This is an archived post from blog.wikimedia.org, which operated under different editorial and content guidelines than Diff.

6 Comments
Inline Feedbacks
View all comments

Talking of Mailman, it has keeps emailing my password in cleartext. Not when I ask for it either — it just does it! PLEASE PLEASE disable this function for all users. It is terrible, terrible security practice.
I know you’re meant to be able to use it exclusively over the email protocol but I think that demographic is something of a fantasy. Do it over HTTP, keep uses safe. For real.

@yannanth: “Disabling this function for all users” requires Mailman version 3 if I understand https://bugs.launchpad.net/mailman/+bug/265179 correctly. Wikimedia’s upgrade to version 3 is planned in https://phabricator.wikimedia.org/T52864

The advice at the end seems more to downplay the behaviour of the WMF staffers than to increase security. You should *never* use the same password for different services. Mailman by default generates a random password for you. You have to actively override that to create a risk.

Could someone please explain why passwords were in configuration files in the clear?
Assuming that a configuration file has to contain a password, why wasn’t a new one generated for that config file alone to use?
If it had been set to something like JxT!A#itAuK#ud9q=ao@rm2!ag+puC=5 it wouldn’t have matched with any existing password on any system anywhere.

Guy Macon, mailman 2 stores passwords (not just irreversible hashes thereof) in order to send reminders. See yannanth’s and aklapper’s comments for details.
Other discussions (partly offtopic): http://thread.gmane.org/gmane.org.wikimedia.listadmins/1 http://thread.gmane.org/gmane.org.wikimedia.listadmins/5

You should *never* use the same password for different services