Enticing Wikipedians back… with bacn

500gfpx-Bacon
Sorry, we don’t mean this kind of bacon.

Bacn. If you have an account on any of the most popular websites on the Internet, you’ve probably seen it – it’s the Internet jargon for those periodic emails you receive whenever your activity on a website wanes, reminding you to come back and see what you’ve missed. There’s a Wikipedia article about it, naturally.
Unlike many other major community sites, Wikipedia doesn’t send any unsolicited reminders to its editors… but what if it did? What if a large chunk of the recent decline in editor numbers is simply due to people not being actively encouraged to come back and contribute again, like they are on virtually all other websites where people freely give their time and effort?

Our experiment

To test this hypothesis, we here at the editor engagement experiments team decided to send some bacn to former highly active Wikipedians who had stopped editing. We dubbed this experiment Necromancy. (Hat tip on the name goes to Jeff Atwood of StackExchange fame, who introduced us to Stackoverflow’s necromancy badge for reviving dead topics.)
Though we chose not to go with our data analyst’s literal necromancy-themed email idea of encouraging lapsed users to “fight the dark armies of ignorance once more!”, we did do our best to sound friendly and informal in these messages, and we addressed them from members of our team who have Wikipedia editing experience. While many Wikipedia users may be accustomed to receiving emails from automated mailer systems, we figured that sending from a person who could be directly replied to would increase our chances of not being marked as spam.
Since there is no single metric for when a Wikipedian has finally left the project, we contacted three different types of editors: those who hadn’t edited articles for one year, a second group that had been gone for three months, and a third that stopped editing for just 30 days.

The results

We had some pretty interesting results from our three rounds of email. The percentage of people who opened our emails was about 27-28 percent, slightly higher than the standard figure cited by most non-profits, about 20 percent, and definitely higher than the 5 percent open rate of marketing/sales emails. The number of people who clicked through to the login screen was about half that, and in our most successful round, we generated a return-to-editing rate of 5 percent.
Though the open rates stayed more or less the same among the 1-year, 3-month, and 1-month lapsed editors, rates of return to editing were higher for those who had been gone for less time. Wikipedians who haven’t edited for a year are probably gone for good, but those who were only gone for a month may just need a small reminder to jump back in again.

Conclusions and future work

So, what does this mean for editor retention? Well, we’re probably not going to start sending everyone who has ever made an edit to Wikipedia daily reminders to contribute. But, if we can expect that one email to recently lapsed editors will bring back about 5 percent and get them to edit again, and we can predict from current editor trends that about 5,000 Wikipedians will stop editing over the course of this year, this means that a little bit of bacn will prevent 250 of those Wikipedians from disappearing for good. It may not be the silver bullet that reverses editor decline, but it’s a pretty good start.
Another step toward encouraging editors to return via email has already been enabled. Unrelated to our tests, community members have worked to ensure that an editor on any Wikimedia project can opt in to receiving emails when a page on their watchlist is changed.
If you’d like to know more about our current and planned experiments, check out our documentation on Meta, feel free to comment below, or ping us on our freenode IRC channel – #wikimedia-e3
Maryana Pinchuk and Steven Walling,
on behalf of the Editor Engagement Experiments team at the Wikimedia Foundation

† As reported by the 2012 eNonprofit Benchmarks Study.

Archive notice: This is an archived post from blog.wikimedia.org, which operated under different editorial and content guidelines than Diff.

9 Comments
Inline Feedbacks
View all comments

don’t know why I ever write an answer to these requests for an answer, the odd chance that anyone will read it is probably infinitesimal. But here are some thoughts which I think will help: 1. Why on earth is there no Facebook like/share buttons on the wiki topic pages? this would drive traffic and increase the engagement, ultimately resulting in more editors. 2. Scrap the wiki editor, or at least bring it up to a currentday standard, with true wysiwyg, inline dropdowns and other software guided text entry stuff which would decrease the learning curve, and alleviate up front… Read more »

+1 to Carsten’s proposal

> 2. Scrap the wiki editor, or at least bring it up to a currentday standard
Efforts are underway: http://www.economist.com/blogs/babbage/2011/12/changes-wikipedia
Re share/like buttons: if you can find a way to do it in a way that doesn’t compromise privacy and that the community can agree on, then I don’t know why you couldn’t fix it by changing a template or two.

there are some academic papers out there that support the idea that giving people reminders to contribute will get them to contribute, though it’s not necessarily clear whether the contributions will be sustained over time. i’m glad to see the effect somewhat observed on wikipedia too. what also apparently helps is if the emails have specific contribution goals or suggestions (such as “we are trying to get X more articles written this week) or if people are compared with others (such as “editors like you have edited Wikipedia 10 times this week and you’ve edited 6 times”). nice job guys!… Read more »

>”The percentage of people who opened our emails was about 27-28 percent” How was that tracked? web bugs in email, token in link that gets recorded when people request the page. /me thinks web bugs if used would be on the questionable side. Just a token in the link is probably ok I guess – I know people need/like analytics, but I’m not really a fan of tracking in general unless widely discussed. To be totally honest, I don’t really think this is an appropriate use of wikipedians email addresses unless the individual opted in, or there was a largish… Read more »

Hi bawolff: On the privacy issue… we checked with legal first, naturally, but the privacy policy is extremely clear: “The email address put into one’s user preferences may be used by the Foundation for communication.” We only emailed editors who have opted in to email though, so these are people that could be emailed at any time through the Special:Emailuser function. If you’re interested in how we sent and tracked these emails, there is a lot of detail on Meta documentation page. To give you a short answer: we used the Foundation’s installation of CiviCRM, which does track open rates… Read more »

To be a proper scientific experement you should have similar groups to whom you don’t send an e-mail. How many would have continued after 30 days inactivity?

Very true, HenkvD. We did indeed have similarly-sized control groups (who received no email) for each set of emails sent, and a small percentage of users did return independently. If you’re interested, you can find more details about our methodology and results on the Meta page.

Actually, you might have a much better success rate if you tried to find why people actually left in the first place and then possibly attempt to fix it. Sure, that would take lot more manual labor, but it might result in a better working environment for us all. For starters, there could be a little questionnaire about the causes of leaving for former active Wikipedians (which those who would take a wikivacation for a specific period could just turn off).