Wikipedia Teahouse pilot helps new editors make contributions

It seems like just yesterday that the Teahouse first opened its doors on English Wikipedia. For the past 3 months, experienced Wikipedians have been welcoming new editors into a space designed to promote community building and support interaction, a space where newbies can get answers from seasoned editors for all their editing questions, great and small. Teahouse guests are enjoying this new opportunity to learn to do things the Wikipedia way, and some early data suggests that they may be going on to edit Wikipedia more often as a result.
As the Teahouse pilot period wraps up and we look to phase 2 of the project, the project team has published a pilot report and metrics containing all findings from the pilot period. We are excited to share some highlights from what we’ve learned so far!

Participation

During the 13-week pilot, 586 volunteers participated in the Teahouse. A small group of dedicated Teahouse hosts and other experienced Wikipedians sent more than 7000 invitations to new editors, 400 of whom dropped in for a virtual cup of tea and sought answers to their questions about editing. Over 500 questions were answered and more than 200 guests introduced themselves in the Teahouse’s guestbook.
Overall, 70 percent of participants we surveyed — both new and experienced editors — indicated satisfaction with their Teahouse experience, and only 5 percent were dissatisfied. Some of our favorite comments from new editors: “the Teahouse is what changed me from being a gawker into being a (very newbie) editor” and “Teahouse overall is very inviting and proves to be quite useful.” As a result of the “helpful”, “friendly” and “quick” answers that new editors receive in the Teahouse (the median response time for questions is 29 minutes), 25 percent of Teahouse guests become repeat customers.
Experienced editors, too, have found participation in the project rewarding. Some have commented on the power of Teahouse’s social support model: “I liked that the Teahouse is a ground for new users so they have lots of support, and that hosts and new users interact a lot with each other,” said one experienced editor.
Ryan Vesey, a Wikipedian with over 10,000 edits, said, “I think the Teahouse is the best thing I have seen happen to the encyclopedia since I started editing.”
Another Teahouse host, NtheP, noted that Teahouse has things to teach experienced participants too: “The other big plus for me has been how much I’ve learned either from others answers or in researching answers myself.”
Host Rosiestep hopes the project will have long-term impact: “I feel proud to be associated with the Teahouse pilot project; talk page comments from new editors abound with notes about how they appreciate the service provided. If just one of those folks becomes prolific as a result of the Teahouse encounter, then job well done here.”

Impact

Among several goals for the pilot, we wanted to test whether offering a social support model to new editors helped to engage and retain more editors, and if new models like this one might have some impact on Wikipedia’s gender gap. Teahouse does appear to be an effective strategy to help meet the challenge of retaining editors in Wikipedia’s complex culture.
In order to determine whether participating in Teahouse has a positive impact on an editors likelihood of continuing to edit Wikipedia or on their editing behaviors, we compared the editing behaviors of a sample of 200 new editors who participate in the Teahouse with 2 similar control groups who did not participate: a sample of new editors who met the basic criteria for being invited but who were not (Control A) and a sample of new editors who were invited to participate but did not show up (Control B). We found that our sample of Teahouse guests edited 5-10x the number of articles as the members of the control groups, and made 2.5-8x the number of total edits across all namespaces. We also found a 1.5-2x increase in the volume of article content created by Teahouse guests that had survived through the end of the pilot compared to new editors in our control groups. Furthermore, 33 percent of the Teahouse participants were still making edits 2 weeks after the end of the pilot period, compared to 9 percent and 11 percent for the two control groups; of those editors who were still active, Teahouse guests averaged between 6x and 2x as many edits as members of Control A and B, respectively.
We also found that 28 percent of Teahouse participants are women, up from 9 percent of editors on Wikipedia in general. With lots more work to do on the gender gap, Teahouse does appear to be one strategy we can work on to encourage and help retain more female editors.
All of these numbers are good news for those of us looking for new ways to support, engage, and retain more new editors on Wikipedia. Although this analysis only measures short-term retention, it is an encouraging sign given that most editors who stop editing very early in their Wikipedia careers do not edit again. We intend to measure retention of members of these groups at 3, 6, and 9 months to gain a better understanding of the impact of Teahouse on long term editor retention.

The Future of the Teahouse

With Wikipedia’s Teahouse, we have demonstrated the potential to positively impact the new editor experience, retain and incorporate more editors into the community, and complement other approaches to editor support in the Wikipedia ecosystem. A successfully completed pilot is not the end of the Teahouse story, though. There is still a lot of work to be done in order to take the space from a pilot to a scalable and sustainable project, and additional features and systems need to be built to ensure that Teahouse will continue to be of value to Wikipedians for years to come. WMF will continue to partner with the community to help take Teahouse to the next level.
We’ll keep you updated as the Teahouse team begins work on phase 2 of the project.  Meanwhile, come stop by for a cup of wiki-tea at the Teahouse anytime!
Siko Bouterse, Head of Community Fellowships Program
Sarah Stierch, Community Fellow