Thank you for your feedback and comments about the usability beta through the survey and via the usability project wiki. The usability beta initially started off with the new skin called Vector and the toolbar (release nickname: Acai) in early July and it was enhanced with the navigable table of contents within the editing box and dialogues for links, tables, and search & replace (release nickname: Babaco) at the end of September.
The Babaco features have to be activated through the user preferences. The most recent enhancement was to move the tool for watching and unwatching pages into the top navigation (star icon). This feature was implemented based on a high volume of related user feedback. Navigation tabs now collapse in case tabs start overlapping when the screen resolution is reduced, or where tab widths are wider because of language specific characters.
The usability beta was visited and tested by close to 380,000 users by the end of November 2009. The beta has been drawing roughly 100,000 users every month, and close to 300,000 users have kept the beta enabled as of December 1, 2009. As I briefly summarized in my blog post in September, the beta program was adopted relatively well by the beta users of English Wikipedia (83% retention rate), and in other English language projects such as the English Wikinews (95% retention rate). Spanish and Portuguese Wikipedia beta users have the second highest retention rate at 81%. German, Russian, Chinese, French and Italian Wikipedia beta users are retained in the range between 70% and 79%. Retention rate for Polish and Japanese was relatively low, with 65% and 60% respectively.
The list of the beta retention rates for the top ten most visited Wikipedia languages is summarized in this table and the list of all projects in all languages can be found here. The opt-out survey and feedback forms were available to beta users and the survey responses provided rich quantitative and qualitative data to understand how the beta is received and understand the reasons for what people like about the beta and leaving the beta.
One of the questions we sought to answer via the opt-out surveys was the impact of different browser types. The usability tech team has been pounding on Internet Explorer related problems and we thought the browser type will have significant impact on the beta adoption pattern. According to our own browser statistics, about 56% of users access Wikipedia through a version of Microsoft Internet Explorer, and about 30% of users use a version of Mozilla Firefox. The browser distribution pattern for the beta users for the ten mostly visited Wikipedia language family indicates stronger favor towards Firefox over Internet Explorer except for Japanese and Chinese Wikipedia whose browser distribution is dominated by Internet Explorer instead of Firefox. If we slice the beta retention rate by browser distribution, we see the the repeated low retention rate for Internet Explorer users. However the variance of retention rates by browser types is not significant enough to conclude that browser distribution has a substantial impact on the beta adoption.
The qualitative survey responses brought up language-specific issues. Japanese Wikipedia beta users found the font used in the new interface is too small. Chinese beta users also expressed the difficulty adjusting to the smaller font in addition to the perceived slowness of the new interface. Switching between Traditional and Simplified Chinese was not well supported. We hope issues with overlapping tabs expressed by German users have been addressed by the recently released collapsible tabs. Finally, we are aware that we often break popular user gadgets, and we appreciate that developer communities gradually embrace the usability updates so that the gadgets are compatible with the usability beta. The survey comments are available in ten languages so far, and we will continue making the surveys available for more languages.
The usability beta continues to evolve and the usability team is actively working on the next release, Citron. This release will have the new features such as collapsed templates, form interface for editing templates, and side-by-side preview. Citron is currently scheduled to be available in January. Beta feedback will be incorporated in this release to address language specific issues which were surfaced from the survey.
As the general acceptance of the usability beta is supported by a majority of beta users, we would like to start discussing the timing of making the new editing interface and the toolbar the default interface after resolving any remaining language specific issues.
This survey data was analyzed and put together by the team effort. I would like to thank Howie Fung, Product Consultant, for organizing and analyzing the survey data, Nimish Gautam, for integrating machine translation into the qualitative survey, and Roan Kattouw for tracking and normalizing the data.
Naoko Komura, Program Manager, Usability Initiative
Can you help us translate this article?
In order for this article to reach as many people as possible we would like your help. Can you translate this article to get the message out?
Start translation