A comprehensive evaluation of Wikimedia contributors for recognition

Translate this post

KLRWP Awardees who were felicitated during Telugu Wiki decennial celebrations on 16-17 Feb 2014 (from L to R: Veeven, Sasi, Ravichandra, JVRK Prasad, Ahmed Nisar, Palagiri, & Bhaskarnaidu ; Chaduvari, M Pradeep, and Chava kiran could not make it to the function.) Photo: Pavan Santhosh S

The Telugu Wikipedia (Tewiki) community has recognized prolific contributors to Wikimedia projects with a special award called “Komarraju Lakshmana Rao Wikimedia Puraskaram” (KLRWP) on the tenth anniversary of Telugu Wikipedia on December 10th, 2013. The award is named after Komarraju Lakshmana Rao, the first modern encyclopedist in Telugu, perhaps the first among Indian languages (c.1912-13). It is a fitting tribute to honor Lakshmana Rao in the centennial anniversary of such a pioneering effort. Each award carried a citation and cash prize of 10,000 Indian Rupees (equaling about 166 USD as of April 2014). This blog post gives details about the process, its implementation and lessons learned from it.
Recognition on wiki projects is usually done by awarding barnstars to editors by peer contributors; when an editor crosses a significant edit count or makes substantial contributions to a project. Other approaches that have been tried include gift coupons for books, travel and accommodation, scholarships for events, and cash prizes for winning essays by students. There have been no monetary prizes for the people who have been contributing, for several years. WMDE started Zedler-Preis für Freies Wissen (Zedler Prize for Free Knowledge) in 2012. The award categories include best article, best community project and best external project. The awards are in the form of scholarships for events or in cash for external entities.
For the Telugu Wikipedia community the disadvantages that were observed with the traditional approaches were as follows: The recognition process was dependent on a single person as opposed to a group. This led to an opinion that recognitions are dependent on number of edits, partiality, and incomplete quantitative analysis of all factors. Person dependency also led to a delay in recognizing the right candidates. Competitions only attracted new people who most often did not turn out to be active editors. Very few Wikimedians are getting recognized with scholarships for Wikimania or national conferences. Competitions also have a tendency to make people support fellow Wikimedians less.

During the WikiConference India 2011, a simple process was pioneered by WMIN cofounder and EC member Arunram for Noteworthy Wikimedian Recognition. This consisted of a simple proposal with a few points about editor contributions, which was deliberated by a committee. The winners were recognized in the conference and an electronic barnstar was uploaded on their talk pages. We built upon this process by making the proposal more detailed and the evaluation more exhaustive.
A selection council of volunteer Wikipedians who had achieved some recognition in the past were constituted with the basic premise that they will not be eligible for the award. A template which allowed for a comprehensive view of various contributions like edits, images, policy development, projects, and outreach, including contributions to sister language wiki projects, was prepared and reviewed. The evaluation model was also developed with guidelines for minimum and maximum scores, with appropriate variation for specific Wikimedia project like Wiktionary and Wikisource. An example proposal was also provided using the contributions of Vyzasatya, a key and long time Telugu Wikipedian, who was on the team. Criticism from some members about the proposal and evaluation were addressed and the framework was improved where possible. Rules were developed for nominations, which could be done by the candidates or their peers. Endorsements of the candidature were also provided for in the nomination template.
Wide publicity was given in the village pumps, mailing lists and social media channels regarding the awards. For the benefit of the wider Wikimedia community, we have provided relevant remarks on the nomination pages in English and allowed nominations in Telugu or English.
After a candidate was proposed by his or her peers, the nominee’s acceptance of the nomination was sought, as some of the Wikimedians may not be interested in participating. We got 28 applications for the awards, out of which 18 were complete based on the acceptance criteria outlined in the initial announcement. Since many nominations did not provide enough detail or statistics, the selection council prepared a document with detailed statistics of contributions under each namespace along with relevant charts.
The selection council analyzed all proposals based on quantitative metrics like wiki statistics and qualitative assessment of each nominee’s contribution and its impact on growth and development. Factors considered in the judging process include:

  1. Number of edits in article name space
  2. Files contribution
  3. Help documentation
  4. Template, MediaWiki, Bot
  5. WikiProjects, Support to wiki development
  6. Support to peer Wikimedians, wiki behavior
  7. Online publicity
  8. Physical publicity, participating in physical meetings
  9. Awareness & following the right wiki process
  10. Non Telugu wiki projects contribution

The council held nine virtual meetings from the time of formation of the council until the finalization of the award. Various tools like Google docs, wiki statistics (for gathering user’s edit counts, for graphics), desktop sharing and Skype for conference calls were used.
As this is the debut year for the awards, the selection council decided to award 10 recipients, recognizing their contributions during the first decade of Telugu Wikipedia and its sister projects. In the order of final assessment score, the awardees are Chaduvari, M Pradeep, Chava kiran, Veeven, Palagiri, Ravichandra, Ahmed Nisar, Veera.sj, JVRK PRASAD and Bhaskaranaidu. They were presented with cash prizes and certificates during the Telugu Wiki Ten Celebrations on February 16th and 17th, 2014. Pictures, and citation remarks in Telugu and English were prepared and shared widely.
‌Feedback was obtained from the participants and the community regarding the process. The process was appreciated for its comprehensiveness in evaluation of contributions, and transparency in its development and implementation. The selection council also became aware of the practices that aided or hindered the growth of wikis. The former were highlighted in the analysis of each candidate’s contributions. These included breadth and depth of contribution, knowledge of wiki policies and courteousness in online discussion. Those that hindered growth included contributions without understanding the scope of wiki project, rude behavior in online discussions, violation of copyright, and conflicts of interest. These were consolidated as a group and shared with the community.
It was an arduous effort by the selection council members, Ravi Vyzasathya, Arjuna Rao Chavala, A. Rajasekhar, T. Sujatha and Radhakrishna Arvapally. The council thanks Arjuna Rao Chavala, selection council secretary for taking responsibility for this initiative. We also thank the Wikimedia Foundation for providing the necessary funds to support this award and the Tewiki 10 Celebrations committee for their encouragement and support. We would like to thank all the people who participated in the nomination and proposal development process and also those who helped to improve the process by their feedback during its roll out.
Authors: Radhakrishna Arvapally, Arjuna Rao Chavala

Archive notice: This is an archived post from blog.wikimedia.org, which operated under different editorial and content guidelines than Diff.

Can you help us translate this article?

In order for this article to reach as many people as possible we would like your help. Can you translate this article to get the message out?

1 Comment
Inline Feedbacks
View all comments

Thanks for the post! The kind of competition and process is very similar to the Wikimedia Italia award 2009. 🙂 http://wiki.wikimedia.it/wiki/Wikimedia_news/numero_25/en#What_happened_last_month We always rely on juries of experienced editors for evaluation, as long as the members know how to work with each other it’s even a pleasurable job. The WMIT award was focused on added value the award could get for Wikimedia; for “ordinary” wiki work during across the year, the it.wiki community also has also a tradition of an annual community-attributed barnstar nomination and poll. This is easy to start on any wiki without money or time investment. 😉… Read more »