A bit of background
In April of 2014 I found myself digging deep into analytics in search of possible improvements and insight into what we do as a chapter. What brought me there? One of our most renowned programs, Wikimedians in Residence. A Wikimedian in Residence (WIR) is a person who, as a Wikimedia contributor, accepts a placement within an institution to facilitate open knowledge in a close working relationship between the Wikimedia movement and the institution. They work to facilitate content improvements on Wikimedia projects, but more importantly serve as ambassadors for open knowledge within the host organization.
Wikimedia UK has been involved with WiR in the UK with varying degrees of support and supervision. Since the creation of the chapter, we always felt that the program was worth running, seeing it as one of the key ways we can engage with external organizations. However, I never knew for sure, if that was just a feeling. Toward the end of 2013 we decided to explore these notions.
Why and how to evaluate
As I focused on my questions about program impact, I embarked on a review process of the program, which eventually included: a questionnaire for all the key parties, online surveys, meetings, group discussions, the analysis of existing materials (e.g. residents’ reports) and creation of a review document.
In January of this year I planned to survey the Residents and host institutions about their views on the program. Since I wasn’t sure what to ask, I reached out to the Program Evaluation and Design team for help.
Their stringent approach was worth it. We boiled down the issues around what I actually wanted to find out from the survey. Doing that before creating the questions was a revelation to me. The questionnaire went much deeper than I had originally anticipated. This meant that when we worked on creating the survey questions, every point was there for a specific reason and in a sensible order. With their help, I developed three surveys: for residents, residency hosts and another for community member input.
I was impressed with the amount of feedback that was shared. The Residents were clearly committed to the project and keen on telling me what could make the program more successful. At the same time I ran interviews with the host institutions. By that stage I was deeply entrenched in the review process. Discovering more about the program increased my appetite for a deeper analysis. This culminated in an April brainstorming meeting aimed at completing an analysis of the strengths, weaknesses, opportunities and threats (SWOT analysis) of our Wikipedian in Residence program.
With the data collection completed, I then examined all the reports and case studies produced by the residents and summarized them in terms of the impact made to Wikimedia projects. (Click here to read Overview of the residencies.)
Lessons learned
After running the program for a long time, one may assume they know everything about it. I was surprised to hear from many Residents that it often took them a couple of months to fully understand what their role within the host organization was. I had assumed that they would have connected with one another to share the resources they created without my help, but this was not the case.
Before doing this research, I did not appreciate how important it is to have a ‘team’ within the institution working with the residents. Having a line manager and/or senior staff support seemed to be one of the main reasons some residencies were more successful than others.
With the data pulled from the report, the program evaluation and design team helped in the elaboration of an infograph (see below). This resource seeks to showcase the numbers behind the program. How do the efforts of the wikimedians in residence impact the Wikimedia projects? Overall, Wikimedia UK invested only 30% of the total cost to fund in-house residents over the course of their term. Each residency is singular, with variations from one to the next, however they also have many points in common. Take a look and follow the colors to single out residencies. The graphics are not exact but an approximation, due to gaps in reporting. If you would like to add more data to these graphics, please email eval@wikimedia.org.
Looking Ahead: An improved WiR program
The aim of the review was to assess the program, focusing on the feedback of successful models for the residencies and analyzing key obstacles to greater success. Six months later, with some volunteer support, I finished a review report. (Click here to read the summary). What I appreciated most about this project was how I was able to analyze an existing program and see how it could run better, rather than stopping it and trying something completely new. Innovation is usually expected to arise from brand new initiatives, but I found it motivating and useful to find novelty looking deep into WiR.
- Duration of residencies – residencies should be longer to ensure impact (e.g. 9-12 months for larger organizations)
- Project goals – should be clearer for each residency to improve assessing impact. They should be reflected on the job description. Better reporting should follow.
- Sharing of information – set up a forum for the sharing of advice, information and best practice between institutions and between residents.
- Supporting the program – additional capacity is needed for supporting the residents and the program. This will be considered in the future.
References
- Final review
- Check out the surveys I used:
- Watch the video of the Survey Strategies virtual meet-up, where I share reflections and commentary on the process and what I learnt from the survey process on a recorded hangout:
Daria Cybulska, Wikimedia UK
Can you help us translate this article?
In order for this article to reach as many people as possible we would like your help. Can you translate this article to get the message out?
Start translation