Equity Landscape Data: Community Pilot Results & Next Steps

The Global Data & Insights team’s Equity Landscape metrics project completed the first pilot cohort for data use and consultation in Feb-March 2022.

The purpose of the pilot and consultation was to gather feedback and suggestions from community use case participants either reflecting on the project and metrics design, or directly piloting the dashboard design options and metrics to understand their geographic space in terms of movement engagement signals.

The Pilot

The first cohort of the pilot launched in February and had nine participants who joined under a non-disclosure agreement (NDA) as one of the targeted community audiences. After an initial doodle poll to arrive at a common meeting time, two office hour slots were selected based on the time preference indicated by the participants. The virtual office hour sessions were hosted on Google Meet. The first sessions were held with two options in February, the first on a Friday from 4-5 PM UTC, with seven participants attending. The second, on a Tuesday from 5-6 PM UTC, with five participants attending.

The initial pilot session provided a basic overview of the pilot and data privacy boundaries, as well as a demonstration on how to navigate the data of the equity landscape within the dashboard testing space. The following sessions provided more demonstrations and specific walk-throughs with the data to answer key use case questions which had been selected as most relevant by the pilot cohort:

1. What countries may be underrepresented in the grants or affiliates ecosystem? 
2. How many other affiliates may be competing for resources in this same space? 
3. Which countries have a strong affiliate presence and reader base but are lacking in editorship? 
4. To what extent does a country engage in various languages and projects? 
5. Does a country have a well-balanced movement organizer ecosystem to support an international event or extensive collaborative partnership?  

All demonstrations were recorded and made available to those who could not attend the sessions live. Upon the group’s request, we continued to host optional office hours every two to three weeks for technical support and for pilot participants to share back their experiences. We also asked pilot participants to alert us to bugs in the data, and help us to understand barriers within their user experience which would be used to improve our tooling design to ensure accessibility.

The Outcome

Throughout the pilot sessions, community participants provided input and feedback via email and direct talk page comments to the ten Directed Review Questions and Design Considerations. We captured and analyzed roughly 29 points of input from the meta portal comments from 5 community commenters who were actively involved as well as participant feedback gathered during the course of our first pilot sessions with 9 community participants.

The results have been analyzed and posted on Meta-Wiki. For the most part, we must continue to pilot and consult more closely over the coming months with some of our key movement organizing and governance bodies. We must ensure further dialogue on both the metrics design, as well as connect for future sense-making around the metrics.

Challenges & Barriers

The pilot process revealed important challenges that we encountered as a team. We faced time-zone challenges as we tried to find a common time for the monthly office hours. Given the community participants were geographically dispersed spanning from Latin America to Eastern Asia, we could not find a single time that worked for all. Language barrier was another challenge. The office hour sessions were hosted in English and all communications were drafted in English, and some participants faced difficulty in understanding and interpreting the purpose of the sessions due to the technical language required to discuss the Equity Landscape. Lastly, connectivity issues also brought challenges during calls where internet bandwidth could be overwhelmed when attempting to run both the video call and the Google data studio at the same time.

What Next?

We are looking to continue with the pilot period. We continue to monitor the talk page for new pilot volunteers and we plan to present some targeted demonstration workshops in July to preview the metrics to key use cases. 

We are also exploring translation support for key documentation and working to connect with key stakeholder groups who would benefit from a guided workshop and demo session.

If you are part of a strategic Wikimedia organizing or governance group interested in attending a demonstration workshop (1) watch for announcements and/or (2) reach out on our talk page if you have a specific workshop request!