How We Judge: Building Trust in Wikimedia Campaign Evaluation
Speakers
Dev Jadiya (User:Dev Jadiya) from India maintains WhichTool, clip2commons and WikiScriptSync on Toolforge, and co-maintains Superset on Wikimedia Cloud VPS.
He is Liaison to the Volunteer Supporters Network, South Asia Outreach Lead for the WikiVibrance Project, and a core team member of WikiConference India 2026 and the South Asia Newsletter. He founded Wiki Open Learning and mentors in the EduWiki Hub Mentorship Programme.
He has served on five campaign juries across three continents - the International Juries for Wiki Loves Monuments 2025 and Wiki Loves Folklore 2026, national juries in India and the Philippines, and Wiki Science Competition 2025 in India, which he led.
Abstract
Thousands of images are submitted each year to Wikimedia photography campaigns such as Wiki Loves Monuments, Wiki Loves Folklore, Wiki Loves Birds, and Wiki Science Competition. While the final shortlisted images are publicly celebrated, the evaluation systems behind them are rarely discussed in detail. Yet judging frameworks directly influence contributor trust, fairness, and long-term campaign credibility.
Drawing from experience as an International Jury member for Wiki Loves Monuments and National Jury member for Wiki Loves Folklore (India) and Wiki Loves Birds (Philippines), as well as leading Wiki Science Competition 2025 in India, this session examines how judging structures operate across local and international contexts. It reflects on scoring models, shortlisting workflows, conflict-of-interest safeguards, documentation practices, and reviewer onboarding processes.
Inconsistent evaluation criteria, unclear eligibility rules, and undocumented decision-making can unintentionally create opacity or perceived bias. Even when decisions are fair, lack of structured transparency can weaken trust. This lightning talk identifies practical, implementable improvements — including standardized scoring templates, clearer licensing and eligibility documentation, reviewer training guidance, and transparent shortlisting workflows.
Rather than focusing on participation numbers or campaign promotion, this session centers on governance architecture: how evaluation systems shape freedom, equity, and reliability within Wikimedia campaigns. By strengthening judging transparency and documentation practices, communities can improve credibility, reduce ambiguity, and support more equitable participation across regions and experience levels.
Additional information
Wikimedia campaigns rely on volunteer trust. Transparent evaluation systems protect contributor freedom by ensuring submissions are judged consistently and without informal influence. Equity is strengthened when criteria are clearly documented and applied uniformly across regions. Structured review workflows reduce dependence on insider knowledge and create fair participation environments. Reliability emerges when judging systems are well-documented, conflict-of-interest safeguards are respected, and scoring processes are consistent. Transparent evaluation increases long-term confidence in Wikimedia competitions. Reliable judging systems directly affect long-term contributor trust and campaign sustainability.
• Photography campaign organizers • Affiliate leaders and program coordinators • Jury members and prospective jurors • Contributors participating in competitions • Governance and movement strategy contributors • Commons community members This session is particularly useful for communities organizing local or national campaigns who seek to improve evaluation transparency and contributor trust.
Everyone can participate in this session