Workshop 'Reaping the benefits of open and reproducible research' @ AGILE'26

About

AGILE has made significant steps towards more open and reproducible research (ORR) publications: Reproducible publications at the conference or reproducibility education at PhD schools help to improve the quality of the scientific contributions from and for the community. It is now time to start reaping the benefits of ORR publications by increasing the uptake of results and building on published research outputs to create a collaborative ecosystem of active, iterative exchange. The participants of the workshop “Reproducible Research” at AGILE 2026 conference on June 16 will take the first pioneering steps towards this goal.

One core objective of the workshop is to increase the practice of advanced open research practices and explore the replication of published AGILE research papers. We define replication as keeping the overall research question of the study but changing either (a) input data (e.g., a different geographic area of study) to see if the analysis outcomes are replicable, or (b) method (e.g., a significantly different clustering or machine learning algorithm) to see if the analysis is robust.

To kick-start the practical replications, this call solicits suggestions for studies to reproduce from you. During the second half of the workshop, you pitch your replication or robustness check proposal to the workshop participants. We then form groups to begin work on a preregistration for a study. After the workshop, the groups continue with the replication study and submit at a suitable journal. We are in discussion with several journals to ensure that workshop outputs fit their scope.

We aim for a maximum of 6 pitches. If we receive more than 6 proposals for a replication study, the workshop organizers will select 6 pitches based on the following criteria:

• Relevance: Is the proposed replication study relevant to the AGILE community and scientifically sound?

• Suitability: Is the scope for a replication clearly defined?

• Feasibility: Can the replication be conducted with the available/suggested resources?

Should we still have more than 6 submissions after applying these criteria, contributions that propose to replicate studies published in the AGILE proceedings will be given priority.

What should your proposal for a replication study look like?

1. Only one proposal per participant
2. Up to two pages, including figures and references.
3. Outline of the proposed replication study
4. Short justification using the criteria mentioned above
5. Assuming that you want to play a key role in the replication study: What required skills do you bring, and what other skills does the team need?

Please submit as PDF via e-mail to f.o.ostermann@utwente.nl by 11 May 2026. We will notify you about acceptance by 22 May.

Objective(s)

1. To determine whether further improvements or extensions to the AGILE reproducibility guidelines and reviews of AGILE proceedings are necessary, desirable, and feasible. And if so, to develop concrete actions on how to do so.

2. To determine strategies and develop concrete action items on how to increase the uptake of open research practices beyond citations (e.g., replications) for AGILE research papers in the conference proceedings and beyond.

Planned outcome(s)

1. A short white paper on the future of ORR at AGILE.

2. Action plan for reproducing and replicating one or several AGILE full papers.

Worhshop format

The workshop is open for all researchers, whether they have a lot of experience in open and reproducible research practices, or none at all. We will start with a 15 minute introduction to bring everyone to the same page, before participants can share their experiences with ORR in general and at AGILE in particular, first by filling in a short, structured survey, before engaging in open discussion based on the findings. We will work collaboratively on a white paper that addresses objective 1, before moving on to objective 2. Here we invite participants to pitch for a paper to replicate. The workshop organizers will provide at least one suggestion as well. Based on these suggestions, we form groups of interested participants to brainstorm about research designs for the replication studies. We end the workshop with forming one or more groups of researchers willing to go forward with one or more such replication studies, similar to https://doi.org/10.1080/15230406.2025.2591124, aiming to publish them within a year (ready for next AGILE conference).

The program of the half-day workshop will be:

• 30 minutes introduction / setting the scene / participant survey
• 60 minutes of writing the short white paper / position statement

Short break followed by

• 15 minutes (total) of up to 6 pitches for replication studies
• 60 minutes group work on research design around the pitched studies
• 15 minutes wrap-up and moving forward