Reproducible Research @ AGILE '23

The Reproducible AGILE team is happy to contribute to the AGILE conference 2023 by organising the reproducibility review, continueing an exemplary practice in Open Scholarship from the previous years. For the fourth time, a reproducibility review is conducted with all accepted full papers based on the mandatory reproducible paper guidelines.

The reproducibility reviews are published on OSF at 👉 https://osf.io/2k56f/ 👈.

Report

Reproducible AGILE community member Carlos presented an overview on the reproducibility review process and its results on behalf of reproducibility committee chair Daniel Nüst at the final day of the conference. The slides are published at https://osf.io/7ugm6.

10 papers of 14 accepted manuscripts were successfully (partially) reproduced. This corresponds to 71% percent of accepted papers.

A more extensive written report is published in the document about the reproducibility review process.

Review details

Find all details about the conference and the submission guidelines, including information about the reproducibility guidelines and review on the conference website: https://agile-online.org/conference-2023.

If you want to get a quick overview about previous reproducibility reviews, take a look at the presentation by reproducibility committee chair Daniel Nüst at the Munin conference. All non-rejected submissions, i.e., papers that are accepted or have minor revisions only, will be screened for reproducibility based on the information provided in the Data and Software Availability section.

The reproducibility review happens after the scientific review, because some papers are irreproducible for good reason (e.g., conceptual work) and are still published. Successfully reproduced papers will receive the Reproducible AGILE badge, see the 2020 proceedings for some examples.

Learn more about the process in the review process and reports document.

Reproducibility committee

• Daniel Nüst, Co-Chair (TU Dresden, Germany), daniel.nuest@tu-dresden.de
• Carlos Granell, Co-Chair (Universitat of Jaume I, Spain), carlos.granell@uji.es
• Eftychia Koukouraki (University of Münster, Germany)
• Jakub Krukar (University of Münster, Germany)
• Philipp Friese (Technical University of Munich, Germany)
• Rémy Decoupes (INRAE / UMR TETIS, Montpellier, France)
• Arun Sharma ( University of Minnesota, USA)
• Nina Wiedemann (ETH Zürich, Switzerland)
• Mehtab Alam Syed (CIRAD / UMR TETIS, Montpellier, France)
• Alexander Kmoch (University of Tartu, Estonia)