Workshop series and reproducibility initiative at the AGILE International Conference on Geographic Information Science
© 2024 workshop organization committee.
Theme based on Hyde.
Workshop series and reproducibility initiative at the AGILE International Conference on Geographic Information Science
© 2024 workshop organization committee.
Theme based on Hyde.
The Reproducible AGILE team is happy to contribute to the AGILE conference 2024 by organising the reproducibility review, continueing an exemplary practice in Open Scholarship from the previous years. For the fifth time, a reproducibility review is conducted with all accepted full papers based on the mandatory reproducible paper guidelines.
The reproducibility reviews are published on OSF at 👉 https://osf.io/qvr4s/ 👈.
Reproducibility committee chair Carlos presented an overview on the reproducibility review process and its results at the final day of the conference. The slides are published at https://osf.io/kynu7.
Eight out of 14 accepted full papers were (partially) reproduced. Even though the absolute number of reproductions dropped with respect to previous editions (10 papers in 2023, 16 in 2022), the percentage of reproductions still remained at about 60%. The drop of 10 percentage points compared to the last two editions (70%) can be explained by the high proportion of non-reproducible papers in 2024.
A more extensive written report is published in the document about the reproducibility review process.
Find all details about the conference and the submission guidelines, including information about the reproducibility guidelines and review on the conference website: https://agile-online.org/conference-2024.
If you want to get a quick overview about previous reproducibility reviews, take a look at the presentation by reproducibility committee chair Daniel Nüst at the Munin conference. In short, all non-rejected full paper submissions, i.e., papers that are accepted or have minor revisions only, will be screened for reproducibility based on the information provided in the Data and Software Availability section.
The reproducibility review happens after the scientific review, because some papers are irreproducible for good reason (e.g., conceptual work) and are still valuable scientific contributions. Successfully reproduced papers will receive the Reproducible AGILE badge, see the 2020 proceedings for some examples.
Learn more about the process in the review process and reports document.
‼️ We are looking for more reproducibility reviewers ‼️
If you are interested to learn more about computational reproducibility and contribute to the cultural change in GIScience and in the AGILE community, please nominate yourself to become a reproducibility reviewer by sending an email to Carlos. See below for more information.
