Reviewer Guidelines


An Overview of ing.grid’s Open Peer Review: Reviewer Perspective

Once the submission passes an initial editor screening, the editors publish it on the preprint server and the peer review process begins. ing.grid uses an open peer review process in which authors and reviewers communicate in public comments which are moderated by the editors.

As a reviewer, you should:

  • Review the assigned submission within the amount of time specified by the editor (4 weeks)
  • Prepare a review comment addressing Quality Requirements and Formal Requirements specified below. Beside a thorough review, suggest one of the following: revise/accept/reject.
  • Submit your review in the form of comments via the link you have received in the e-mail invitation. You may choose to remain anonymous.
  • Once all reviews have been submitted, the editor will make the reviews public. The authors as well as other visitors of the ing.grid preprints website will be able to see your review.
  • The editor will make the decision whether the paper needs to be revised or whether it is accepted/rejected.
    • In case the submission needs to be revised, the authors will submit a new version of the submission or update the software and/or dataset accordingly. They will also respond to your review comments. You will be invited for another round of peer review to check whether your concerns have been addressed.
    • If the submission is accepted/rejected, no action is needed from you as the reviewer. The editors will pass the submission to the ing.grid journal for copyediting and typesetting before it is ultimately published as an article.

The Reviewer Guidelines are structured as follows:

  • Quality Requirements and Formal Requirements contain requirements that hold for all submission types.
  • Review Requirements for Manuscript Submission
  • Review Requirements for Software Submission
  • Review Requirements for Dataset Submission

Quality Requirements

As the reviewer, you need to check whether the submission fulfills the following quality requirements:

  • The novelty and originality of the content presented in each submission is clear.
  • The work is clearly placed in the state of the art with reference to relevant literature
  • The presented work is the result of substantial scholarly effort.
  • FAIR Principles: The submission aligns with the FAIR principles and meets domain-relevant community standards of FAIR data. The minimal requirements for adherence to FAIR principles for ing.grid are listed below in the relevant sections.

Formal Requirements

As a reviewer, you must ensure that the following formal requirements are met.

Authorship

Format

  • The submission is of appropriate length. Recommended maximum manuscript length is 25 pages.

Graphs, Images and Tables

  • If possible, colour images are accessible to all, including those with impaired colour vision.

Language, Units and Abbreviations

  • The submission is well-written. It uses a consistent English variation in the entire text. The language is grammatically and orthographically correct. The text is structured in a coherent way, allowing the reader to follow easily.
  • Ideally, authors make efforts to use inclusive language.
  • All units in the manuscript follow the international system of units (SI) or an SI equivalent is given where it was necessary for authors to use different units.
  • All abbreviations used in the manuscript or descriptor are either defined by authors or can be considered common knowledge.

References

  • Every reference is cited in the text using a numeric referencing style such as IEEE.
  • Data and preprint references are also acceptable in submissions to ing.grid.

Availability of Supplementary Material

  • Links to the software or dataset repositories of software and dataset submissions lead to the correct material. Links to other supplementary material are also functional.

Manuscript Submission Requirements

Overview

ing.grid is open to manuscript submissions that include but are not limited to the following:

  • FAIR data management methods for various domains of engineering sciences
  • education concepts for FAIR data management all stages of an engineering career
  • best practice examples of FAIR data management methods, workflows and operationalisation
  • tutorials related to FAIR data management
  • collaboration and interdisciplinary research
  • Discussions of the seven core aspects of FAIR data management defined in the scope.

Manuscript Requirements

All manuscripts must contain:

  • Title
  • Author list (including ORCIDs, if existing)
  • Affiliations (including PIDs, where possible)
  • Abstract
  • Keywords
  • Theoretical background
  • State of the art
  • Methods (if applicable, methods related to creating data/software)
  • Results
  • Discussion
  • Conclusion
  • Code and/or Software availability (if applicable)
  • Acknowledgements
  • Author Contributions
  • Conflicts of interest
  • Funding sources (optional)
  • References

Furthermore, the following points should be addressed in the manuscript:

  • Authors are strongly encouraged to provide supplementary material, especially software and data. If a full version of utilised research software or datasets is available as supplementary material, authors should ensure findability and accessibility of this material by providing links or DOIs to the repositories where the software or dataset is deposited.
  • Whenever applicable, manuscripts should include minimal examples of utilised research software and sample dataset or course material which should be submitted as supplementary material.
  • When providing supplementary material, authors should orient themselves on Software and Dataset Submission Guidelines which are designed to help ensure findability and accessibility. Descriptors for supplementary material must not be provided, but the respective relevant points should be addressed in the manuscript if applicable.

Software Review Requirements

Overview

Software submissions consist of a software descriptor and the software itself with an optionally included dataset as supplementary material. The software descriptor is a written document describing the software. It is this document that authors submit to ing.grid through the submission form. The descriptor must contain a link to the software repository.

ing.grid publishes software descriptors about research data management software. This definition includes all software with a clear relation to research data management (RDM). Software submissions may either be dedicated tools for supporting scientists in good RDM practice or general research software which also addresses specific RDM issues.

This includes but is not limited to software that:

  • solves complex modelling problems in an engineering context;
  • supports functioning of research instruments or the execution of research experiments;
  • extracts knowledge from large datasets

Reviewers must ensure that the following requirements for both the software itself and the software descriptor are met.

Software Requirements

Software published in inggrid follows the FAIR Principles for Research Software:

  • Findable: Software, and its associated metadata, is easy for both humans and machines to find
  • Accessible: Software, and its metadata, is retrievable via standardized protocols
  • Interoperable: Software interoperates with other software by exchanging data and/or metadata, and/or through interaction via application programming interfaces (APIs), described through standards
  • Reusable: Software is both usable (can be executed) and reusable (can be understood, modified, built upon, or incorporated into other software)

As a reviewer, you are to check whether the dataset aligns with them by reviewing the following minimal criteria:

Accessibility and Findability:

  • The software code is deposited to a repository. The code is open to be accessed, viewed, browsed and cloned anonymously. The software can be retrieved under its URI via download, cloning or similar. (F, A)
  • Retrieval must be possible without using a proprietary protocol (e.g. download wizard) (A)

Licensing:

  • Software submissions does not depend on proprietary software. (R)
  • The software is licensed under an Open Source license as defined by the OSI. The repository contains a plain text licensing file. It should also include copyright and licensing statements of any third-party software that are legally bundled with the code in compliance with the conditions of those licenses. (R)

Documentation:

  • The repository should contain a README file or similar documentation explaining installation procedure, system requirements as well as purpose, scope and usage. The installation procedure should be automated as far as possible, ideally by a package management system. (R)
  • Installation instructions clearly and comprehensively detail all required dependencies, including required version numbers. Additionally, repositories should make use of language and domain standard automated dependency management tooling. For example: Python applications should use required.txt or a common virtual environment definition, node applications should include a package.json file, C++applications should include a CMakeList.txt or makefile. Similarly, approaches such as the use of docker deployments are also expected. (I, R)
  • The software should be described by metadata using a formal, accessible, shared, and broadly applicable language for knowledge representation (cf. https: //codemeta.github.io/). Metadata clearly and explicitly include the identifier of the software they describe (F, R)
  • A working example of the software is included. Required minimal example data should be included in the software repository. Ideally, a reference to a data repository using a DOI is provided in which users can find a dataset relevant to the software as supplementary material. (R)

Maintenance:

  • Instructions on how to contribute to the project should be included in the repository. This may be as minimal as submitting an issue or as broad as providing instructions on how to set up a development environment, what coding styles are used, whether automated tests should be run or written or which other minimal testing process is used, what the code architecture is, etc. (R)

Software Quality:

  • The software code complies with formal standards regarding commenting, formatting conventions, coding standards and structure. It is easily readable and makes use of community or language-accepted common code structures. (I, R)
  • The repository should use version control. (F, R)
  • The repository should allow issue tracking and allow submission of issues. (R)
  • The software adopts a testing method, preferably by automated tests, though a clearly described testing procedure is also acceptable. (R)

Software descriptor requirements

All software descriptors must contain:

  • Title
  • Author list (including PIDs)
  • Affiliations (including PIDs, where possible)
  • Abstract
  • Keywords
  • Software code identified by title, creator, publisher and location (e.g. URL), version number
  • Software license (an open source license is mandatory)
  • State of the art and novelty of the approach
  • Statement of need
  • Purpose and scope/limits of the application
  • Utility of the software
  • User notes
  • Sample data availability and reference to related datasets
  • Acknowledgements
  • Author Contributions
  • Conflicts of interest
  • Funding sources (optional)
  • References

Furthermore, the following points should be addressed in the software descriptor:

  • Findability of software: The software descriptor contains a link to the repository where the software is deposited and a version ID.
  • Authorship: The authorship of the software should be consistent with the authorship of the submitted software descriptor. If there are differences, authors have clarified these in the descriptor.
  • Novelty: The authors are expected to provide a technical analysis evaluating the source code via: a discussion of how the software compares to other similar and related alternative software in terms of scope and provided functionality; commentary on how near or far the source code is to meeting domain and language known best practices such as adherence to coding styles, package management, documentation, and use of automated testing, collaboration, and deployment infrastructure and tools; optionally, a demonstration of the software's performance numerically via standardized benchmarking techniques or otherwise relevant quantifiable metrics.
  • A statement of need should clearly indicate the utility of the software to the community and the software should have a clear (research) data management application. Readers should be able to recognise the relevance of the software for supporting them in their daily challenges of data management and research. Thus, a statement of need, the application and a user story should be part of the submitted software descriptor.
  • Software Documentation: Software architecture and usage should be sufficiently described.

Dataset Review Requirements

Overview

Data submissions consist of a data descriptor and a dataset with an optionally included software repository as supplementary material. The data descriptor is a written document describing the dataset. It is this document that the authors submit to ing.grid through the submission form. The descriptor must contain a link to the dataset.

ing.grid publishes data descriptors about datasets relevant to an engineering or data management context. The datasets should themselves be examples of good (research) data management (RDM) practice.

Datasets may be for example:

  • experimental data (images, laboratory measurements, …)
  • simulation data
  • geospatial data
  • field data
  • survey results

Reviewers must ensure that the following requirements for the data itself as well as for the data descriptor are met:

Data requirements

Data published in inggrid follows the FAIR principles:

  • Findable: Data, and its associated metadata, is easy for both humans and machines to find
  • Accessible: Data, and its metadata, is retrievable via standardised protocols
  • Interoperable: The data need to interoperate with applications or workflows for analysis, storage, and processing
  • Reusable: Data is both usable and reusable

As a reviewer, you are to check whether the dataset aligns with the following minimal criteria:

Accessibility and findability:

  • The data is deposited in a repository that offers a DOI (digital object identifier) as unique and persistent identifier and that is listed in the index re3data. (F, A)
  • The dataset is available in open access, with an option for reviewers to access it anonymously. (A)

Documentation:

  • The repository should, if possible, contain a README file or similar documentation explaining purpose, scope and usage.
  • All necessary information required for replicating the measurements, reproducing the figures or other representations is provided in the dataset or descriptor.
  • The dataset should be described by metadata using a formal, accessible, shared, and broadly applicable language for knowledge representation. (I)
  • Metadata contains the DOI of the dataset (F)
  • If possible, (meta)data should use standardised vocabularies (I)

Licensing:

  • The repository provides a clear and accessible data usage license. It is recommended that authors use the Creative Commons Attribution (CC-BY 4.0) license. (R)

Quality:

  • The dataset should use a format that is commonly used in the respective community and is non-proprietary. (R)
  • The content of the dataset complies with formal standards and conventions for example regarding data structure and uncertainty quantification. The data must be logically and consistently organised as well as being easily readable and usable. (R)

Dataset descriptor requirements

All dataset descriptors contain:

  • Title
  • Author list (including PIDs)
  • Affiliations (including PIDs where possible)
  • Abstract
  • Keywords
  • Dataset identified by title, DOI, creator, publisher and location (e.g. URL), publication date, subject field, version
  • Dataset license (an open access license is mandatory)
  • State of the art and novelty of the approach
  • Description of the content of the dataset
  • Data Collection or Generating Methods
  • Description of Employed Tools
  • Technical validation
  • Quality evaluation
  • Usage notes
  • Code availability with link and license
  • Acknowledgements
  • Author Contributions
  • Conflicts of interest
  • Funding sources (optional)
  • References

Furthermore, the following points should be addressed in the dataset descriptor:

  • Dataset findability: The dataset is referenced in the data descriptor using the DOI.
  • Authorship: The authorship of the data should be consistent with the authorship of the submitted software descriptor. If there are differences, authors have clarified these.
  • Supplementary material should include a minimal software example to illustrate the data set's generation and/or processing.
  • Novelty: The utility of the data set to the community is sufficiently described. Readers should be able to recognise the relevance of the data set for the field. In order for the data to be reusable, the methods are made as transparent as possible and the precision of the data needs to be quantified using domain relevant measures of uncertainty.
  • Documentation: Methods of data collection/generation should be sufficiently described. The novelty of the approach and the scholarly effort of gathering the data is made clear. Both hardware and software tools applied need to be specified appropriately (including calibration, code and suitable controls). Estimation of errors or uncertainty is disclaimed, and common standards for comparison are used.
  • Reproducibility: All necessary information required for replicating the measurements, reproducing the figures or other representations is provided in the dataset or descriptor.
  • Ideally, a minimal example for using the dataset is provided. Authors are strongly encouraged to provide a reference to a software repository using a link and version ID.