Survey setup

Goal of the survey

The overall goal of the EVERSE project is to evaluate and improve the quality of research software. This can be facilitated by collecting best practices on the one hand, and providing and sharing experiences especially in those areas where quality guidance and support is needed.

To this end, the goal of the survey should lay the ground to

  • Identify communities and resources providing and well applying best practices (lighthouse practices)
  • Identify commonalities throughout the research clusters (common practices)
  • Identify gaps and deficits in software research practices (areas of improvement)

Questionnaire setup

Respondents

For the initial landscaping, the research areas and practices should be covered as widely as possible to be able to identify the areas of improvements. This will, due to limited responses, increase the inaccuracy of the survey. On the other hand, allowing more specific responses from experts in the fields would help to identify lighthouse practices.

Therefore, the respondents are asked to distinguish between two answering modes:

  1. Reporting on their own practices, or their group and institution’s practices (single view),
  2. Assuming a general view on the research field they represent and estimating from best knowledge how the practices in the research field are established (representative view).

While everyone can answer the survey from a single view, a dedicated list of respondents is requested to fill the survey from a representative view, as well as specific experts. This information is captured in the first part (P).

Software levels

In order to differentiate practices at different levels of software applicability, three software levels are introduced (analysis code, prototype tools, research software infrastructure). Participants are asked to exemplify these software levels for their area and give descriptions (D) and evaluate practices then according to these levels (L1-L3).

Sections

Elements of the survey are structured according to the section and functionality:

  • Personal information: used to categorize the answers and evaluate the profile of the survey participant
  • Software level evaluation: evaluation of the applicability of software levels
  • Software levels: categorization per software level (L1 - L3)
  • Software description: description of the software per level
  • Guidelines: Guidelines and financing of quality practices
  • Technical software quality practices: elements of technical software practices
  • Management practices: aspects of software project management and incentivisation

Filling the survey

The survey was open for the collection of answers in August and September 2024. It is still available, and additional results can serve to enhance the outcomes presented here.