Infrastructure support
Infrastructure support
Infrastructure for Planning
| Description | Provider | What is missing | |
|---|---|---|---|
| 1 | I am aware of ad-hoc support within time-limited grants, e.g. Eu grant “ATRIUM” provides support for DH researchers to visit one of the partners and receive targeted training for their use case. But it is limited within the project, it is not a long-term solution. | Institute or Organization | |
| 7 | Argos an open extensible service - available as a standalone service (OpenDMP) and as a OpenAIRE service (ARGOS) - that simplifies the management, validation, monitoring and maintenance of Data Management Plans. It allows actors (researchers, managers, supervisors etc) to create actionable DMPs that may be freely exchanged among infrastructures for carrying out specific aspects of the Data management process in accordance with the intentions and commitment of Data owners. | Institute or Organization | |
| https://argos.openaire.eu/splash/about/how-it-works.html | |||
| 9 | At the Netherlands eScience Center we do both Planning and Development. | Institute or Organization | We are currently considering adding support for just the planning phase. Usually, we only do planning in order to do development, but there may be demand for only supporting planning. |
| 11 | GitHub, GitLab, Miro | Commercial; Open/free platform; Institute or Organization | |
| 12 | GitHub projects https://docs.github.com/en/issues/planning-and-tracking-with-projects/learning-about-projects/best-practices-for-projects | Commercial; Open/free platform | This platform meets the requirements for maintaining and planning Open Source projects |
| 17 | Institute or Organization | ||
| 18 | - github.com | Open/free platform |
Infrastructure for Development
| Description | Provider | What is missing | |
|---|---|---|---|
| 2 | GitHub https://github.com | Commercial; Open/free platform; Institute or Organization | |
| GitLab https://gitlab.com | |||
| BSC Gitlab instance https://gitlab.bsc.es | |||
| 3 | For the code development using large datasets it is dire need of storage but also computing power (eg. GPU), this is the reason to use Instritute’s server in order to be able to analyse this heavy analysis. | Institute or Organization | There is not always enough storage or GPU available. |
| 6 | software repositories gitlab.cern.ch which are tighly bundled with storage and compute infrastructure e.g. at CERN in addition to other repositories e.g. github.com | Open/free platform; Institute or Organization | |
| CERN “jira” instance for issue tracking | |||
| document servers | |||
| 8 | Git repositories, also hosted specifically for the collaboration: https://git.km3net.de | Institute or Organization | specific introduction to software development for early career scientists if they have to do this as part of their research. |
| 9 | At the Netherlands eScience Center we do both Planning and Development. | Institute or Organization | |
| 10 | Many instances of institutional gitlab or the central GitHub/Gitlab platform (gitlab.com GitHub.com) are used for software development, see e.g.: | Commercial; Open/free platform; Institute or Organization; Large scale infrastructure | There is no combined discussion and development platform - issues are sometimes used for discussions and further planning. |
| https://gitlab.in2p3.fr/ | |||
| Many large infrastructure even have their own it services, see e.g. https://information-technology.web.cern.ch/services | |||
| 11 | GitHub, GitLab | Commercial; Open/free platform; Institute or Organization | |
| 12 | GitHub/GitLab | Commercial; Open/free platform | No |
| 13 | Code repositories (GitHub, GitLab,…) | Commercial; Open/free platform; Institute or Organization | |
| 15 | self-hosted gitlab | Commercial; Open/free platform; Institute or Organization; Large scale infrastructure | |
| github | |||
| gitlab | |||
| 17 | Institute or Organization | ||
| 19 | Does github qualify? Version control is one dimension of quality so are some of the security analyses/scans. With actions you can build CI/CD pipelines, have code coverage reports generated, run linters, package and push to package registries… | Commercial | |
| Some of those features would also find under Maintenance | |||
| 22 | The core development team provides support to anyone willing to contribute to the project in terms of resolving issues that the individual contributor may not be privy to given the size of the whole project. There are regular developer workshops held by the core dev team which contain demos and usage guidelines for new contributors. The core team also lists a number of examples on usage and development of the project that are easy to replicate and follow. | Institute or Organization | The project does not provide compute resources to contributors. Having state of the art compute resources (eg. dedicated nodes at a HPC) can help accelerate the decelopment process by having a standardized test suite for new algorithms without requiring developers to create local builds of the project which almost always causes user-hardware related dependency issues. |
| 25 | repositories, debuggers, containers, linting tools, profilers | Open/free platform; Institute or Organization | |
| 26 | https://github.com/ | Institute or Organization; Other | |
| https://posit.co/ | |||
| 27 | https://codebase.helmholtz.cloud, https://helmholtz.software, https://www.hifis.net | Open/free platform; Institute or Organization; Large scale infrastructure | a more professional software project management |
Infrastructure for Maintenance
| Description | Provider | What is missing | |
|---|---|---|---|
| 2 | GitHub issues and CI | Commercial; Open/free platform; Institute or Organization; Large scale infrastructure | There are limitations related to number of CI jobs which can be run, which is a problem for researchers who do not have proper hardware resources and setup in house to check for correctness, issues, hazards, etc… |
| GitLab issues and CI | |||
| Several software package infrastructures, like npmjs.com , pypi.org , bioconda , biocontainers or similar (helpful to widespread new releases) | |||
| 8 | git repository | Institute or Organization | |
| 9 | NLeSC also does a little maintenance, but not as a primary core activity. | Institute or Organization | University partners regularly lament the great lack of funding for maintenance. |
| 10 | Integrated CI/CD platforms are used on all levels - usually based on the development platforms listed above. | Commercial; Open/free platform; Institute or Organization; Large scale infrastructure | Structural maintenance of software is hard to achieve as it usually needs a transfer of responsibility from the initial (group of) developers to the maintainers - which mostly are not identical. So far, there is no clear route of hand over, especially if the initial developer (group) is not reachable (e.g. left the field). Clear guidelines are missing here. |
| 11 | GitHub, GitLab | Commercial; Open/free platform; Institute or Organization | |
| 12 | GitHub through actions | Commercial; Open/free platform | GitHub actions have a number of free uses per month. While one may connect webhooks, it is not straightforward (for non computer scientists) to delegate the execution to external servers |
| 15 | gitlab CI | Open/free platform; Institute or Organization | |
| github actions | |||
| mybinder https://mybinder.org/ | |||
| 17 | Institute or Organization | ||
| 22 | Maintenance is done by the core team using git versioning and standard CI/CD pipelines | Open/free platform | Given the size of the project the team is doing their best to maintain the project efficiently. |
| 25 | repositories | Open/free platform; Institute or Organization | |
| 26 | https://github.com/ | Institute or Organization; Other | |
| https://posit.co/ | |||
| 27 | https://codebase.helmholtz.cloud | Open/free platform; Institute or Organization; Large scale infrastructure | no |
Infrastructure for Archiving
| Description | Provider | What is missing | |
|---|---|---|---|
| 2 | Zenodo https://zenodo.org | Commercial; Open/free platform; Institute or Organization; Large scale infrastructure | There are limitations about the size of what it is |
| B2SHARE https://b2share.eudat.eu | |||
| osf.io https://osf.io | |||
| Software Heritage https://www.softwareheritage.org/ | |||
| 4 | Data repository used by public universities in the region of Madrid, Spain. | Other | We use a data repository as software repository. |
| https://edatos.consorciomadrono.es/ | |||
| 8 | Zenodo: https://zenodo.org/ | Large scale infrastructure | |
| 9 | - DANS: https://dans.knaw.nl/nl/ | Open/free platform; Institute or Organization | |
| - Zenodo, of course. | |||
| 10 | OSSR is one infrastructure, based on Zenodo: https://zenodo.org/communities/escape2020 | Open/free platform; Large scale infrastructure | There are quite some archives available - also community-based and -centric. However, usually there is not enough long-term support for those. |
| The software heritage is one on a completely different level: https://www.softwareheritage.org | |||
| 11 | Zenodo, Software Heritage, HZDR Archive | Open/free platform; Institute or Organization | |
| 12 | Zenodo/Software Heritage (urls are provided above) | Open/free platform | There is a lack of open infrastructure for depositing Docker images (DockerHub may delete them if not used. GitHub archive has no commitments for long term preservation, as far as I know) |
| 15 | Zenodo | Large scale infrastructure | |
| Software Heritage | |||
| 19 | Based on the answer I’ve given in 3.1 I’d mention CLARIN (https://www.clarin.eu/content/clarin-nutshell)) and it’s b-center repositories. That said, the repositories are definitely not only for EOL software. | Institute or Organization; Large scale infrastructure | |
| 26 | https://github.com/ | Institute or Organization; Other | |
| https://posit.co/ | |||
| 27 | https://helmholtz.software, https://helmholtz.software, https://www.softwareheritage.org | Open/free platform; Institute or Organization; Large scale infrastructure | no |
Auto-created summary
Summary of Mentioned Practices
1. OpenDMP and ARGOS
- Description: Open extensible service for managing Data Management Plans (DMPs) that assists in the management, validation, monitoring, and maintenance of DMPs, facilitating exchanges among infrastructures.
- Relevance: Key tool for managing research data, ensuring compliance with data management requirements.
- Link: ARGOS
2. GitHub and GitLab
- Description: Widely used platforms for code hosting, development, version control, and project management, with capabilities like CI/CD pipelines.
- Relevance: Central tools for software development and maintenance across various projects.
- Links: GitHub, GitLab
3. Zenodo and Software Heritage
- Description: Platforms for archiving research outputs, including software, data, and publications, offering long-term preservation solutions.
- Relevance: Crucial for ensuring the availability and integrity of research outputs.
- Links: Zenodo, Software Heritage
4. Netherlands eScience Center
- Description: Provides infrastructure support for both planning and development phases in projects, though currently considering focusing solely on planning.
- Relevance: Supports research efforts by combining planning with development support for comprehensive project management.
5. CERN GitLab and Issue Tracking
- Description: GitLab instances with integrated storage and compute infrastructure, along with a JIRA instance for issue tracking.
- Relevance: Essential for managing large-scale, complex scientific projects with significant computational needs.
6. Open Source Software Repositories
- Description: Repositories such as npmjs.com, pypi.org, and bioconda, are used for distribution and maintenance of software packages.
- Relevance: Supports the longevity and accessibility of software through widespread distribution.
7. Data Repositories like B2SHARE and CLARIN
- Description: Platforms for sharing and preserving data within specific communities or institutions, ensuring compliance with data governance policies.
- Relevance: Important for fostering collaboration and ensuring data integrity and availability.
- Links: B2SHARE, CLARIN
Linked URLs and Descriptions
| URL | Description |
|---|---|
| https://argos.openaire.eu/splash/about/how-it-works.html | Description of how the ARGoS DMP tool functions. |
| https://docs.github.com/en/issues/planning-and-tracking-with-projects/learning-about-projects/best-practices-for-projects | GitHub documentation on best practices for planning and tracking projects. |
| https://github.com | GitHub platform for hosting, version control, and collaboration. |
| https://gitlab.com | GitLab platform for hosting and managing software development projects. |
| https://gitlab.bsc.es | BSC’s GitLab instance for software development. |
| https://git.km3net.de | Git repository hosting specifically for KM3NeT collaboration. |
| https://gitlab.in2p3.fr | IN2P3’s GitLab instance used for software development. |
| https://information-technology.web.cern.ch/services | CERN’s IT services and infrastructure offerings. |
| https://mybinder.org/ | MyBinder platform for making repositories executable by anyone, anywhere. |
| https://edatos.consorciomadrono.es/ | Data repository used by public universities in Madrid, Spain. |
| https://zenodo.org | Zenodo platform for archiving and sharing research outputs. |
| https://osf.io | Open Science Framework (OSF) for collaboration, management, and archiving projects. |
| https://www.softwareheritage.org | Software Heritage: an archive of software source code for ensuring the preservation of software. |
| https://posit.co/ | Platform formerly known as RStudio, providing tools for data science teams. |
| https://codebase.helmholtz.cloud, | Helmholtz Cloud’s codebase repository for software sharing and development. |