Welcome to the HPC-Pipes workshop

PLEASE READ BEFORE COURSE!

Course requirements

Required preparation

You are expected to complete the required setup, including tool installation (Docker) and account creation (UCloud).

  • Docker - click on Download Docker Desktop.

Note for Windows users: Installing Docker Desktop requires administrative privileges on your computer. You will be prompted to enter your KU username and password during the installation process. Additionally, Docker Desktop depends on the Windows Subsystem for Linux (WSL), which will also need to be installed.

As for other software, we will provide access to a Danish HPC platform, UCloud, with all the necessary software pre-installed. Please read Using UCloud for exercises carefully.

If you prefer to run the exercises on your personal laptop or a different server, please ensure you have the following software installed:

Using UCloud for exercises

Warning

Follow the instructions below if you have an account at a Danish university. You will need your institutional email to proceed. Unfortunately, this will not work for those without a university email.

  1. Create an account on UCloud with your institution’s credentials
  2. Use the link below to join our workspace where you will find a setup environment1

 

Invite link to UCloud workspace

 

  1. You’re all set! You will receive instructions on how to navigate through UCloud during the course.

Reading material (optional)

  • The Turing way. It offers comprehensive guidance on reproducible research practices, including setting up computational environments and managing reproducible workflows.
  • Mölder, Felix, et al. “Sustainable data analysis with Snakemake.” F1000Research 10 (2021). Link to article. Best practices using Snakemake to develop your pipelines.
  • Check our content on HPC pipes.

Agenda

Day 1

Time Activity Time Activity
8:45 Morning coffee (optional)
9:00 Intro to HPC & onboarding 12:00 Lunch break
9:45 HPC resources 13:00 Exercise - software mgmt
10:15 Coffee break 14:15 Coffee break
10:30 Software mgmt I 14:30 Computations mgmt I
11:15 Software mgmt II 16:00 Discussions & Wrap-up

Environments exercises Snakemake I exercises

Day 2

Time Activity Time Activity
8:45 Morning coffee (optional)
9:00 Computations mgmt II 12:00 Lunch break
9:45 Exercise - integration 13:00 Nextflow, nf-core example
10:15 Coffee break 14:15 Coffee break
10:30 Exercise - implementation 14:30 Build your own pipeline
11:15 Computations mgmt III 16:00 Discussions & Wrap-up

Snakemake II exercises Snakemake III exercises

Discussion and feedback

We hope you enjoyed the workshop. As data scientists, we also would be really happy for some quantifiable info and feedback - we want to build things that the Danish health data science community is excited to use. Please, fill up the feedback form before you head out for the day 2.

 

Nice meeting you and we hope to see you again!

About the National Sandbox project

The Health Data Science Sandbox aims to be a training resource for bioinformaticians, data scientists, and those generally curious about how to investigate large biomedical datasets. We are an active and developing project seeking interested users (both trainees and educators). All of our open-source materials are available on our Github page and can be used on a computing cluster! We work with both UCloud, GenomeDK and Computerome, the major Danish academic supercomputers.

Footnotes

  1. link activated a week before the workshop.↩︎

  2. link activated on the day of the workshop.↩︎