Welcome to the HPC-Launch workshop
Please, read before course!
You are expected to make sure you can sign in to UCloud, SDU’s HPC platform on which we will be running this course. All data, exercises, and tools will be provided on UCloud. Please use your university ID to sign in (instructions below). If you run into problems, please write us (respond to the email that got you to this page).
A Danish university/institution account is required to create a UCloud account. If you don’t have one, click on Local setup below. However, you will need access an HPC to be able to submit SLURM job scripts (a cluster management and job scheduling system).
Access Sandbox resources
Our first choice is to run our tutorials on UCloud, the supercomputer located at the University of Southern Denmark. Anyone using these resources needs the following:
- A Danish university ID so you can sign on to UCloud via WAYF1.
Basic ability to navigate in Linux. You don’t need to be an expert, but it is beyond our ambitions (and course material) to teach you how to code from zero.
Click the invite link below to accept our the invitation to the Sandbox workspace (after accessing UCloud for the first time). This way, we can provide you with compute resources for the active sessions of the workshop2.
Invite link to UCloud workspace
- Create an account on GitHub. Follow these instructions.
Are you a Windows user working at a hospital? If so, please contact us, as one of the exercises involves creating SSH keys, and you may not have the necessary permissions to do this on your system.
- You’re all set! You will receive instructions on how to navigate through UCloud during the course.
- Download the slides (link active on May 22nd) and RDM data lifecycle roadmap:
Independent users
If you are interested in running the exercises locally, click on the box below. Not recommended for participants in HPC-Launch workshop.
You are expected to complete the required setup, including tool installation and account creation.
Git for version control of your projects.
GitHub account for hosting and collaborating on projects.
pip for managing Python packages.
Cookicutter for creating folder structure templates (
pip install cookiecutter).md5sum. See below examples on how to install.
conda. Install Miniforge which comes preconfigured for use with the conda-forge channel.
Terminal
# ---- cookiecutter ----- pip install cookiecutter # ---- md5sum from coreutils package----- # On Ubuntu/Debian apt-get install coreutils # On macOS brew install coreutils
Highly recommended
- Zenodo account for archiving and sharing your research outputs
- DeiC DMP
If you run into any issues installing the software, don’t worry! We will provide access to a Danish HPC platform, UCloud, with all the necessary software pre-installed. Please read above carefully.
Reading material
About Research Data Management (RDM):
About High-Performance Computing (HPC):
Agenda
A more detailed program will be posted just before we begin.
DAY 1
| Time | Activity | Time | Activity |
|---|---|---|---|
| 8:45 | Morning coffee (optional) | ||
| 9:00 | Introduction to the Sandbox project | 12:00 | Lunch break |
| 9:15 | Introduction to HPC: the basics | 13:00 | HPC exercises |
| 10:15 | Coffee break | 14:15 | Coffee break |
| 10:30 | Intro to RDM | 14:30 | HPC workflow II |
| 11:15 | HPC workflow | 15:00 | HPC exercises |
DAY 2
| Time | Activity | Time | Activity |
|---|---|---|---|
| 8:45 | Morning coffee (optional) | ||
| 9:00 | Recap | 12:00 | Lunch break |
| 9:15 | HPC workflow III | 13:00 | HPC wrap up |
| 10:15 | Coffee break | 14:15 | Coffee break |
| 10:30 | HPC exercises | 15:00 | DK HPC solutions & resources |
Discussion and feedback
We hope you enjoyed the workshop. As data scientists, we also would be really happy for some quantifiable info and feedback - we want to build things that the Danish health data science community is excited to use. Please, fill up the feedback form before you head out for the day 3.
About the National Sandbox project
The Health Data Science Sandbox aims to be a training resource for bioinformaticians, data scientists, and those generally curious about how to investigate large biomedical datasets. We are an active and developing project seeking interested users (both trainees and educators). All of our open-source materials are available on our Github page and can be used on a computing cluster! We work with both UCloud, GenomeDK and Computerome, the major Danish academic supercomputers.
Footnotes
Other institutions (e.g. hospitals, libraries, …) can log on through WAYF. See all institutions here.↩︎
To use Sandbox materials outside of the workshop: remember that each new user has hundreds of hours of free computing credit and around 50GB of free storage, which can be used to run any UCloud software. If you run out of credit (which takes a long time) you’ll need to check with the local DeiC office at your university about how to request compute hours on UCloud. Contact us at the Sandbox if you need help or want more information.↩︎
link activated on the day of the workshop.↩︎