Package managers

Managing software

You need to use the Coder app as we had configured it for the Cookiecutter exercises. If your job is not longer running, submit a new one!

  1. Name and version of the app to run: Coder - 1.117.0.
  2. Job settings: selecting a 1 CPU standard node with 3GB memory.
  3. Add folders to access while in this job (select your own drive!)
    • You need access to our shared folder so that you can get the correct software environment and other material for the exercises (shared/HPCLab_workshop).
    • You need to select your hpcLaunch in your own drive, to save any output from the exercise! You won’t have writing permissions on our shared drive.
  4. Optional Parameters. Choose shared/HPCLab_workshop/setup.sh as the initialization file.
  5. Click on the Submit button (and wait!)

Once the Coder app is running, open a new terminal.

1. Exploring an existing environment

Conda is not pre-installed in the Terminal or Coder app on UCloud, so how can we use conda? with a local installation of Miniconda, say miniconda3, mounted inside the app. If you want to know how to do this, follow the instructions on UCloud docs (Setting Up Conda on UCloud).

In the interest of time, we will skip the installation of Miniconda and instead use the version available on our shared drive. There is nothing you need to do, as you have already mounted the shared drive, which includes a pre-installed Miniconda setup. The initialization script (setup.sh) adds Conda to the search path and activates the environment automatically, so it is ready to use.

Let’s deactivate the environment before starting the exercise.

conda deactivate

Did you notice something? The hpclab-env text indicating that a Conda environment was active has disappeared.

We will run the following commands to get familiar with Conda environments.

  • What is the conda version? 
conda info
conda --version
  • How many different environments are available to you (excluding base which is automatically created with Conda)? 
conda env list
  • Let’s explore one of the environments. To do this, let’s activate hpclab-env environment
conda activate hpclab-env
  • How many packages are available in this environment? 
conda list |grep -v '#' | wc -l

Imagine you need to share your environment with a collaborator so they can replicate your analysis. How do we do this?

  • Export the environment specifications and save them to your personal drive (e.g., .yml)
conda env export --from-history > <yourname-hpclab>.yml
  • Deactivate the environment again.
conda deactivate

2: Building your own conda environment

Let’s create a new directory in your personal drive:

cd /work/hpcLaunch/day2
mkdir envs

Now, add a new environment using the argument --prefix PATH (e.g., /work/hpcLaunch/day2/envs/<name-env>). We need to do this as miniconda is installed in a directory that you don’t have write rights to.

Locally, you would typically run the command: conda create --name <myenv>

Warning

Always specify the location using --prefix PATH regardless of the UCloud app you use, especially if the app has conda pre-installed (e.g. Jupyterlab). The path must be on your personal drive or a shared drive with colleagues that you have access to. Otherwise, the environment won’t be saved, as you’re working within a temporary container instance.

  • Create the environment
conda create --prefix /work/hpcLaunch/day2/envs/<test-env>
  • Confirm the environment location
Proceed ([y]/n)? y
  • Double-check that the new environment exists
conda env list

The new environment doesn’t have a name as we use prefix to create the env outside the envs subdirectory in the Conda installation but don’t worry about it!

  • Activate the env
conda activate /work/hpcLaunch/day2/envs/<test-env>
  • By default, we won’t have conda-forge or bioconda added to the environment. Let’s fix this.
conda config --remove channels defaults
conda config --add channels bioconda 
conda config --add channels conda-forge
  • Add the latest Python version to the environment
Search available versions > 3.12
conda search "python>=3.12"
Install the latest
conda install python=3.14.4
  • Install samtools, a tool for manipulating DNA sequencing data
conda install -c bioconda samtools
  • Uninstall samtools
conda remove samtools
  • Deactivate your environment
conda deactivate

You can cancel the job when you are done with this exercise (‘Stop Application’).