Quick Start¶

This page walks you through your first job on the cluster, from logging in to reading the results. It takes about 15 minutes. Every step links to a page with more details.

Note

You need an account on the cluster first. If you do not have one yet, see Get Access.

1. Log In¶

Connect to the login node from your computer:

ssh your_user_name@login.hpc.imtm.cz

You are now in your home directory on the login node. The login node is shared by all users and is meant for editing files, transferring data and submitting jobs. Computations always run on compute nodes through the scheduler.

2. Upload Your Data¶

Open a second terminal on your computer and copy your input data to the cluster:

scp -r my_data your_user_name@login.hpc.imtm.cz:~/first-job/

For large datasets, use rsync and your project directory, see Data Transfer. For this tutorial, we create a small input file directly on the cluster instead:

mkdir -p ~/first-job && cd ~/first-job
seq 1 1000000 > numbers.txt

3. Find Your Software¶

Software is provided as modules. Search for the program you need and see which versions are available:

module avail                        # List all available modules
module spider Python                # Search for a module and its versions

If your software is not available, you can install it yourself with Conda, a Python virtual environment or a Singularity container. See Lmod for more about modules.

4. Write a Job Script¶

A job script is a Bash script with #PBS lines that tell the scheduler what resources your job needs. Create the file first.pbs:

nano first.pbs

#!/bin/bash
#PBS -N first_job                       # Job name
#PBS -l select=1:ncpus=2:mem=2gb        # 1 chunk with 2 CPUs and 2 GB of memory
#PBS -l walltime=00:10:00               # Maximum runtime 10 minutes
#PBS -j oe                              # Merge output and errors into one file

cd ${PBS_O_WORKDIR}                     # Go to the directory where you ran qsub

echo "Job ${PBS_JOBID} runs on $(hostname) with ${NCPUS} CPUs"
awk '{ sum += $1 } END { print "Sum:", sum }' numbers.txt > result.txt
echo "Done"

Save the file with Ctrl+O and exit nano with Ctrl+X. See Resource Allocation for how to choose CPUs, memory and walltime.

5. Submit the Job¶

qsub first.pbs

The command prints the ID of your job, e.g. 12345.pbs-server.

6. Check the Job¶

qstat -u $USER

The S column shows the state of the job: Q waiting in the queue, R running. When the job finishes, it disappears from this list. Use qstat -x -u $USER to see finished jobs too. See Managing Jobs.

7. Read the Results¶

When the job finishes, its output file appears in the directory from which you submitted it:

ls
cat first_job.o*                        # Output of the job
cat result.txt                          # Result of the computation

Check that the job finished successfully and how many resources it actually used:

qstat -xf <job_id> | grep -E "Exit_status|resources_used"

An Exit_status of 0 means success. If it is different, see Exit Status and Troubleshooting.

Next Steps¶

I want to...	Read
Work with large data efficiently	Scratch Space
Try commands on a compute node interactively	Interactive Jobs
Use many CPUs or several nodes	Parallel Jobs
Run the same job for many input files	Job Arrays
Run a Snakemake or Nextflow pipeline	Snakemake, Nextflow
Install Python or R packages	Python, Conda, R
Keep a session running after disconnecting	Tmux
Look up a command quickly	PBS Cheat Sheet