Running Jobs¶

This guide explains how to create and submit a job script to the OpenPBS scheduler on the IMTM HPC cluster. It also covers how to allocate resources such as CPUs, memory, and walltime.

Warning

All jobs must be submitted via the scheduler using qsub. Running tasks directly on compute nodes is strongly discouraged, as all processes started outside of jobs share a limit of 4 CPUs and 16 GB of memory per node, enforced by cgroup restrictions. This limit is shared by all users logged in to the node.

Tip

If you need to run processes manually on compute nodes, consider using Interactive Jobs. These allow you to run commands directly on compute nodes from the command line, making them ideal for manual tasks and troubleshooting.

Create a Job Script¶

Use any text editor (e.g., nano, vim) to create a PBS job script:

nano job.pbs

Example content for job.pbs:

#!/bin/bash
#PBS -N my_job                     # Job name
#PBS -l select=1:ncpus=4:mem=8gb   # Resource request directive
#PBS -l walltime=02:00:00          # Maximum runtime (hh:mm:ss)
#PBS -j oe                         # Merge stdout and stderr
#PBS -o output.log                 # Output file

module load mysoftware      # Load any necessary modules
./my_program                # Run your application

Common Directives¶

Directives start with #PBS at the beginning of the job script, before the first command. The same options can also be passed directly to qsub on the command line, where they take precedence over the directives in the script.

Directive	Purpose
`#PBS -N my_job`	Job name
`#PBS -l select=1:ncpus=4:mem=8gb`	Requested resources
`#PBS -l walltime=02:00:00`	Maximum runtime (hhss)
`#PBS -l place=scatter`	Placement of chunks on nodes
`#PBS -j oe`	Merge stdout and stderr into one file
`#PBS -o output.log`	Standard output file
`#PBS -e error.log`	Standard error file
`#PBS -J 1-10`	Create a job array
`#PBS -W depend=afterok:1234`	Start after another job finishes
`#PBS -v MY_VAR=value`	Pass an environment variable to the job
`#PBS -V`	Pass all environment variables of the current shell to the job

For example, the following command overrides the walltime and passes a variable to the job script:

qsub -l walltime=04:00:00 -v INPUT=sample_01.dat job.pbs

Note

Jobs start in your home directory, not in the directory from which you submitted them. Use cd ${PBS_O_WORKDIR} at the beginning of the script to work in the submission directory.

Job Environment Variables¶

Commonly used job environment variables give your job script quick access to essential details.

Variable	Purpose
`PBS_JOBID`	Unique identifier for the running job
`PBS_JOBNAME`	Human-readable job name (`#PBS -N` or script filename)
`PBS_NODEFILE`	Path to a file listing all allocated cores/nodes
`PBS_O_WORKDIR`	Directory from which `qsub` was invoked
`PBS_QUEUE`	Name of the queue in which the job is executing
`NCPUS`	Number of CPU cores allocated on the node
`OMP_NUM_THREADS`	Number of threads for OpenMP applications
`SCRATCH`	Path to the job's scratch directory (only if scratch is requested)

Submitting the Job¶

Submit your script to the scheduler:

qsub job.pbs

You will receive a job ID as confirmation.

Monitoring the Job¶

Check your jobs:

qstat -u $USER

Get detailed info for a specific job:

qstat -f <job_id>

For more ways to check, hold, modify and debug your jobs, see Managing Jobs.

Canceling a Job¶

To stop a job:

qdel <job_id>