• Home
      • |
    • Engineering Compute Cluster – Running Jobs

Engineering Compute Cluster – Running Jobs

The cluster uses the Slurm scheduler. You describe what your job needs (partition, time, memory, and cores) in a script, submit the script, and Slurm runs it on a node when resources are available. Please do not run heavy computations directly on the login node.

Choosing a Partition

  • sbatch: all 20 CPU nodes, low priority (the default).
  • isecompute: nodes 1 to 10 at high priority, exclusive to the ISE department.
  • gpu: the single GPU node with one A100 80 GB GPU.

Each CPU node has two processors with 12 cores each (24 cores per node).

Required Job Settings

Every job script should specify at least these four values:

  • Partition (-p)
  • Time limit (-t), in the format days-hours:minutes
  • Memory (--mem), in megabytes unless you add a unit such as G
  • Number of cores (-n)

The example below also sets the number of nodes (-N), output and error file names, and email notifications. These are optional.

Example Job Script

This example generates a file of random numbers and sorts it. It requests one core and 100 MB of memory for up to two hours.

#!/bin/bash
#SBATCH -p sbatch                 # partition (queue)
#SBATCH -N 1                      # number of nodes
#SBATCH -n 1                      # number of cores
#SBATCH --mem 100                 # memory pool for all cores (in MB)
#SBATCH -t 0-2:00                 # time (D-HH:MM)
#SBATCH -o slurm.%N.%j.out        # STDOUT
#SBATCH -e slurm.%N.%j.err        # STDERR
#SBATCH --mail-user=NETID@msstate.edu
#SBATCH --mail-type=ALL

for i in {1..100000}; do
  echo $RANDOM >> SomeRandomNumbers.txt
done
sort SomeRandomNumbers.txt

To use it:

  1. Copy the script into a file. For example, nano myscript.sh opens an editor where you can paste it.
  2. Change NETID in the mail-user line to your own NetID. If you do not want email notifications, delete the mail-user and mail-type lines.
  3. Submit the job:
sbatch myscript.sh

Requesting the GPU

To use the A100 GPU, submit to the gpu partition and request the GPU with --gres=gpu:1. Slurm only assigns the GPU to jobs that request it, so a job that names the partition without --gres will run without GPU access.

#!/bin/bash
#SBATCH -p gpu                    # GPU partition
#SBATCH -N 1                      # number of nodes
#SBATCH -n 8                      # number of cores
#SBATCH --mem 32G                 # memory pool for all cores
#SBATCH --gres=gpu:1              # request the GPU
#SBATCH -t 0-4:00                 # time (D-HH:MM)
#SBATCH -o slurm.%N.%j.out        # STDOUT
#SBATCH -e slurm.%N.%j.err        # STDERR

nvidia-smi                        # confirm the GPU is visible to the job
./my_gpu_program

Load CUDA or any other modules your program needs before running it (run module avail to see the exact module names). There is only one GPU, so GPU jobs run one at a time.

Interactive Jobs

To work interactively on a compute node (for example, to test commands or debug a program), request a shell with srun:

srun -p sbatch -n 1 --mem 4G -t 0-1:00 --pty bash

When the shell starts, you are on a compute node and can run commands directly. Type exit to end the session and release the resources. Add --gres=gpu:1 and use -p gpu for an interactive session on the GPU node.

Estimating When Your Job Will Run

To see when Slurm expects to start your job without actually submitting it, use:

sbatch --test-only myscript.sh

Using Software Modules in Jobs

Load the modules your program needs before it runs. In a job script, put the module load command before your program:

module load openfoam/2412

You can also run module initadd NAME to load a module automatically in every new shell, which helps MPI programs and other jobs that run across several nodes. See Engineering Compute Cluster – Overview for the list of available software.

Checking and Managing Jobs

squeue -u NETID     # show your queued and running jobs
scancel JOBID       # cancel a job
sinfo               # show partition and node status
sinfo -o "%P %l"    # show the maximum time limit for each partition

More Examples

Additional Slurm example scripts are available on the cluster in /usr/local/share/slurm.

See Also