ScaleMP-Large Shared Memory System



One of the resources available on Guillimin is a collection of nodes which behaves like a single computer with a large shared memory. This system, called scalemp, consists of 11 nodes with 12 cores per node and 8GB of memory per core. Because of software running on these nodes, scalemp can be used like a single node with 132 cores and a 1000GB of memory. Therefore, this system is particularly useful for researchers who require access to large amounts of memory.

More information about ScaleMP can be found here.

Gaining Access to ScaleMP

To get access to the scalemp system, users must first get an account on Guillimin. All Guillimin users have access to the ScaleMP machine through the scalemp queue (see Using ScaleMP below)

Using ScaleMP

Jobs can be submitted to the ScaleMP system by using the queue called 'scalemp'. Consider this example submission script smpsubmit.bat:

#PBS -l nodes=1:ppn=14
#PBS -l walltime=0:10:00
#PBS -o scalemp.out
#PBS -e scalemp.err
#PBS -N smptest

cd /home/user/smpdir

mpiexec -n 14 ./mpiprog

This job can be submitted to the scalemp queue with the qsub command:

user@lg-1r14-n03$ qsub -q scalemp ./smpsubmit.bat

Several examples of programs designed specifically for the scalemp system are stored in the directory /opt/ScaleMP/examples. It is a good idea to make a copy of these programs in your home directory so that you may alter, compile, and run them:

user@lm-2r02-n15$ mkdir ScaleMPexamples
user@lm-2r02-n15$ cp -R /software/ScaleMP/examples/* ScaleMPexamples/. 

This example folder contains examples of many different types of jobs (e.g. Serial, MPI, OpenMP, MKL, Pthread, etc). Each type of job contains an example run-script and useful information. Please identify the type of job that best resembles your application, carefully follow the relevant guidelines, and modify a copy of the example run-script as appropriate for your application.

Software Available on ScaleMP

 Software module nameDESCRIPTIONLink for more information
APPLICATIONS test test test
test test test
test test test
test test test