Main Content Region

Using sbcast to copy files to a collection of compute nodes

This sbcast command can only be run from a slurm allocation or a slurm batch submitted script. sbcast - transmit a file to the nodes allocated to a SLURM job. This command can be used in situations where you need to distribute a single data file to all your compute nodes (on each compute nodes' local file system) The basic syntax is
Where *SOURCE* is your input file and *DEST* is the target location on each assigned host, this will usually be the same on every node unless you do something clever with environment variables to create unique names. For example if you wish to copy an CHKPOINT file for a compute job that requires the working directory to be on a local file system on a compute node, you can do something like this
pdsh -w $SLURM_NODELIST mkdir -p /tmp/JobID/
sbcast ~/Calculations/SomeDFTCode/jobID/CHKPOINT /tmp/JobID/CHKPOINT
The pdsh command is a *parallel distributed shell* for executing commands in an efficient way across nodes.