This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
Back to: Home Computing Wiki Page
qsub on sigma0
Connect to sigma0 with SLC6 as OS
ssh -p 28 -X -o ServerAliveInterval=60 -C -c aes128-ctr -m hmac-sha1 <username>@sigma0.physi.uni-heidelberg.de
Setup your local environment
The simplest way.
source /cvmfs/lhcb.cern.ch/lib/LbEnv
lb-run Urania/latest bash
to submit a few jobs (about 2-50) that take more than a couple minutes each
Build a bash script that creates job specific bash-scripts and submits the jobs from individual job folders.
#!/bin/bash
#set -x #echo on
# A sample Bash script
echo 'Running Script'
workdir=/work/username/workfolder
#a, x, and y are some variables that I want to loop over with my executable
a=27
x=$(($a + 1))
for ((y=0; y<=$a; y=y+2))
do
i=$((i+1))
mkdir -p $workdir/jobs/runjob$i
rm -rf $workdir/jobs/runjob$i/* #cleanup old jobs
touch $workdir/jobs/runjob$i/runjob$i.sh #create the job bash script
echo "#!/bin/bash" >> $workdir/jobs/runjob$i/runjob$i.sh
echo "$workdir/my_executable $a $x $y" >> $workdir/jobs/runjob$i/runjob$i.sh
cd $workdir/jobs/runjob$i
echo "submitting--> $workdir/my_executable $a $x $y as ./runjob$i.sh" #output to check everything is being calculated as you want
qsub -V -b n -l os=slc6 -l ujl=20 -cwd -j yes ./runjob$i.sh # submit the job. -V passes the current environment
sleep 2 #be nice to the submission system
done
exit 1
to submit a few hundred jobs use the array feature of qsub
‘qsub’ command combined with the -t option allow for the submission of an ‘array’ of jobs. When a job is launched via ‘qsub -t n[-m[:s]]’ you will have the environment variables SGE_TASK_ID and SGE_TASK_FIRST, and if you add m you will have SGE_TASK_LAST, and if you add s you will have SGE_TASK_STEPSIZE. So -t sets the ‘index numbers’ associated with the job, like so:
- n is the first index number
- m is the last index number (optional)
- s is the step size (optional, defaults to 1)
runarray_jobcreater.sh
#!/bin/bash
#set -x #echo on
# A sample Bash script
echo 'Running Script'
workdir=/work/username/workfolder
#a, x, and y are some variables that I want to loop over with my executable
a=2700
x=$(($a + 1))
for ((y=0; y<=$a; y=y+2))
do
i=$((i+1))
mkdir -p $workdir/jobs/
rm -rf $workdir/jobs/* #cleanup old jobs
echo "$a $x $y" >> $workdir/jobs/parameters.$i #make a unique parameter file for each task so that we can use the $SGE_TASK_ID to easily reference
done
cd /$workdir/jobs/
#submits $i jobs as an array of jobs to the queue "-t 1-$i"
#-V passes the current environment
#
qsub -t 1-$i -V -b n -l os=slc6 -l ujl=20 -cwd -j yes ./runarray.sh
exit 1
runarray.sh
#!/bin/bash
#
#
workdir=/work/username/workfolder
echo "Task id is $SGE_TASK_ID"
#executes a version of your executable that will read in the parameter file that you have specifically created for this
$workdir/my_executable $workdir/jobs/parameters.$SGE_TASK_ID
imported from lhcbmedia wiki If you want to submit jobs to only some of the machines on our cluster(e.g. leave one machine free for other users), you can specify it the queues options(chosing also a machines):
qsub -masterq fgslc5.q@nu*,fgslc5.q@lambda*,fgslc5.q@alpha*,fgslc5.q@beta*,fgslc5.q@gamma*,fgslc5.q@eta*,fgslc5.q@delta*,fgslc5.q@lhcb-raid03*