diff --git a/computing_qsub.md b/computing_qsub.md new file mode 100644 index 0000000..ee509f8 --- /dev/null +++ b/computing_qsub.md @@ -0,0 +1,91 @@ +# qsub on sigma0 +Connect to sigma0 with SLC6 as OS +```shell +ssh -p 28 -X -o ServerAliveInterval=60 -C -c aes128-ctr -m hmac-sha1 @sigma0.physi.uni-heidelberg.de + +``` + +# Setup your local environment +The simplest way. +```shell +source /cvmfs/lhcb.cern.ch/lib/LbEnv +lb-run Urania/latest bash +``` +## to submit a few jobs (about 2-50) that take more than a couple minutes each +Build a bash script that creates job specific bash-scripts and submits the jobs from individual job folders. + +```sh +#!/bin/bash +#set -x #echo on +# A sample Bash script +echo 'Running Script' +workdir=/work/username/workfolder +#a, x, and y are some variables that I want to loop over with my executable +a=27 +x=$(($a + 1)) + for ((y=0; y<=$a; y=y+2)) + do + i=$((i+1)) + mkdir -p $workdir/jobs/runjob$i + rm -rf $workdir/jobs/runjob$i/* #cleanup old jobs + touch $workdir/jobs/runjob$i/runjob$i.sh #create the job bash script + echo "#!/bin/bash" >> $workdir/jobs/runjob$i/runjob$i.sh + echo "$workdir/my_executable $a $x $y" >> $workdir/jobs/runjob$i/runjob$i.sh + cd $workdir/jobs/runjob$i + echo "submitting--> $workdir/my_executable $a $x $y as ./runjob$i.sh" #output to check everything is being calculated as you want + qsub -V -b n -l os=slc6 -l ujl=20 -cwd -j yes ./runjob$i.sh # submit the job. -V passes the current environment + sleep 2 #be nice to the submission system + done +exit 1 +``` + +## to submit a few hundred jobs use the array feature of qsub +‘qsub’ command combined with the -t option allow for the submission of an ‘array’ of jobs. When a job is launched via ‘qsub -t n[-m[:s]]’ you will have the environment variables SGE_TASK_ID and SGE_TASK_FIRST, and if you add m you will have SGE_TASK_LAST, and if you add s you will have SGE_TASK_STEPSIZE. So -t sets the ‘index numbers’ associated with the job, like so: + - n is the first index number + - m is the last index number (optional) + - s is the step size (optional, defaults to 1) + +***runarray_jobcreater.sh*** +```sh +#!/bin/bash +#set -x #echo on +# A sample Bash script +echo 'Running Script' +workdir=/work/username/workfolder +#a, x, and y are some variables that I want to loop over with my executable +a=2700 +x=$(($a + 1)) + for ((y=0; y<=$a; y=y+2)) + do + i=$((i+1)) + mkdir -p $workdir/jobs/ + rm -rf $workdir/jobs/* #cleanup old jobs + echo "$a $x $y" >> $workdir/jobs/parameters.$i #make a unique parameter file for each task so that we can use the $SGE_TASK_ID to easily reference + done + +cd /$workdir/jobs/ +#submits $i jobs as an array of jobs to the queue "-t 1-$i" +#-V passes the current environment +# +qsub -t 1-$i -V -b n -l os=slc6 -l ujl=20 -cwd -j yes ./runarray.sh +exit 1 +``` + +***runarray.sh*** +```sh +#!/bin/bash +# +# +workdir=/work/username/workfolder + +echo "Task id is $SGE_TASK_ID" + +#executes a version of your executable that will read in the parameter file that you have specifically created for this +$workdir/my_executable $workdir/jobs/parameters.$SGE_TASK_ID +``` + +_imported from lhcbmedia wiki_ + If you want to submit jobs to only some of the machines on our cluster(e.g. leave one machine free for other users), you can specify it the queues options(chosing also a machines): +``` +qsub -masterq fgslc5.q@nu*,fgslc5.q@lambda*,fgslc5.q@alpha*,fgslc5.q@beta*,fgslc5.q@gamma*,fgslc5.q@eta*,fgslc5.q@delta*,fgslc5.q@lhcb-raid03* +``` \ No newline at end of file