PrepareJob

From BIOcrunch
(Difference between revisions)
Jump to: navigation, search
(Examples)
 
(31 intermediate revisions by 3 users not shown)
Line 1: Line 1:
To run your job on the cluster, prepare it on your local machine, then copy it to the cluster. Beside the datafiles you will need a jobfile containing all you setting.
+
To run a job on the cluster, prepare the data files on your local machine, then [[File Transfers|copy it to the cluster]]. Beside the data files, you will need a job file with the settings as outlined below.
  
Name your job file and your data files in a consistent way:
+
Name your job and data files in a consistent way:
 
* Job file: myjobfile.sh
 
* Job file: myjobfile.sh
* User: Mustermann
+
* User: user01
 
* Email: max@mustermann.de
 
* Email: max@mustermann.de
  
These jobfiles are written in a sort of "shell" command syntax of Linux systems.
+
* Job files are written as a sort of "shell" commands that are common on Linux systems
 +
* Lines starting with "##" or "# " are comment lines to document some stuff for better reading afterwards
 +
* Lines starting with "#$" are commands for the job scheduler "qsub"
 +
* Lines starting not with "#" are commands executed
 +
* '''Make sure that the skript does not contain empty lines! Delete them or comment them out using "#"'''
  
Lines starting with "##" or "# " are comment lines to document some stuff for better reading afterwards.
 
 
Lines starting with "#$" are commands for the job scheduler "qsub".
 
 
Lines starting not with "#" are commands to execute programs in this job.
 
  
 
== Examples ==
 
== Examples ==
You may use one of the following examples as template for you job. Make sure to change your username, mailadress, filenames and eventually program option in the coloured lines.
+
You may use one of the following examples as a template for you job. Lines that need to be modified are highlighted. Make sure to change username, email adress, filenames, and, if necessary, program options. The last line of each script calls the program you want to run, including all parameters. The number of CPUs is stored in $NSLOTS and is set later when starting the job.
  
 
=== Beast ===
 
=== Beast ===
<source lang="bash" line start="1" highlight="4,13,24,25,28" enclose="div">
+
<source lang="bash" start="1" highlight="4,7,18,19,22" enclose="div">
 
#!/bin/csh
 
#!/bin/csh
 
#
 
#
## Username (Which account to be charged cpu time)
+
## User name (Which account to be charged cpu time)
#$ -A Mustermann
+
#$ -A user01
 
#
 
#
 +
## Send email to users
 +
#$ -M max@mustermann.de
 
#
 
#
## If you want to make it queue now, get rid of the -a option by adding ## before $
+
## Send mail at beginning/end/on suspension
## If you want to make it queue later, use this format [[CC]yy]MMDDhhmm[.SS]
+
#$ -m bes
##$ -a 201407300600
+
#
 +
## Export these environmental variables
 +
#$ -v PVM_ROOT,LD_LIBRARY_PATH=/share/apps/beaglenew
 +
#
 +
## The job is located in the current working directory.
 +
#$ -cwd
 +
## Filenames for output and error log files
 +
#$ -o myjobfile.out
 +
#$ -e myjobfile.err
 +
#
 +
env >./xenv
 +
/share/apps/beast/bin/beast -beagle -beagle_CPU -beagle_instances $NSLOTS -overwrite ./myjobfile.xml > ./myjobfile.out
 +
</source>
 +
 
 +
=== Beast2 ===
 +
<source lang="bash" start="1" highlight="4,7,18,19,22" enclose="div">
 +
#!/bin/csh
 
#
 
#
 +
## User name (Which account to be charged cpu time)
 +
#$ -A user01
 
#
 
#
## Send mail to these users
+
## Send email to users
 
#$ -M max@mustermann.de
 
#$ -M max@mustermann.de
 
#
 
#
## Mail at beginning/end/on suspension
+
## Send email at beginning/end/on suspension
 
#$ -m bes
 
#$ -m bes
 
#
 
#
Line 41: Line 60:
 
## The job is located in the current working directory.
 
## The job is located in the current working directory.
 
#$ -cwd
 
#$ -cwd
## This are the filenames, where to write outputs and error-logs too
+
## Filenames for output and error log files
 
#$ -o myjobfile.out
 
#$ -o myjobfile.out
 
#$ -e myjobfile.err
 
#$ -e myjobfile.err
 
#
 
#
 
env >./xenv
 
env >./xenv
/share/apps/beast180/bin/beast -beagle -beagle_CPU -beagle_instances $NSLOTS -overwrite ./myjobfile.xml > ./myjobfile.out
+
/share/apps/beast210/bin/beast -beagle -beagle_CPU -beagle_instances $NSLOTS -overwrite ./myjobfile.xml > ./myjobfile.out
 
</source>
 
</source>
  
 
=== Mr Bayes ===
 
=== Mr Bayes ===
<source lang="bash" line start="1" highlight="4,13,24,25,28" enclose="div">
+
<source lang="bash" start="1" highlight="4,7,18,19,22" enclose="div">
 
#!/bin/csh
 
#!/bin/csh
 
#
 
#
## Username (Which account to be charged cpu time)
+
## User name (Which account to be charged cpu time)
#$ -A Mustermann
+
#$ -A user01
 
#
 
#
 +
## Send email to users
 +
#$ -M max@mustermann.de
 
#
 
#
## If you want to make it queue now, get rid of the -a option by adding ## before $
+
## Send email at beginning/end/on suspension
## If you want to make it queue later, use this format [[CC]yy]MMDDhhmm[.SS]
+
#$ -m bes
##$ -a 201407300600
+
#
 +
## Export these environmental variables
 +
#$ -v PVM_ROOT,LD_LIBRARY_PATH=/share/apps/beaglenew
 +
#
 +
## The job is located in the current working directory
 +
#$ -cwd
 +
## Filenames for output and error log files
 +
#$ -o myjobfile.out
 +
#$ -e myjobfile.err
 +
#
 +
env >./xenv
 +
mpirun -np $NSLOTS /share/apps/mrbayes/mb ./beispiel.nex </dev/null
 +
</source>
 +
 
 +
=== Raxml ===
 +
<source lang="bash" start="1" highlight="4,7,18,19,22" enclose="div">
 +
#!/bin/csh
 
#
 
#
 +
## User name (Which account to be charged cpu time)
 +
#$ -A user01
 
#
 
#
## Send mail to these users
+
## Send email to users
 
#$ -M max@mustermann.de
 
#$ -M max@mustermann.de
 
#
 
#
## Mail at beginning/end/on suspension
+
## Send email at beginning/end/on suspension
 
#$ -m bes
 
#$ -m bes
 
#
 
#
 
## Export these environmental variables
 
## Export these environmental variables
#$ -v PVM_ROOT,LD_LIBRARY_PATH=/share/apps/beaglenew
+
#$ -v PVM_ROOT
 
#
 
#
 
## The job is located in the current working directory.
 
## The job is located in the current working directory.
 
#$ -cwd
 
#$ -cwd
## This are the filenames, where to write outputs and error-logs too
+
## Filenames for output and error log files
 
#$ -o myjobfile.out
 
#$ -o myjobfile.out
 
#$ -e myjobfile.err
 
#$ -e myjobfile.err
 
#
 
#
 
env >./xenv
 
env >./xenv
mpirun -np $NSLOTS /share/apps/mrbayes/mb322 ./beispiel.nex </dev/null
+
mpirun -np $NSLOTS /share/apps/raxml/raxmlHPC-MPI-SSE3.icc -s sequencefile.phy -n outputfile.phy -m PROTGAMMAWAG
 
</source>
 
</source>
  
=== Raxml ===
+
=== Exabayes ===
<source lang="bash" line start="1" highlight="4,13,24,25,28" enclose="div">
+
<source lang="bash" start="1" highlight="4,7,18,19,22" enclose="div">
 
#!/bin/csh
 
#!/bin/csh
 
#
 
#
## Username (Which account to be charged cpu time)
+
## User name (Which account to be charged cpu time)
#$ -A Mustermann
+
#$ -A user01
 
#
 
#
 +
## Send email to users
 +
#$ -M max@mustermann.de
 
#
 
#
## If you want to make it queue now, get rid of the -a option by adding ## before $
+
## Send email at beginning/end/on suspension
## If you want to make it queue later, use this format [[CC]yy]MMDDhhmm[.SS]
+
#$ -m bes
##$ -a 201407300600
+
 
#
 
#
 +
## Export these environmental variables
 +
#$ -v PVM_ROOT
 +
#
 +
## The job is located in the current working directory.
 +
#$ -cwd
 +
## Filenames for output and error log files
 +
#$ -o myjobfile.out
 +
#$ -e myjobfile.err
 +
#
 +
env >./xenv
 +
mpirun -np $NSLOTS /share/apps/exabayes/exabayes -f aln.phy -q aln.part -n myRun -s 57913 -c config.nex -R 2 -C 2
 +
</source>
 +
 +
=== Iq-Tree ===
 +
<source lang="bash" start="1" highlight="4,7,18,19,22" enclose="div">
 +
#!/bin/csh
 +
#
 +
## User name (Which account to be charged cpu time)
 +
#$ -A user01
 
#
 
#
## Send mail to these users
+
## Send email to users
 
#$ -M max@mustermann.de
 
#$ -M max@mustermann.de
 
#
 
#
## Mail at beginning/end/on suspension
+
## Send email at beginning/end/on suspension
 
#$ -m bes
 
#$ -m bes
 
#
 
#
Line 105: Line 164:
 
## The job is located in the current working directory.
 
## The job is located in the current working directory.
 
#$ -cwd
 
#$ -cwd
## This are the filenames, where to write outputs and error-logs too
+
## Filenames for output and error log files
 
#$ -o myjobfile.out
 
#$ -o myjobfile.out
 
#$ -e myjobfile.err
 
#$ -e myjobfile.err
 
#
 
#
 
env >./xenv
 
env >./xenv
mpirun -np $NSLOTS /share/apps/raxml/raxmlHPC-MPI-SSE3.icc -s sequencefile.phy -n outputfile.phy -m PROTGAMMAWAG
+
/share/apps/iqtree/iqtree-omp -omp $NSLOTS -s example.phy -m TEST
 
</source>
 
</source>
 
{{tutorial_bar}}
 

Latest revision as of 15:37, 24 May 2017

To run a job on the cluster, prepare the data files on your local machine, then copy it to the cluster. Beside the data files, you will need a job file with the settings as outlined below.

Name your job and data files in a consistent way:

  • Job file: myjobfile.sh
  • User: user01
  • Email: max@mustermann.de
  • Job files are written as a sort of "shell" commands that are common on Linux systems
  • Lines starting with "##" or "# " are comment lines to document some stuff for better reading afterwards
  • Lines starting with "#$" are commands for the job scheduler "qsub"
  • Lines starting not with "#" are commands executed
  • Make sure that the skript does not contain empty lines! Delete them or comment them out using "#"


Contents

[edit] Examples

You may use one of the following examples as a template for you job. Lines that need to be modified are highlighted. Make sure to change username, email adress, filenames, and, if necessary, program options. The last line of each script calls the program you want to run, including all parameters. The number of CPUs is stored in $NSLOTS and is set later when starting the job.

[edit] Beast

#!/bin/csh
#
## User name (Which account to be charged cpu time)
#$ -A user01
#
## Send email to users
#$ -M max@mustermann.de
#
## Send mail at beginning/end/on suspension
#$ -m bes
#
## Export these environmental variables
#$ -v PVM_ROOT,LD_LIBRARY_PATH=/share/apps/beaglenew
#
## The job is located in the current working directory.
#$ -cwd
## Filenames for output and error log files
#$ -o myjobfile.out
#$ -e myjobfile.err
#
env >./xenv
/share/apps/beast/bin/beast -beagle -beagle_CPU -beagle_instances $NSLOTS -overwrite ./myjobfile.xml > ./myjobfile.out

[edit] Beast2

#!/bin/csh
#
## User name (Which account to be charged cpu time)
#$ -A user01
#
## Send email to users
#$ -M max@mustermann.de
#
## Send email at beginning/end/on suspension
#$ -m bes
#
## Export these environmental variables
#$ -v PVM_ROOT,LD_LIBRARY_PATH=/share/apps/beaglenew
#
## The job is located in the current working directory.
#$ -cwd
## Filenames for output and error log files
#$ -o myjobfile.out
#$ -e myjobfile.err
#
env >./xenv
/share/apps/beast210/bin/beast -beagle -beagle_CPU -beagle_instances $NSLOTS -overwrite ./myjobfile.xml > ./myjobfile.out

[edit] Mr Bayes

#!/bin/csh
#
## User name (Which account to be charged cpu time)
#$ -A user01
#
## Send email to users
#$ -M max@mustermann.de
#
## Send email at beginning/end/on suspension
#$ -m bes
#
## Export these environmental variables
#$ -v PVM_ROOT,LD_LIBRARY_PATH=/share/apps/beaglenew
#
## The job is located in the current working directory
#$ -cwd
## Filenames for output and error log files
#$ -o myjobfile.out
#$ -e myjobfile.err
#
env >./xenv
mpirun -np $NSLOTS /share/apps/mrbayes/mb ./beispiel.nex </dev/null

[edit] Raxml

#!/bin/csh
#
## User name (Which account to be charged cpu time)
#$ -A user01
#
## Send email to users
#$ -M max@mustermann.de
#
## Send email at beginning/end/on suspension
#$ -m bes
#
## Export these environmental variables
#$ -v PVM_ROOT
#
## The job is located in the current working directory.
#$ -cwd
## Filenames for output and error log files
#$ -o myjobfile.out
#$ -e myjobfile.err
#
env >./xenv
mpirun -np $NSLOTS /share/apps/raxml/raxmlHPC-MPI-SSE3.icc -s sequencefile.phy -n outputfile.phy -m PROTGAMMAWAG

[edit] Exabayes

#!/bin/csh
#
## User name (Which account to be charged cpu time)
#$ -A user01
#
## Send email to users
#$ -M max@mustermann.de
#
## Send email at beginning/end/on suspension
#$ -m bes
#
## Export these environmental variables
#$ -v PVM_ROOT
#
## The job is located in the current working directory.
#$ -cwd
## Filenames for output and error log files
#$ -o myjobfile.out
#$ -e myjobfile.err
#
env >./xenv
mpirun -np $NSLOTS /share/apps/exabayes/exabayes -f aln.phy -q aln.part -n myRun -s 57913 -c config.nex -R 2 -C 2

[edit] Iq-Tree

#!/bin/csh
#
## User name (Which account to be charged cpu time)
#$ -A user01
#
## Send email to users
#$ -M max@mustermann.de
#
## Send email at beginning/end/on suspension
#$ -m bes
#
## Export these environmental variables
#$ -v PVM_ROOT
#
## The job is located in the current working directory.
#$ -cwd
## Filenames for output and error log files
#$ -o myjobfile.out
#$ -e myjobfile.err
#
env >./xenv
/share/apps/iqtree/iqtree-omp -omp $NSLOTS -s example.phy -m TEST
Personal tools
Namespaces

Variants
Actions
Navigation
Tools