LSF - Get ID of submitted job

I

6

12

Say I submit a job using something like bsub pwd. Now I would like to get the job ID of that job in order to build a dependency for the next job. Is there some way I can get bsub to return the job ID?

Iridectomy answered 23/8, 2012 at 14:33 Comment(0)

I

8

Just as a reference, this is the best solution I could come up with so far. It takes advantage of the fact that bsub write a line containing the ID to STDOUT.

function nk_jobid {
    output=$($*)
    echo $output | head -n1 | cut -d'<' -f2 | cut -d'>' -f1
}

Usage:

jobid=$(nk_jobid bsub pwd)

Iridectomy answered 23/8, 2012 at 14:41 Comment(2)

@AndreyRubshtein thats bash. – Americium 8/1, 2016 at 22:19

I'm wondering how to pass a variable inside the $(nk_jonid bsub $VARIABLE) – Americium 8/1, 2016 at 22:22

S

9

Nils and Andrey have the answers to this specific question in shell and C/C++ environments respectively. For the purposes of building dependencies, you can also name your job with -J then build the dependency based on the job name:

bsub -J "job1" <cmd1>
bsub -J "job2" <cmd2>
bsub -w "done(job1) && done(job2)" <cmd>

There's a bit more info here.

This also works with job arrays:

bsub -J "ArrayA[1-10]" <cmd1>
bsub -J "ArrayB[1-10]" <cmd2>
bsub -w "done(ArrayA[3]) && done(ArrayB[5])" <cmd>

You can even do element-by-element dependency. The following job's i-th element will only run when the corresponding element in ArrayB reaches DONE status:

bsub -w "done(ArrayB[*])" -J "ArrayC[1-10]" <cmd3>

You can find more info on the various things you can specify in -w here.

Sternick answered 12/9, 2013 at 15:11 Comment(2)

(+1) Good thing you noticed the actual intent of the question. – Bayless 12/9, 2013 at 18:14

@jsmedmar Actually, it does. I'll add some detail to the answer. – Sternick 9/1, 2016 at 1:19

I

8

Just as a reference, this is the best solution I could come up with so far. It takes advantage of the fact that bsub write a line containing the ID to STDOUT.

function nk_jobid {
    output=$($*)
    echo $output | head -n1 | cut -d'<' -f2 | cut -d'>' -f1
}

Usage:

jobid=$(nk_jobid bsub pwd)

Iridectomy answered 23/8, 2012 at 14:41 Comment(2)

@AndreyRubshtein thats bash. – Americium 8/1, 2016 at 22:19

I'm wondering how to pass a variable inside the $(nk_jonid bsub $VARIABLE) – Americium 8/1, 2016 at 22:22

B

4

In case you are using C++, you can use the lsblib, LSF C API to submit jobs. The input and the output are structs. In particular, the output struct contains the job id.

#include <lsf/lsbatch.h>    
LS_LONG_INT lsb_submit (struct submit *jobSubReq, struct submitReply *jobSubReply)

Bayless answered 12/5, 2013 at 15:19 Comment(0)

W

1

If you just want to view the JOBID after submission, most of the time I will just use bhist or bhist -l to view the running jobs and details.

$ bhist
Summary of time in seconds spent in various states:
JOBID   USER    JOB_NAME  PEND    PSUSP   RUN     USUSP   SSUSP   UNKWN   TOTAL
8664    F14r3   sample       2       0    187954  0       0       0       187956

Whang answered 25/11, 2013 at 16:1 Comment(0)

M

0

$jobid = "0"
bsub pwd > $jobid
cat $jobid

Main answered 26/4, 2013 at 7:54 Comment(0)

G

0

you can put the job id into the name of the log file.

bsub -oo '%J' <rest of your command>

a log file will appear in the current directory that is the job id. You can then do what you want.

there are other values you can use besides %J. have a look at the bsub man page.

simple example:

> for job in {1..4} ; do bsub -oo '%J' sleep 1s ; done
> bjobs -o 'id cresreq' *
JOBID COMBINED_RESREQ
13217166 select[(type = any ) && (hi&&type == any )] order[r15s:pg]
13217167 select[(type = any ) && (hi&&type == any )] order[r15s:pg]
13217169 select[(type = any ) && (hi&&type == any )] order[r15s:pg]
13217170 select[(type = any ) && (hi&&type == any )] order[r15s:pg]

Goosefish answered 2/1 at 19:27 Comment(0)

Recommended topics

Hot tags