qstat and long job names

Question

How can I get qstat to give me full job names?

I know qstat -r gives detailed information about the task, but it's too much and the resource requirements are included.

The qstat -r output is like:

 131806 0.25001 tumor_foca ajalali      qw    09/29/2014 15:49:41                                    1 2-100:1
       Full jobname:     tumor_focality-TCGA-THCA-ratboost_linear_svc
       Hard Resources:   distribution=wheezy (0.000000)
                         h_rt=72000 (0.000000)
                         mem_free=15G (0.000000)
                         h_vmem=15G (0.000000)
                         h_stack=256M (0.000000)
       Soft Resources:   
 131807 0.25001 vital_stat ajalali      qw    09/29/2014 15:49:41                                    1 2-100:1
       Full jobname:     vital_status-TCGA-LGG-ratboost_linear_svc
       Hard Resources:   distribution=wheezy (0.000000)
                         h_rt=72000 (0.000000)
                         mem_free=15G (0.000000)
                         h_vmem=15G (0.000000)
                         h_stack=256M (0.000000)
       Soft Resources:   

Right now my only option is to grep the output as I need:

$ qstat -r | grep "Full jobname" -B1
--
 131806 0.25001 tumor_foca ajalali      qw    09/29/2014 15:49:41                                    1 2-100:1
       Full jobname:     tumor_focality-TCGA-THCA-ratboost_linear_svc
--
 131807 0.25001 vital_stat ajalali      qw    09/29/2014 15:49:41                                    1 2-100:1
       Full jobname:     vital_status-TCGA-LGG-ratboost_linear_svc

Can I do it better to have a nicer output?

Tusche

To get the full job names of all the actual jobs of a given user: qstat -f | grep -C 1 username@ You can get more information with -C 2, -C 3, etc. – Occlude

Answer 1

This on is a bit messy, but it works as a simple solution to have in the command history. All standard tools. Output is pretty much the same as what you get from a normal qstat call, but you won't get the headers:

One-liner:

qstat -xml | tr '\n' ' ' | sed 's#<job_list[^>]*>#\n#g' \
  | sed 's#<[^>]*>##g' | grep " " | column -t

Description of commands:

List jobs as XML:

qstat -xml

Remove all newlines:

tr '\n' ' '

Add newline before each job entry in the list:

sed 's#<job_list[^>]*>#\n#g'

Remove all XML stuff:

sed 's#<[^>]*>##g'

Hack to add newline at the end:

grep " "

Columnize:

column -t

Example output

351996  0.50502  ProjectA_XXXXXXXXX_XXXX_XXXXXX                user123  r   2015-06-25T15:38:41  [email protected]  1
351997  0.50502  ProjectA_XXX_XXXX_XXX                         user123  r   2015-06-25T15:39:26  [email protected]  1
351998  0.50502  ProjectA_XXXXXXXXXXXXX_XXXX_XXXX              user123  r   2015-06-25T15:40:26  [email protected]  1
351999  0.50502  ProjectA_XXXXXXXXXXXXXXXXX_XXXX_XXXX          user123  r   2015-06-25T15:42:11  [email protected]  1
352001  0.50502  ProjectA_XXXXXXXXXXXXXXXXXXXXXXX_XXXX_XXXX    user123  r   2015-06-25T15:42:11  [email protected]  1
352008  0.50501  runXXXX69                                     usr1     r   2015-06-25T15:49:04  [email protected]  1
352009  0.50501  runXXXX70                                     usr1     r   2015-06-25T15:49:04  [email protected]  1
352010  0.50501  runXXXX71                                     usr1     r   2015-06-25T15:49:04  [email protected]  1
352011  0.50501  runXXXX72                                     usr1     r   2015-06-25T15:49:04  [email protected]  1
352012  0.50501  runXXXX73                                     usr1     r   2015-06-25T15:49:04  [email protected]  1
352013  0.50501  runXXXX74                                     usr1     r   2015-06-25T15:49:04  [email protected]  1

Cheslie

It works, but it's poorly formatted, and has a bunch of extra whitespaces and new lines. It'll be nice if you could fix them and give an example output. – Tusche

That was strange. I've added an example output, I have no extra whitespaces and newlines here. But if your terminal window isn't wide enough, then it will wrap to a new line since the wish (as I understood it) was to not truncate the job names. It could be that the the solution is a bit fragile and does not work that well on other nix distros/sge versions? (I have SGE 8.1.4 and CentOS 5.11 and 6.6, bash 3.2, zsh 5.0.5) – Cheslie

Great answer! Creating an alias takes some effort to escape quotation marks, so I post it here: alias detqstat='qstat -xml | tr '"'"'\n'"'"' '"'"' '"'"' | sed '"'"'s#<job_list[^>]*>#\n#g'"'"' | sed '"'"'s#<[^>]*>##g'"'"' | grep " " | column -t' – Gershom

Nice. I was able to create a clean alias without the necessity to escape characters by replacing double quotes with single quotes in the grep command: alias qstata="qstat -xml | tr '\n' ' ' | sed 's#<job_list[^>]*>#\n#g' | sed 's#<[^>]*>##g' | grep ' ' | column -t" – Bambara

For me, this didn't include the column headings like qstat alone does. – Rumormonger

Answer 2

Maybe an easier solution: set SGE_LONG_JOB_NAMES to -1, and qstat will figure out the size of the name column:

export SGE_LONG_JOB_NAMES=-1
qstat -u username

Works for me.

Cheers!

Boydboyden

In what version was the introduced? The system we use has SGE 6.2u5, the man page for qstat only lists SGE_LONG_QNAMES – Feisty

@Jonno_FTW: I used this on Univa Grid Engine (UGE) 8.5.0. – Boydboyden

Answer 3

This script works pretty well. It looks like it is from cambridge. http://www.hep.ph.ic.ac.uk/~dbauer/grid/myqstat.py

For Python 3:

#!/usr/bin/python
import xml.dom.minidom
import os
import sys
import string    

f=os.popen('qstat -u \* -xml -r')

dom=xml.dom.minidom.parse(f)


jobs=dom.getElementsByTagName('job_info')
run=jobs[0]

runjobs=run.getElementsByTagName('job_list')


def fakeqstat(joblist):
    for r in joblist:
        try:
            jobname=r.getElementsByTagName('JB_name')[0].childNodes[0].data
            jobown=r.getElementsByTagName('JB_owner')[0].childNodes[0].data
            jobstate=r.getElementsByTagName('state')[0].childNodes[0].data
            jobnum=r.getElementsByTagName('JB_job_number')[0].childNodes[0].data
            jobtime='not set'
            if(jobstate=='r'):
                jobtime=r.getElementsByTagName('JAT_start_time')[0].childNodes[0].data
            elif(jobstate=='dt'):
                jobtime=r.getElementsByTagName('JAT_start_time')[0].childNodes[0].data
            else:
                jobtime=r.getElementsByTagName('JB_submission_time')[0].childNodes[0].data

            print(jobnum, '\t', jobown.ljust(16), '\t', jobname.ljust(16),'\t', jobstate,'\t',jobtime)
        except Exception as e:
            print(e)

fakeqstat(runjobs)

For Python 2:

#!/usr/bin/python
import xml.dom.minidom
import os
import sys
import string
#import re


f=os.popen('qstat -u \* -xml -r')

dom=xml.dom.minidom.parse(f)


jobs=dom.getElementsByTagName('job_info')
run=jobs[0]

runjobs=run.getElementsByTagName('job_list')


def fakeqstat(joblist):
        for r in joblist:
                jobname=r.getElementsByTagName('JB_name')[0].childNodes[0].data
                jobown=r.getElementsByTagName('JB_owner')[0].childNodes[0].data
                jobstate=r.getElementsByTagName('state')[0].childNodes[0].data
                jobnum=r.getElementsByTagName('JB_job_number')[0].childNodes[0].data
                jobtime='not set'
                if(jobstate=='r'):
                        jobtime=r.getElementsByTagName('JAT_start_time')[0].childNodes[0].data
                elif(jobstate=='dt'):
                        jobtime=r.getElementsByTagName('JAT_start_time')[0].childNodes[0].data
                else:
                        jobtime=r.getElementsByTagName('JB_submission_time')[0].childNodes[0].data



                print  jobnum, '\t', jobown.ljust(16), '\t', jobname.ljust(16),'\t', jobstate,'\t',jobtime


fakeqstat(runjobs)

Inkberry

Thanks. I changed your copy/pasted code to support python3. It works now. – Tusche

@Inkberry os.popen will be replaced in the future, but I can't make your solution work with subprocess.Popen: do you know why? – Anticipate

Answer 4

I am currently writing my own qstat wrapper in order to get a clean, useful and customizable output.

Here is the github repository. The project has grown too much for the code to be pasted in this message.

It comes with an installer and should work without any problem with both Python 2.7 and 3 (the installation script makes the modifications if needed). qjobs -h provides some help on the available options. I will write a more complete documentation in the following days on the github wiki.

I will update this message as often as possible to stick to the current state of the project. Please feel free to comment here (or on github) to ask for features/report problems.

In the near future, I will try to add a fully interactive mode to browse the job list more easily. Of course, the classic text output will be still available (it could be useful to e-mail the output, or for a quick check of the pending/running jobs).

Example output

Command qjobs gives:

5599109   short_name        r    2015-06-25 10:27:39   queue1
5599110   jobName           r    2015-06-25 10:35:39   queue2
5599111   a_long_job_name   qw   2015-06-25 10:40:39
5599112   foo               qw   2015-06-25 10:40:39
5599113   bar               qw   2015-06-25 10:40:39
5599114   baz               qw   2015-06-25 10:40:39
5599115   beer              qw   2015-06-25 10:40:39

tot: 7

r: 2   qw: 5

Command qjobs -o gives:

tot: 7

r: 2   qw: 5

Command qjobs -o inek -t gives (e is elapsed time since start/sub time, the format is customizable using the Format Spec. Mini-Language of Python; k is complete queue name, with domain):

5598985   SpongeBob        522:02 (21.75 days)   [email protected]
5598987   ping_java        521:47 (21.74 days)   [email protected]
5598988   run3.14          521:46 (21.74 days)   [email protected]
5598990   strange_job_42   521:42 (21.74 days)   [email protected]
5598991   coffee-maker     521:39 (21.74 days)   [email protected]
5598992   dumbtask         521:29 (21.73 days)   [email protected]

qjobs -i gives a complete list of the available 'items'. Each of this item is available as:

• a column output (with -o ITEMS);
• as a criteria to count the job and produces total output, with -t (e.g. -t s to count by state as in the two first examples);
• as a criteria to sort the job with -s, default is -s ips meaning that the job list is sorted by ID, then by priority and finally by state before being printed.

The result of qjobs -i is:

i: job id
p: job priority
n: job name
o: job owner
s: job state
t: job start/submission time
e: elapsed time since start/submission
q: queue name without domain
d: queue domain
k: queue name with domain
r: requested queue(s)
l: number of slots used

Bitt

Answer 5

Thanks to JLT for nice simple code. I've expanded it a bit to fit my needs and make it look nice.

Sample Output:

Job ID             Job Name                   Owner   Status  
------  ------------------------------------  ------  ------  
201716  AtacSilN100400K                       mtsige  R       
201771  IsoOnGrap400K                         mtsige  R       
202067  AtacOnSilica400K                      mtsige  R       
202100  AtacGrapN100400K                      mtsige  R       
202135  AtacOnSilc400K                        mtsige  R       
202145  AtacOnGrap400K                        mtsige  R       
202152  AtacOnGraphN3360K                     mtsige  R       
202161  AtacticSilicaN10                      mtsige  R       
202163  AtacGrapN10                           mtsige  R       
202169  AtacSilcN10                           mtsige  R       
202192  wallpmma07                            am110   R       
202193  wallpmma03                            am110   R       
202194  att03wpm_95solps                      am110   R       
202202  AtacticSilicaN3                       mtsige  R       
203260  8test18_trop_2p                       ico     R       
203359  parseAll_Bob/Sub951By50/Cyl20A_atom1  oge1    R       
203360  parseAll_Bob/Sub951By50/Cyl30A_atom1  oge1    R       
203361  parseAll_Bob/Sub951By50/Cyl30A_atom2  oge1    R      

Code:

#!/opt/bin/python3
import os
import xml.etree.ElementTree as ET

#Fields
fields=['Job_Id','Job_Name','Job_Owner','job_state']
names=['Job ID','Job Name','Owner','Status']

#Get job info
f = os.popen('qstat -x')
tree = ET.parse(f)
root = tree.getroot()
n_fields=len(fields)
jobs=[[job.find(field).text for field in fields] for job in root]
max_lengths=[len(name) for name in names]
sep='  '

#Identify max characer length per field
for j in jobs:
    for i in range(n_fields):
            #Chop off anything after and including '@' or '.' from all fields
            if j[i].find('@')>0:
                    j[i]=j[i][:j[i].find('@')]
            if j[i].find('.')>0:
                    j[i]=j[i][:j[i].find('.')]
            if(len(j[i])>max_lengths[i]):
                    max_lengths[i]=len(j[i])

#Field names
for i in range(n_fields):
    print('{s:^{length}}'.format(s=names[i],length=max_lengths[i]),end=sep)
print()

#Dashes
for i in range(n_fields):
    print('-'*max_lengths[i],end=sep)
print()

#Jobs
for j in jobs:
    for i in range(n_fields):
            if j[i].find('@')>0:
                    j[i]=j[i][:j[i].find('@')]
            print('{s:<{length}}'.format(s=j[i],length=max_lengths[i]),end=sep)
    print()

Polack

Answer 6

For me the script of Physical Chemist didn't work so I wrote a very simple script using the xml.tree.ElementTree module which i regard as somewhat easier than xml.dom.minidom

import os
import xml.etree.ElementTree as ET
f = os.popen('qstat -x')
tree = ET.parse(f)
root = tree.getroot()
print "Job_Id   walltime state     nodes       Job_Name"
print "------   -------- ----- --------------- --------------------------"
for job in root:
    print job.find('Job_Id').text, " ",
    print job.find('resources_used').find('walltime').text, " ",
    print job.find('job_state').text, " ",
    print job.find('Resource_List').find('nodes').text, " ",
    print job.find('Job_Name').text

Cruiser

Answer 7

If you just want the names:

qstat -f | grep 'Job_Name'

Example of output:

Job_Name = File.output
Job_Name = file.out

Mcmahan

Answer 8

A poor KISS solution :

qstat -xml -f -u \* | fgrep JB_name | wc -l

Subclavius

Answer 9

python code

import xmltodict
import subprocess as sp
import pandas as pd

qstat_xml = sp.check_output(['qstat','--xml'], stderr=sp.STDOUT)  # read xml
stat_dict = xmltodict.parse(qstat_xml) # convert to dict
job_list = stat_dict['Data']['Job'] # select job_list
job_df = pd.DataFrame(job_list) # convert to dataframe
print('columns', job_df.columns) # print available columns
column_list = ['Job_Id', 'Job_Name']
selection_df = job_df[column_list]  # select columns
print(selection_df)

Diazole