Extracting site information from Operations Portal (Compute Elements)

bdii
lavoisier
queries
(Bruce Becker) #1

Problem : I would like to get the distribution graph of local resource management systems (LRMS) at sites. IE, something like :

  • Torque: 100 sites * 600000 cores
  • LSF: 2 sites * 10000 cores
  • HTCondor: 16 sites * 350000 cores

I could do this by querying a top-level information source like the GOC, top-bdii or operations portal. Of all of these, I think the operations portal may have the easiest API to work with.

@Cyril_Lorphelin - can you suggest which parts of the Operations Portal I can query to get this?

(Bruce Becker) #2

Poking about on the EGI wiki I found the trick to do this with an LDAP search

ldapsearch -x -LLL -H ldap://lcg-bdii.cern.ch:2170  \ #search a top-level bdii
-b mds-vo-name=local,o=grid \ #using the GLUE-1.3 information tree
'glueceinfojobmanager' |\ # selecting the GlueCEInfoJobManager attribute
grep GlueCEInfoJobManager |\ #extract only those lines from the huge stream of text
uniq -c |\ # Count the unique entries numerically
sort -g |\ # sort the result numerically
awk '{print $1 " " $3}' # and only print the first and second fields.
Number CE Type
2 AliEn
5 HTCondorCE
57 condor
106 slurm
161 sge
197 arc
433 lsf
1162 pbs
(Cyril Lorphelin) #3

I’ve added a new method to the API to get values for LRMS .

I’ve lot of differences with your query .

In glue 2 , here is the summary :

http://ccosvms0089.in2p3.fr:8080/lavoisier/VAPOR_LRMS?accept=html

options : site , country and details

By default this is a summary by lrms, but you can have the details of the service with the use of details=yes

http://ccosvms0089.in2p3.fr:8080/lavoisier/VAPOR_LRMS?details=yes&accept=json

http://ccosvms0089.in2p3.fr:8080/lavoisier/VAPOR_LRMS?site=IN2P3-CC&accept=json

http://ccosvms0089.in2p3.fr:8080/lavoisier/VAPOR_LRMS?country=italy&accept=xml

1 Like