DIRAC job submission - missing bits

(Bruce Becker) #1

We have many use cases which assume that their workload management will be done with DIRAC. There is a catch-all instance at in production at https://dirac.egi.eu and a dev instance at https://dirac5.grid.cyfronet.pl/DIRAC/

There are actually two bits of DIRAC - the front end web interface and the back-end platform that talks to the grid:

  1. The front end talks OpenID Connect and AuthN/Z’s users who access the web interface
  2. The back end talks x.509 and AuthN/Z’s users who want to use platform functions - job submission, data catalogue, etc.

Also : the front end talks to the backend!

This is the missing bit - we need a piece of plumbing that gets a x.509 cert to the platform, when the user does things on the frontend. Clearly we want some interaction with the Master Portal here. A feature might be:

Feature: I can submit a job which needs an X.509 certificate, using only federated credentials
    Any user coming from certain federated identity providers can submit a job with DIRAC

Scenario: A user from EGI SSO can submit a job
  Given a known user authenticates to the DIRAC web interface
  Then they can see an authenticated view of it
  When they click on submit job
  Then a proxy certificate is retrieved by DIRAC from the MasterPortal

A typical flow could be implemented like this:

  1. User goes to web interface.
  2. Clicks on “log me in”
    1. Usual flow through Check In
    2. User has context in the front-end - service authorises the user based in some group management system.
  3. User wants to submit a job - clicks on “submit job” – currently, this requires uploading a cert or proxy.
    1. Frontend contacts Master Portal to request an x.509 proxy cert, sending the claims of the user
    2. Master Portal sends back a proxy cert to some end point of the frontend (callback url, e.g.)
  4. Proxy certificate is used to submit the job.
  5. Job runs.
  6. User wants to get the status. The proxy cert has now expired. Do 3 again.

Can we get some critique of this?

1 Like
(Bruce Becker) #2

@nliam was kind enough to provide the detailed flow which looks like this:

The DIRAC portal is the “VO Portal” in this diagram.

1 Like
(Mischa Salle) #3

Hi Bruce, Steps 3.1/.2 in your original post are not completely accurate. Step 3. should be initiating a OIDC flow towards the MasterPortal:

  • The MasterPortal (which functions as caching intermediate towards the RCauth online CA) would function as both an OIDC OP and also provide a protected endpoint /getproxy.
  • Dirac would function as OIDC client / relying party. It would initiate an OIDC flow to the MasterPortal. It would use the resulting access token to request a proxy from the /getproxy endpoint.

One technical point: the actual login at the MasterPortal is a second OIDC flow towards the RCauth online CA, where the user can login via e.g. EGI checkin, eduGAIN (iff R&S + Sirtfi). This sounds annoying/difficult, but it’s possible to ‘hint’ the MasterPortal from Dirac to always go the EGI Checkin. That would make that part essentially transparent from user perspective.

See for integration guidance:

some demo’s can be found at https://rcdemo.nikhef.nl/ In particular the gsiftp demo shows how a portal can obtain a proxy to – in that case – browse a storage element. One disclaimer/note: for the ‘VOMS’-enabled demo’s there, you (still) need to be manually enrolled in the VO. Let me know whether you run into trouble.

(Tsaregorodtsev) #4

Hi, Can we consider some other variations of the scenario above. When a user logs in to the DIRAC portal the Check-IN authentication is already done. Now to obtain the user proxy in order to submit jobs we can use the mechanism of PUSP proxy delivered to the DIRAC ProxyManager service (requested from a trusted IP). This mechanism was introduced in EGI a while ago in order to access cloud resources and its support was implemented in DIRAC already.

One question that I have is - are the proxies generated on the fly either through PUSP or from the Master Portal will be actually recognized by the resources providers, both computing and storage elements. This especially concerns the storage providers which require that each file has a personal ownership.

(Bruce Becker) #5

hi @Andrei ! thanks for the reply…

Let me start off with your question:

I generated such a proxy:

ssh proxy@ssh.aai.egi.eu > proxy.pem 
Connection to ssh.aai.egi.eu closed.
becker@tank ~> openssl x509 -in proxy.pem -noout -subject
subject=DC = eu, DC = rcauth, DC = rcauth-clients, O = IGTF, CN = Bruce Becker qQZS9z3MpXmIGPwS, CN = 1110102011, CN = 1309174257

That is a short-lived certificate issued to me, by RCAuth. I’m not sure what the other CN extensions are describing, maybe @nliam or @msalle can describe that. But it should be possible to do a dirac-proxy-init with that certificate. The storage endpoints should definitely support the RCAuth CA… I guess it would be easy to find out.

Can you tell me where the PUSP proxy is being generated? Is there a DIRAC instance now which is generating these proxies? How is this being done?

I don’t want to presume that I understand this, so allow me to ask some questions:

  1. I imagine that this is a call to a proxy generating service, which has a Robot Certificate in it. Correct?
  2. I imagine that this call is unauthenticated, and allowed only due to the strict 1-1 mapping between the portal and the proxy service - is this correct?
  3. I imagine that the link between the actual user and the robot proxy is done in a DIRAC tracking database internally to DIRAC, right? This was what was done with the Catania Science Gateway Framework at one point too

Maybe @glarocca has some idea about this.

(Tsaregorodtsev) #6

Hi Bruce,

ssh proxy@ssh.aai.egi.eu > proxy.pem 
Connection to ssh.aai.egi.eu closed.
becker@tank ~> openssl x509 -in proxy.pem -noout -subject
subject=DC = eu, DC = rcauth, DC = rcauth-clients, O = IGTF, CN = Bruce Becker qQZS9z3MpXmIGPwS, CN = 1110102011, CN = 1309174257

I guess this command should work with the public key uploaded to my CheckIN profile - is it correct ? For me this does not work. The subject is the proxy - is it changing each time you get the proxy ? I wonder is this GUID above qQZS9z3MpXmIGPwS regenerated with each new proxy ? If this is a stable subject then it can be probably used. However, dirac-proxy-init always starts from a certificate and not from a proxy for a good security reason. So, I do not see how this can solve the problem. The only way I see is that DIRAC gets hold of the user’s private key to connect to the ssh.aai.egi.eu on behalf of the user. This is also very questionable

As for the PUSP proxies, you can see how it is used here : https://wiki.egi.eu/wiki/Usage_of_the_per_user_sub_proxy_in_EGI

So, yes, it is generated by a service from a Robot certificate in response to an https request from a trusted IP. The resulting proxy also carries the user identification name and the resulting subject is treated by DIRAC as a user specific DN. This would allow users to authenticate to the DIRAC portal with their Check-IN identity and all the further transactions will be treated with the user PUSP proxy obtained by DIRAC internally without the need for the user to know about it. So, for the moment, to me this looks to be the simplest solution.

(Mischa Salle) #7

Hi Andrei, Bruce, there are now several discussions going through each other, which might make it a bit tricky to follow. Some technical points:

  • the proxies you can obtain from a MasterPortal are standard RFC proxies coming from the RCauth CA (https://rcauth.eu/)
  • the extra CN=... fields are just the extra CNs for the different proxy levels (for legacy proxies, that would say CN=proxy).
  • RCauth is IGTF approved under the IOTA profile and is supported under EOSC.
  • PUSP proxies are essentially RFC proxies from robot certificates, making use of the fact that in the RFC3820, the CN field of such an RFC proxy is basically opaque. As such only services that ‘understand’ PUSP will use that user information, all other services will treat them as being a robot user. This leads to very strict requirements on the infrastructure that is allowed to support them as such (see e.g. https://documents.egi.eu/public/ShowDocument?docid=2734)
  • To support IOTA CAs some extra (authorization) software is needed on the services, see EGI IGTF Release notes

Also note that the ssh command you’re showing is not the flow which I would integrate with Dirac, what I would like to integrate is that Dirac gets a full personal (as opposed to Robot) RFC proxy from a MasterPortal.

I’m afraid I need to run now, perhaps my (partial) answers clarify already a bit.

(Enol Fernández Del Castillo) #8

The problem with PUSP, as already hinted by @msalle, is that this relies on extending the CN field of the proxy in a way that may not be properly recognised by all services you need to interact with. This is a technology that fixed the lack of a properly supported Online CA in the infrastructure but now with RCAuth it makes little sense to keep relying on it.

The MasterPortal will give you a proxy with a stable user DN that will not change from one invocation to another and that can be understood at every service that accepts proxies in the infrastructure. The interaction to get it just involves a few HTTP calls and the user will need to consent but once that’s done you are good to go. There is no need at all to access any user private key!

(Tsaregorodtsev) #9

It would be nice if the MasterPortal will give DIRAC an individual user proxy with just “few HTTP calls”. I hope that DIRAC will not need to hold any user secret to be able to retrieve such proxies. Ideally, this should be as easy as getting PUSP proxies. Note that DIRAC needs to deliver user proxies to all user jobs and sometimes several times per job. For that DIRAC has a ProxyManager service pretty much like a MyProxy service which holds loooong user proxies and which is used to deliver shorter delegated proxies to the DIRAC components needing it. So, if we will be able to get from the MasterPortal long user proxies, then we will not put heavy loads on the MasterPortal afterwards as we will serve short proxies from the DIRAC ProxyManager. So, I am looking forward to see what is the exact procedure to follow to get user proxies from the MasterPortal. Note also that users will need to get proxies in a command line console (a la voms-proxy-init). This hopefully will be possible to do in an as simple way as in the Bruce’s ssh example.

(Baptiste Grenier) #10

As pointed by @msalle there are some documentation and demos, you may have a look at the following code example to see the code of an example of integration: https://rcdemo.nikhef.nl/demobasic/oidc_getproxy_demo_source.php

1 Like
(Bruce Becker) #11

Actually, I started this whole discussion to see how we could avoid forcing users to do this. It is DIRAC that should get the proxies from the Master Portal for the user, to avoid forcing the user to do things “out of band” – i.e. generate proxies on the command line and upload them to DIRAC. I would like to agree on this feature – regardless of whether it is implemented now. If we can agree that we would like users in principle to avoid having dealing with certificates by design. They should be free to choose which means of authentication to use.

This should be possible with the functionality of CheckIn, but I can’t find the right links to the docs.

1 Like
(Baptiste Grenier) #12

Isn’t it this link I previously posted? https://wiki.egi.eu/wiki/AAI_guide_for_SPs#Integrating_Science_Gateways_with_RCauth_for_obtaining_.28proxy.29_certificates

(Baptiste Grenier) #13

And yes one of the main objectives of this is to protect users from the cert pain, and Check-in with RCauth is meant to provide all the required services.

I think that AppDB VMops already can do this for Elixir users, using this AFAIK: https://github.com/IASA-GR/appdb-core/blob/master/application/controllers/AaimpxController.php

(Tsaregorodtsev) #14

Indeed this was not clear from my comment that we do not want to force user certificates. We must not require from users to deal with certificates, of course However, in order to work with DIRAC from a command line a user must have a proxy certificate recognized by DIRAC, this is part of the client/server protocol. So, we are considering a solution for an SSO user authentication invoked from the command line. As a result of the authentication operation, the user will get a certificate proxy locally which will be used for all the operations involving interactions with DIRAC services. From the user perspective there are no certificates visibly involved. So, I think we agree on this feature.

1 Like
(Baptiste Grenier) #15

And this also leads to some instructions with code, when using APIs: https://wiki.nikhef.nl/grid/RCAuth.eu_MasterPortal_VOPortal_integration_guide

(Baptiste Grenier) #16

Sounds great, so if I understand correctly you would get the proxy from RCauth too?

(Tsaregorodtsev) #17

We are considering both RCauth and/or PUSP proxies. For the DIRAC internal use a PUSP proxy generated by the DIRAC service itself should be enough. If any third party service will require some other proxy, e.g. RCauth, then we will have to revert to this one. I think we will have the first implementation with the PUSP solution using an internal DIRAC CA.

(Baptiste Grenier) #18

How would this internal DIRAC CA interact with other components such as HTCondor-CE and other X509 based service operated in the infra that a user may want to access? Will it work for “pure OIDC VOs” that don’t rely on VOMS and are expected to get proxies from RCauth and use them for all the x509 services to ge the same identity everywhere?

(Tsaregorodtsev) #19

The DIRAC CA certificates will not interact with services requiring X509. For computing element services, VOs are represented in DIRAC with a single “pilot” identity. This one should be a real X509 certificate issued by a recognized CA, but it si the only one such certificate per user community. Normal users will only need a DIRAC “internal” identity. For other possible services, e.g. storage elements, the access tokens will be dictated by those services. So, we will have to provide whatever they will require (tokens, RCauth, or DIRAC CA certificates if we will manage to convince them to recognize those :wink: ).

(Baptiste Grenier) #20

Hi Andrei, OK, thanks for the clarification. For the record we will have soon some discussions about the status of RCauth/Check-in and also on how all this may be ideally integrated (from an EGI point of view).