CREAM-CE End Of Support Notice

This is a companion topic to the Broadcast message (below) regarding the end of support for CREAM.

CREAM-CE is a piece of middleware long-used for the submission of High-Throughput Compute workloads to arbitrary compute endpoints. There is not really a drop-in replacement, so @egi-ops has some work to do with the middleware providers to co-ordinate the replacement. We will need as much feedback from sites as possible, since this will have quite a big effect on their operating environment.

We would like to hear from site staff responsible for CREAM-CE’s -

  • how are you doing your configuration management?
  • What would you prefer to replace CREAM-CE with?
  • how would you rate the documentation and support channels of the alternative products?
  • What level of support would you expect from EGI Operations and middleware providers
  • what functionality of CREAM-CE would you consider a “deal breaker” if lost?

Let’s hear from you…

Dear Users

The CREAM working group has announced that official support for the CREAM-CE component will cease at the end of the EOSC-hub project, i.e. in Dec 2020. To prepare for this, EGI Foundation and CERN are actively working to help to minimise disruption. This will include helping users migrate to alternative solutions, i.e. ARC-CE or HTCondor-CE.

The CREAM working group will be providing full support until the end of 2019, including one minor release already scheduled. During 2020 only security updates will be released.

If you have any concerns or queries, please open a support ticket

Best regards

EGI Foundation and CERN Operations Teams


Hi Bruce, I’m part of the team managing the BEgrid-ULB-VUB site that is part of EGI through the NGI_NL federation. We are currently using CREAM-CE and our configuration management is done with Quattor. However, we have started moving to Puppet a few month ago, and as an exercise to learn this new tool, we have deployed a small HTCondor-CE testbed. We had many difficulties to make this HTCondor-CE work, and these were mainly due to the OSG dependencies and also to the security model of HTCondor. We also had a hard time finding the way to map user’s DN to pool account using LCMAPS instead of Argus. I don’t know exactly how we can help or contribute… it’s maybe too early. Best Regards, Stéphane Gérard

1 Like

Hi @stephane_gerard !

Thanks for the comment, and thanks for the getting in touch. Configuration management for this component is pretty important, and we worked during 2018 to early 2019 on developing re-usable components. Most of this work is done with Ansible roles – you can find the roles in the GitHub org. There has been quite a bit of work done as well by the CERN Ops community for Puppet, which might be more explicitly helpful to you.

I can also point you to the work by @mayank.sharma and his team. We had a long discussion around this last year, exactly around how to properly do the user and group mapping.

In general, I know that there is a plan in EGI to handle the migration from CREAM to HTCondor CE. Probably the best people to contact are @Matt and @spinoso

If you’re married to Puppet, the best thing would be to try to collaborate with the CERNOps team developing their modules for HTCondor CE and provide them guidelines that match EGI’s needs. @spinoso has a good list of acceptance criteria and knows who is working on what. I wish I could be more helpful, but alas, I am out of this game for the time being :soccer: !