v3.260418

Hotfix for the current chaos: O(112k) publication files are stuck in Acquire, forming a baseline of terminally failed migrations. [#9299]

What's Changed

• Fix a few logging typos in gitlab cicd by @oljemark in #9298
• Mitigate terminally failed migration baseline by adding jitter delays between our Publisher DBS API calls. by @nausikt in #9300

Full Changelog: v3.260415...v3.260418

v3.260415

Main reason of the tag is to deploy the possibility for sites to require ASO via Rucio
as requested by Italian T2's

What's Changed

• Use python3 for wmc httpd by @belforte in #9291
• Add manual pipeline by @belforte in #9293
• deprecate (sidecar) crab-k8s-overlays, switching CI to mono-repo approach by @nausikt in #9279
• Allow sites to force ASO via Rucio. Fix #9294 by @belforte in #9297

Full Changelog: v3.260407...v3.260415

Validation

• validation pipeline https://gitlab.cern.ch/crab3/CRABServer/-/pipelines/14533059 OK

v3.260407

A technical release to try out the new CI test pipeline.
The only change to the code is the very minor fix for Rucio ASO.

What's Changed

• Tune claude-generated script by @belforte in #9289
• refactor test pipeline - version 2 by @belforte in #9287
• during Rucio ASO never wait >8h for running PJ. Fix $9079 by @belforte in #9290

Full Changelog: v3.260330...v3.260407

Validation

• automatic pipeline https://gitlab.cern.ch/crab3/CRABServer/-/pipelines/14467823 : OK

v3.260330

The purpose of this tag is to release Alma9-based CRABServer, TaskWorker for further testing.

What's Changed

• Use local disk to read groups info by @belforte in #9278
• Migration to Alma9 with py3.9 on CRABServer & TW by @nausikt in #9265
• Use timeless importlib for module importing by @nausikt in #9253
• add script from Claude by @belforte in #9286

Full Changelog: v3.260226...v3.260330

v3.260226

Quick patch for v3.260221 in which RenewRemoteProxies is not working.
It was the only place where we still had import htcondor instead if import htcondor2 as htcondor (same for classad)

What's Changed

• use v2 HTCondor API. so it works with HTC 25.x and later by @belforte in #9277

Full Changelog: v3.260221...v3.260226

v3.260221

Main reason is to capture change to min. requires CMSSW version for enabling token in job wrapper.
Other changes are mostly for build pipeline and cleanup and should not affect user submissions.
Yet we are still running v3.251015-stable so we better check this well and update production !

What's Changed

• drop GIT_SSL_CAINFO, GIT_EXEC_PATH to use system git instead. by @nausikt in #9263
• Add again dagman config file by @belforte in #9240
• remove obsolete scripts/Monitor by @belforte in #9266
• rename submission classads CUDA* to new HTCondor vendor agnostic, DESIRED_GPU*, refactored acceleratedParams interface. by @nausikt in #9258
• remove config examples now in puppet. by @belforte in #9267
• Fix python2/3 compatibility for Status Tracking with different CMSSW versions by @aspiringmind-code in #9268
• remove LDAP dependency by @belforte in #9269
• add script GetHighPriorityUsersFromCric.py for #9235 by @belforte in #9274
• only enable token in jobs for CMSSW >=15. For #9273 by @belforte in #9275

Full Changelog: v3.260116...v3.260221

Validation

• CICD pipeline https://gitlab.cern.ch/crab3/CRABServer/-/pipelines/14100592 - failed because some tasks were picked by tw04
• running again after stopping tw04 : pipeline https://gitlab.cern.ch/crab3/CRABServer/-/pipelines/14113733 OK
• pipeline with StatusTracking https://gitlab.cern.ch/crab3/CRABServer/-/pipelines/14118490 - failed most likely because tests started running while tasks had not started yet
• pipeline with Status Tracking only https://gitlab.cern.ch/crab3/CRABServer/-/pipelines/14121173 - failed due to #9213 not being fully solved yet
• I submitted manually all ST tasks for CMSSW_13 from my account and after a few resubmit all completed successfully inclluding publication - OK

v3.260116

The purpose of this tag is to revert resubmission refactoring, wrap-up changes made Since Oct 15. Put them into test after a while.

What's Changed

• workaround for remote proxy renewal failed in delegation mode. by @nausikt in #9256
• Revert "Refactoring resubmission (#9228)" by @aspiringmind-code in #9255
• add resubmissions study script by @belforte in #9259
• [PEP632] deprecate distutils module in py310 by @nausikt in #9260
• [PEP389] introducing of argparse and the deprecation of optparse, since py2.7+, py3.2+ by @nausikt in #9261

Full Changelog: v3.251216...v3.260116

v3.251216

Hotfixes, Follow-up patches of panic releases v3.251214, v3.251215. Actually nothing much goes wrongs with the new codes, Fatal cause by the corrupted image. Other than that is to make atomicReplaceAcrossFS py2 compatible with old CMSSW envs that use python2, downgrade htcondor.

What's Changed

• hotfix, forgot to deprecate retires, delays args from wrapper. by @nausikt in #9247
• make atomicReplaceAcrossFS a py2 compatible. by @nausikt in #9248
• downgrade htcondor to 25.0.3 by @nausikt in #9249

Full Changelog:

• v3.251213...v3.251214
• v3.251214...v3.251215
• v3.251215...v3.251216

v3.251213

This tag purpose is to bring retry/attempt mechanism around other RucioUtils fns, for mitigating CRON monit_reportrecallquota flooding emails to us during current Rucio overloading.

What's Changed

• add DAGMAN_CONFIG file by @belforte in #9236
• Revert "add DAGMAN_CONFIG file (#9236)" by @belforte in #9239
• Clean up all externalconfig references, centralconfig and conn_handlers. by @nausikt in #9237
• add safe lock on run_and_lumis, even in sequentially synchronized pla… by @nausikt in #9233
• Generic withExponentialRetry decorater derived/combined from/with Vijay's Robustified CRAB Rucio client wrapper for robustify monit report recall quota cron. by @nausikt in #9243
• bump htcondor to latest stable 25.4.0. by @nausikt in #9246

Full Changelog: v3.251119...v3.251213

v3.251119

This tag brings refactoring of resubmit, some condor logging and cleanup of obsolete files and arg

What's Changed

• Remove cacheurl arg from submit by @belforte in #9227
• Improve ht condor locator logging by @belforte in #9229
• Refactoring resubmission by @aspiringmind-code in #9228
• remove unused/obsolete files. Fix #9231 by @belforte in #9234

Full Changelog: v3.251024...v3.251119