Galloc issue

[calvinp0]

This pull request refactors how ARC handles memory allocation for jobs, standardizing all internal memory calculations to use base-2 (MiB/GiB) units and ensuring consistent, conservative memory requests across different cluster schedulers and Gaussian input files. It introduces explicit memory overhead factors and reserves headroom for non-Gaussian overhead, preventing over-allocation and improving stability, especially on capped nodes. The changes also update tests to reflect the new memory calculations.

Memory calculation and handling improvements:

• Standardized all internal memory math to use base-2 units (GiB/MiB), introducing constants like MEMORY_GB_TO_MIB, DEFAULT_JOB_MEMORY_OVERHEAD, and CAPPED_JOB_MEMORY_OVERHEAD for clarity and consistency.
• Refactored set_cpu_and_mem to always compute and store submit script memory in MiB, applying the appropriate overhead factor, and updated memory handling logic for different schedulers (SGE/OGE, PBS, Slurm) to use these MiB values.
• Added submit_script_memory_mib attribute throughout the codebase for clarity and updated initialization accordingly.

Gaussian-specific memory allocation:

• Added a fixed GAUSSIAN_MEMORY_HEADROOM_FRACTION (set to 0.90) so Gaussian jobs only consume a portion of the allocated memory, leaving room for scheduler and runtime overhead. Updated set_input_file_memory to use this fraction when setting %mem in Gaussian input files. [1] [2]

Test updates:

• Updated all affected tests to use the new base-2 memory calculations, adjusted expected values for %mem in Gaussian input files, and added assertions to verify the new logic for memory headroom and MiB calculations. [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] [22] [23]

[Copilot]

Pull request overview

Refactors ARC’s job memory handling to use consistent base-2 units (MiB/GiB), apply explicit overhead factors, and reserve headroom for Gaussian %mem to avoid galloc/memory failures on capped nodes.

Changes:

• Standardize internal submit-script memory calculations to MiB with explicit overhead factors and store submit_script_memory_mib.
• Apply a fixed headroom fraction when setting Gaussian %mem, using submit-script memory as the budget source.
• Update troubleshooting and unit tests to reflect the new memory cap/headroom behavior.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
arc/settings/submit.py	Adds clarifying comments around scheduler memory units/templates.
arc/job/adapter.py	Introduces MiB conversion/overhead constants and refactors scheduler memory setting logic.
arc/job/adapters/common.py	Initializes the new submit_script_memory_mib attribute.
arc/job/adapters/gaussian.py	Sets Gaussian %mem from submit-script memory with reserved headroom.
arc/job/trsh.py	Uses configured node-memory cap when attempting Gaussian memory troubleshooting; errors when capped.
arc/job/adapter_test.py	Updates expected scheduler memory values to match new MiB logic.
arc/job/adapters/gaussian_test.py	Updates expected %mem and asserts headroom / new MiB attribute behavior.
arc/job/trsh_test.py	Adds coverage for the “already at node-memory cap” Gaussian troubleshooting path.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

The comment above this submit script says it's a "sample PBS template" and refers to a mem={memory}mb PBS directive, but the actual script below is a Slurm #SBATCH template using --mem-per-cpu={memory}. Please update/remove the comment so it matches the scheduler directives in this block (or move it to the actual PBS template section).

Suggested change

	# ARC passes a base-2-derived MiB integer into this sample PBS
	# template. On our PBS systems, the `mb` suffix is interpreted in
	# base-2, so the directive stays `mem={memory}mb`.
	# ARC passes a base-2-derived MiB integer into this sample Slurm
	# template. The memory request is emitted via the Slurm directive
	# `#SBATCH --mem-per-cpu={memory}`.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 60.39%. Comparing base (70dab25) to head (b0459fd).

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #875      +/-   ##
==========================================
- Coverage   60.47%   60.39%   -0.08%     
==========================================
  Files         102      102              
  Lines       31102    31113      +11     
  Branches     8104     8104              
==========================================
- Hits        18808    18792      -16     
- Misses       9957     9975      +18     
- Partials     2337     2346       +9     

Flag	Coverage Δ
functionaltests	60.39% <ø> (-0.08%)	⬇️
unittests	60.39% <ø> (-0.08%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.