Performance: Fix O(N^2) DB write bottleneck in start_model_runs.R for…

[anushkagupta200615-jpg]

Requested Labels: Type: Enhancement, Type: Performance

Description

This PR refactors the database stamping logic in start_model_runs.R for is_modellauncher workflows. Previously, a nested for loop executing PEcAn.DB::stamp_finished for every run in run_list was located inside the main job-checking while loop.

This PR removes that heavy database loop from the inner waiting-room loop and defers it. Now, the system waits for the overarching jobs to finish cleanly, and then iterates over run_list exactly once at the very end of the function to mark all runs as finished in the database.

Motivation and Context

This change is required to solve a massive $O(N^2)$ scaling bottleneck when dealing with large model ensembles. The previous redundant loop overhead would freeze up the workflow engine to query the database tens of thousands of times unnecessarily, adding extreme latency to modellauncher workflows.

Closes #4044

Review Time Estimate

• Immediately
• Within one week
• When possible

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

• My change requires a change to the documentation.
• My name is in the list of CITATION.cff
• I agree that PEcAn Project may distribute my contribution under any or all of
  • the same license as the existing code,
  • and/or the BSD 3-clause license.
• I have updated the CHANGELOG.md.
• I have updated the documentation accordingly.
• I have read the CONTRIBUTING document.
• I have added tests to cover my changes.
• All new and existing tests passed.