[PULP-1668] Optimize re-sync with content caching

[jobselko]

Related to #7558

📜 Checklist

• Commits are cleanly separated with meaningful messages (simple features and bug fixes should be squashed to one commit)
• A changelog entry or entries has been added for any significant changes
• Follows the Pulp policy on AI Usage
• (For new features) - User documentation and test coverage has been added

See: Pull Request Walkthrough

[dralley]

Looks good! A few things to look at, maybe we can make this even more efficient

[jobselko]

Measured benefit using https://fixtures.pulpproject.org/file-perf/ (20k files) with immediate policy sync:

Without patch:
Sync time: 508 seconds
Re-sync time: 569 seconds

With patch:
Sync time: 513 seconds
Re-sync time: 68 seconds

The biggest impact comes from the QueryExistingArtifacts fix.

Syncing with on-demand policy is already fast and is not significantly affected by this change (Sync time: 92 seconds, Re-sync time: 42 seconds).

[jobselko]

@dralley I will squash the commits before merging, but I am keeping them separate for easier review now.

[dralley]

@jobselko Keep the QueryExistingArtifacts commit separate and split it off into an independent PR. I think we might end up backporting that one, because it's very self-contained and has a high impact. Satellite might want it.

The rest can be squashed and stay in this PR.

Great work! This is a huge performance improvement 🚀 👍

[dralley]

@jobselko Can you squash commits 1, 2, and 4, and address the merge conflict?

I think it's just the wip - reduce redundant queries that needs more attention before merge. We could make that a separate PR also, to get everything else merged today for the release?

[jobselko]

Discussed with @dralley that we are not going to merge this today.