test: auto-retry ollama tests on transient ReadTimeout

[ajbozarth]

Pull Request

Issue

Fixes #1367

Description

Loaded CI runners occasionally stall a non-streaming Ollama request past the backend's 300 s read timeout, surfacing as httpx.ReadTimeout and failing unrelated PRs. A rerun reliably passes — the stall is a transient dead connection, not slow inference, so a fresh request succeeds. (Confirmed from run 28479500311: the failure fired exactly 300 s after the prior test, on a trivial "Say 'hi'" prompt against an already-warm model. The 300 s timeout was added in #1302 to convert a silent 15-minute hang into a prompt failure; this completes that "fail fast, then recover" intent.)

pytest_collection_modifyitems stamps a flaky marker on every ollama-marked test (no per-test annotations):

pytest.mark.flaky(reruns=2, reruns_delay=5, only_rerun="ReadTimeout")

only_rerun="ReadTimeout" means genuine assertion/logic failures still fail on the first run and are never masked. Retries are scoped to all ollama tests since the stall isn't telemetry-specific — telemetry is just the densest cluster of consecutive non-streaming calls. Widening to all e2e later is a one-line change to the marker gate.

Testing

• Tests added to the respective file if code was changed
• New code has 100% coverage if code was added
• Ensure existing tests and github automation passes (a maintainer will kick off the github automation when the rest of the PR is populated)

Verified: the real ReadTimeout flake reproduced locally and passed on auto-retry; a synthetic httpx.ReadTimeout retried twice while an injected AssertionError failed immediately with no rerun.

Attribution

• AI coding assistants used

---

Adding a new component, requirement, sampling strategy, or tool?

• Component
• Requirement
• Sampling Strategy
• Tool

[AngeloDanducci]

LGTM