fix(security): block SSRF via multimodal media URLs in load_file

[he-yufeng]

Fixes #9740 (GHSA-7mpg-jvqj-ccxr).

The bug

swift deploy runs the OpenAI-compatible server binding 0.0.0.0 with api_key=None by default, so it is unauthenticated and network-exposed. A multimodal chat request carries image_url / audio_url / video_url, and every one of them funnels into swift/template/vision_utils.py::load_file (load_image → load_file; _load_audio_librosa / _load_audio_soundfile_pyav → load_file; _resolve_video_local_path → load_file). load_file fetched any http(s) URL server-side with no target validation and followed redirects.

So an unauthenticated client can point a media URL at loopback, a private range, or the cloud instance-metadata endpoint (http://169.254.169.254/...) and read the response back. That is SSRF, and the reported 302 redirect bypass survives a naive host allowlist.

The fix

_assert_media_url_allowed(url) resolves the host and rejects it when any resolved address is private / loopback / link-local / reserved / multicast / unspecified, and rejects non-http(s) schemes. load_file now follows redirects manually and re-validates every hop, so a public URL cannot bounce into an internal address.

It is secure by default. A trusted deployment that legitimately serves media from an internal host can opt out with SWIFT_ALLOW_INTERNAL_MEDIA_URLS=1.

Tests

tests/general/test_vision_utils_ssrf.py covers the blocked ranges (metadata / loopback / private / unspecified / non-http scheme), the public-allowed case, and the opt-out.

Limitation

This validates each hop's resolved address, which stops the reported PoCs (direct metadata/loopback/private and the 302 bypass). It does not fully close a DNS-rebinding race between the resolution check and the socket connect; pinning the validated IP into the connection would be a reasonable follow-up, happy to add it here if you prefer.

[gemini-code-assist]

Code Review

This pull request introduces SSRF protection for multimodal media fetches by validating URLs and manually following redirects up to a limit, rejecting non-public IP addresses unless explicitly allowed via environment variables. It also adds corresponding unit tests. The review feedback highlights three important issues: a critical security bypass in _load_audio_librosa where fallback logic can bypass the SSRF check, a vulnerability to DNS rebinding attacks due to resolving the domain twice, and a potential KeyError if a redirect response lacks a Location header.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[gemini-code-assist]

Critical Security Bypass in _load_audio_librosa

While _assert_media_url_allowed successfully validates URLs, there is a critical bypass in _load_audio_librosa (lines 346-357).

In _load_audio_librosa, the call to load_file(audio) is wrapped in a catch-all except Exception: block:

    try:
        audio_io = load_file(audio)
        return librosa.load(audio_io, sr=sampling_rate, mono=mono)
    except Exception:
        if isinstance(audio, str) and audio.startswith(('http://', 'https://')):
            import audioread
            audio_io = audioread.ffdec.FFmpegAudioFile(audio)

If load_file raises a ValueError due to the SSRF check failing, the except Exception: block will catch it and fall back to fetching the URL directly using audioread.ffdec.FFmpegAudioFile(audio). This spawns an external ffmpeg process to fetch the blocked URL, completely bypassing the SSRF protection.

Remediation:
You should modify _load_audio_librosa to avoid catching the ValueError raised by the SSRF check, or explicitly run _assert_media_url_allowed(audio) before the try-except block if it is a URL. For example:

def _load_audio_librosa(audio: Union[str, bytes], sampling_rate: int, mono: bool = True):
    import librosa
    if isinstance(audio, str) and audio.startswith(('http://', 'https://')):
        _assert_media_url_allowed(audio)
    ...

[gemini-code-assist]

DNS Rebinding Vulnerability

The current implementation is vulnerable to a DNS rebinding attack. An attacker can configure a malicious DNS server for their domain to return a public IP address on the first resolution (during _assert_media_url_allowed), and then return a private/loopback IP address (like 127.0.0.1 or 169.254.169.254) on the second resolution (when session.get(url) is called).

Since requests performs its own DNS resolution internally when making the socket connection, it will resolve the domain again and connect to the private IP, completely bypassing the SSRF protection.

Remediation:
To fully close the DNS-rebinding race, you should pin the resolved and validated IP address for the connection. One way to achieve this in requests without breaking SSL/TLS verification is to implement a custom HTTPAdapter that overrides the connection resolution, or resolve the IP first and use a custom socket factory/resolver.

[gemini-code-assist]

If a server returns a redirect status code (3xx) but does not include a Location header, accessing response.headers['location'] will raise a KeyError and crash the application.

Using .get('location') and checking if it is present prevents this potential crash.

Suggested change

	for _ in range(_MAX_MEDIA_URL_REDIRECTS + 1):
	_assert_media_url_allowed(url)
	response = session.get(url, allow_redirects=False, **request_kwargs)
	if not response.is_redirect:
	break
	url = urljoin(url, response.headers['location'])
	for _ in range(_MAX_MEDIA_URL_REDIRECTS + 1):
	_assert_media_url_allowed(url)
	response = session.get(url, allow_redirects=False, **request_kwargs)
	if not response.is_redirect:
	break
	location = response.headers.get('location')
	if not location:
	break
	url = urljoin(url, location)

[he-yufeng]

Thanks for the careful review. Pushed ebee680b addressing all three points.

1. Audio fallback bypass (critical) — fixed. This was a real hole. _load_audio_librosa wraps load_file in except Exception, so the ValueError the guard raises for an internal URL got swallowed and the audioread/ffmpeg fallback then re-fetched the same URL unvalidated. I now validate the URL up front, before the try, so an internal audio_url is rejected before either path can run. Added a regression test that stubs librosa/audioread and asserts the ffmpeg fallback is never reached for a loopback URL (it fails against the previous commit). _load_audio_soundfile_pyav is not affected: its soundfile/pyav fallbacks operate on the BytesIO that load_file already returned, so they never re-fetch the URL.

2. DNS rebinding (high) — partly addressed, the rest scoped as a follow-up. You're right that validating the hostname and then letting the client re-resolve leaves a rebinding window. What this PR does cover: it re-validates on every redirect hop (so redirect-based pivots are closed) and it blocks every static internal target, which is the unauthenticated http://169.254.169.254/... / http://127.0.0.1/... case from the issue PoC. Fully closing rebinding needs connection-level pinning (resolve once, then connect to the validated IP while preserving SNI and the Host header), and I would rather land that as a focused, CI-verified change than rush it in here untested. Glad to fold it into this PR instead if you'd prefer it in scope.

3. Missing Location header (medium). In practice this could not KeyError: requests defines is_redirect as "location" in self.headers and status in REDIRECT_STATI, so the headers['location'] read was only reachable once the header was present. I switched it to .get('location') with an explicit break regardless, so it is now safe by inspection.

One unrelated note for convenience: the red unittest job is 4 pre-existing import errors in other modules (wandb.proto ... Imports, transformers.loss.loss_grounding_dino) with failures=0. None of them touch vision_utils, and the new test is hermetic (no deps, ffmpeg, or network).