Strip credentials from normalized Git remote URLs#36
Conversation
| # Query-string parameter names that carry authentication material. When a | ||
| # remote URL retains any of these they are dropped during normalization so | ||
| # secrets are never persisted, displayed, logged, or embedded in error text. | ||
| _SENSITIVE_QUERY_KEYS: frozenset[str] = frozenset({ |
There was a problem hiding this comment.
(AI-assisted)
Could we fail closed by dropping the query from normalized Git metadata, or reuse the broader sanitizer predicate?
Exact matching here misses common credential keys such as oauth_token and X-Amz-Signature, so their values can still reach the persisted/displayed URL despite the hard-guarantee claim. Since this normalized remote is used to build repository browse links, dropping the query entirely seems safest; alternatively, centralize the broader token/credential and signed-URL matching used in the related transport sanitizer work. Could we also add regression cases for these two keys?
| stripped = f"https://{scp.group(1)}/{scp.group(2)}" | ||
|
|
||
| parts = urlsplit(stripped) | ||
| if parts.scheme and parts.hostname is not None: |
There was a problem hiding this comment.
(AI-assisted)
Could this parser fail closed for URL-like inputs?
Missing-host and scheme-relative values currently fall through with user:secret@ intact, while an invalid port raises when parts.port is read. Because this helper also handles server-supplied annotations and --git-remote-url, malformed input should produce an omitted/redacted link rather than returning credential-bearing text or raising. Could we wrap URL/host/port parsing and add regression cases for both the fail-open and invalid-port paths?
Volv-G
left a comment
There was a problem hiding this comment.
Approved after review. I left two AI-assisted hardening comments on the URL sanitizer edge cases.
Git remote URLs can embed credentials in their userinfo (https://user:token@host/...), most often from CI checkout URLs or a caller-supplied --git-remote-url. Normalization previously rewrote only SSH/SCP forms and stripped the .git suffix, so credentials in an http(s) URL flowed unchanged into published component annotations, CLI output, logs, and browse links. Make credential removal a guarantee of URL normalization. _normalize_git_url now parses the URL and rebuilds the netloc from host and port only, dropping all userinfo, converts ssh:// to https://, drops userinfo from SCP-style remotes, strips the .git suffix, and redacts sensitive query parameters. Query redaction is fail-closed: a broadened key predicate drops credential -shaped parameters such as oauth_token or X-Amz-Signature in addition to the known keys, while benign params (ref, path) are preserved. URL parsing is also fail-closed. Scheme-relative remotes have their userinfo stripped; a URL-like input whose credential-bearing authority cannot be parsed into a clean host (missing host with userinfo, malformed IPv6) returns a redacted placeholder rather than leaking the raw user:secret@ text or raising; and a malformed textual port is dropped while the credential-free host is kept. ComponentPublisher sanitizes a caller-supplied git_remote_url at construction, and component_inspector sanitizes before emitting transparency reasons and building browse links. Scheme, host, port, and path are preserved; only credential material is removed, and clone/fetch authentication is unaffected. The function is idempotent.
d0b012e to
1687ba8
Compare
Summary
Git remote URLs can embed authentication material in their userinfo (
https://user:token@host/...) — most commonly via CI checkout URLs (e.g. GitLab'shttps://gitlab-ci-token:<token>@gitlab.com/...) or a caller-supplied--git-remote-url. Previously_normalize_git_urlonly rewrote SSH/SCP forms and stripped the.gitsuffix, so credentials in anhttp(s)URL flowed unchanged into published component annotations, CLI output, logs, transparency reasons, and generated browse links. This change makes credential removal a hard guarantee of URL normalization._normalize_git_url(utils.py) is now the canonical sanitizer: it parses the URL, rebuilds the netloc from host + port only (dropping alluser[:password]@userinfo), convertsssh://tohttps://, drops userinfo from SCP-style remotes, strips the.gitsuffix, and redacts sensitive query parameters. It is idempotent.oauth_token,X-Amz-Signature,X-Amz-Credential,X-Amz-Security-Token, and similar future/provider keys) in addition to an explicit known-key set, while short/ambiguous keys (key,sig,auth) stay exact-match only so benign params likeref/pathare preserved.//user:secret@host/...) have their userinfo stripped; a URL-like input whose credential-bearing authority cannot be parsed into a clean host (missing host with userinfo, malformed IPv6) returns a redacted placeholder rather than leaking the rawuser:secret@text or raising; a malformed textual port is dropped while the credential-free host is kept.ssh://, SCP-styleuser@host:path,file://, bare local paths, and Windows drive paths without corrupting valid input.ComponentPublishersanitizes the resolvedgit_remote_urlat construction, so a caller-supplied--git-remote-url(which bypassesget_git_info's normalization) can never persist credentials into annotations.component_inspectorsanitizesgit_remote_urlbefore emitting it in transparency reasons and before building browse links, as defense-in-depth for annotations that arrive from the server.Context
Scheme, host, port, path, and fragment are preserved; only credential material is removed. Actual clone/fetch authentication is unaffected — sanitization applies to the persisted or displayed value, not to any network operation.
Testing
uv run pytest tests/test_component_inspector.py tests/test_component_publisher.py tests/test_utils.py@assertions, sensitive query-param redaction, fail-closed dropping of unknown credential-shaped query keys (oauth_token,X-Amz-*), and the malformed URL-like fail-closed paths (missing host with userinfo, scheme-relative userinfo, invalid textual port, malformed IPv6), plus preservation/idempotency for credential-free URLs and local pathsuv lock --check;git diff --check