Skip to content

Strip credentials from normalized Git remote URLs#36

Open
arseniy-pplx wants to merge 1 commit into
TangleML:masterfrom
arseniy-pplx:transfer/redact-git-url-credentials
Open

Strip credentials from normalized Git remote URLs#36
arseniy-pplx wants to merge 1 commit into
TangleML:masterfrom
arseniy-pplx:transfer/redact-git-url-credentials

Conversation

@arseniy-pplx

@arseniy-pplx arseniy-pplx commented Jul 20, 2026

Copy link
Copy Markdown

Summary

Git remote URLs can embed authentication material in their userinfo (https://user:token@host/...) — most commonly via CI checkout URLs (e.g. GitLab's https://gitlab-ci-token:<token>@gitlab.com/...) or a caller-supplied --git-remote-url. Previously _normalize_git_url only rewrote SSH/SCP forms and stripped the .git suffix, so credentials in an http(s) URL flowed unchanged into published component annotations, CLI output, logs, transparency reasons, and generated browse links. This change makes credential removal a hard guarantee of URL normalization.

  • _normalize_git_url (utils.py) is now the canonical sanitizer: it parses the URL, rebuilds the netloc from host + port only (dropping all user[:password]@ userinfo), converts ssh:// to https://, drops userinfo from SCP-style remotes, strips the .git suffix, and redacts sensitive query parameters. It is idempotent.
  • Query redaction is fail-closed. A broadened key predicate drops credential-shaped parameters (oauth_token, X-Amz-Signature, X-Amz-Credential, X-Amz-Security-Token, and similar future/provider keys) in addition to an explicit known-key set, while short/ambiguous keys (key, sig, auth) stay exact-match only so benign params like ref/path are preserved.
  • URL parsing is fail-closed. Scheme-relative remotes (//user:secret@host/...) have their userinfo stripped; a URL-like input whose credential-bearing authority cannot be parsed into a clean host (missing host with userinfo, malformed IPv6) returns a redacted placeholder rather than leaking the raw user:secret@ text or raising; a malformed textual port is dropped while the credential-free host is kept.
  • Handles percent-encoded userinfo, username-only, password-only/empty-username, IPv6 literals, explicit ports, ssh://, SCP-style user@host:path, file://, bare local paths, and Windows drive paths without corrupting valid input.
  • ComponentPublisher sanitizes the resolved git_remote_url at construction, so a caller-supplied --git-remote-url (which bypasses get_git_info's normalization) can never persist credentials into annotations.
  • component_inspector sanitizes git_remote_url before emitting it in transparency reasons and before building browse links, as defense-in-depth for annotations that arrive from the server.

Context

Scheme, host, port, path, and fragment are preserved; only credential material is removed. Actual clone/fetch authentication is unaffected — sanitization applies to the persisted or displayed value, not to any network operation.

Testing

  • uv run pytest tests/test_component_inspector.py tests/test_component_publisher.py tests/test_utils.py
  • added parametrized tests covering credential stripping (percent-encoded, token-only, password-only, IPv6, ports, GitLab CI token URLs), explicit no-secret / no-@ assertions, sensitive query-param redaction, fail-closed dropping of unknown credential-shaped query keys (oauth_token, X-Amz-*), and the malformed URL-like fail-closed paths (missing host with userinfo, scheme-relative userinfo, invalid textual port, malformed IPv6), plus preservation/idempotency for credential-free URLs and local paths
  • uv lock --check; git diff --check

@arseniy-pplx
arseniy-pplx marked this pull request as ready for review July 20, 2026 18:11
# Query-string parameter names that carry authentication material. When a
# remote URL retains any of these they are dropped during normalization so
# secrets are never persisted, displayed, logged, or embedded in error text.
_SENSITIVE_QUERY_KEYS: frozenset[str] = frozenset({

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(AI-assisted)

Could we fail closed by dropping the query from normalized Git metadata, or reuse the broader sanitizer predicate?

Exact matching here misses common credential keys such as oauth_token and X-Amz-Signature, so their values can still reach the persisted/displayed URL despite the hard-guarantee claim. Since this normalized remote is used to build repository browse links, dropping the query entirely seems safest; alternatively, centralize the broader token/credential and signed-URL matching used in the related transport sanitizer work. Could we also add regression cases for these two keys?

stripped = f"https://{scp.group(1)}/{scp.group(2)}"

parts = urlsplit(stripped)
if parts.scheme and parts.hostname is not None:

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(AI-assisted)

Could this parser fail closed for URL-like inputs?

Missing-host and scheme-relative values currently fall through with user:secret@ intact, while an invalid port raises when parts.port is read. Because this helper also handles server-supplied annotations and --git-remote-url, malformed input should produce an omitted/redacted link rather than returning credential-bearing text or raising. Could we wrap URL/host/port parsing and add regression cases for both the fail-open and invalid-port paths?

@Volv-G Volv-G left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved after review. I left two AI-assisted hardening comments on the URL sanitizer edge cases.

Git remote URLs can embed credentials in their userinfo
(https://user:token@host/...), most often from CI checkout URLs or a
caller-supplied --git-remote-url. Normalization previously rewrote only
SSH/SCP forms and stripped the .git suffix, so credentials in an http(s)
URL flowed unchanged into published component annotations, CLI output,
logs, and browse links.

Make credential removal a guarantee of URL normalization. _normalize_git_url
now parses the URL and rebuilds the netloc from host and port only, dropping
all userinfo, converts ssh:// to https://, drops userinfo from SCP-style
remotes, strips the .git suffix, and redacts sensitive query parameters.
Query redaction is fail-closed: a broadened key predicate drops credential
-shaped parameters such as oauth_token or X-Amz-Signature in addition to the
known keys, while benign params (ref, path) are preserved.

URL parsing is also fail-closed. Scheme-relative remotes have their userinfo
stripped; a URL-like input whose credential-bearing authority cannot be
parsed into a clean host (missing host with userinfo, malformed IPv6) returns
a redacted placeholder rather than leaking the raw user:secret@ text or
raising; and a malformed textual port is dropped while the credential-free
host is kept. ComponentPublisher sanitizes a caller-supplied git_remote_url
at construction, and component_inspector sanitizes before emitting
transparency reasons and building browse links. Scheme, host, port, and path
are preserved; only credential material is removed, and clone/fetch
authentication is unaffected. The function is idempotent.
@arseniy-pplx
arseniy-pplx force-pushed the transfer/redact-git-url-credentials branch from d0b012e to 1687ba8 Compare July 24, 2026 09:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants