Skip to content

OCPBUGS-99755: OTE: fix AWS endpoint resolution for non-standard partitions - #494

Open
tthvo wants to merge 1 commit into
openshift:mainfrom
tthvo:OCPBUGS-99755
Open

OCPBUGS-99755: OTE: fix AWS endpoint resolution for non-standard partitions#494
tthvo wants to merge 1 commit into
openshift:mainfrom
tthvo:OCPBUGS-99755

Conversation

@tthvo

@tthvo tthvo commented Jul 24, 2026

Copy link
Copy Markdown
Member

Description

The BaseEndpoint for ELBv2 and EC2 clients was hardcoded to .amazonaws.com, which is incorrect for non-standard AWS partitions like EUSC (.amazonaws.eu). This caused API calls to fail on EUSC clusters because the requests were sent to the wrong domain.

This PR removes the BaseEndpoint override to let the AWS SDK automatically resolve to an partition-aware endpoint.

See also #464.

Note: Originally, I thought it was due to an outdated AWS SDK deps, but the endpoint resolution is fine (without base endpoint override) thanks to the compatible core config module. Thus, there is no longer any need to upgrade go deps.

Summary by CodeRabbit

  • Bug Fixes
    • Improved AWS load balancer and EC2 connectivity by stopping the use of hardcoded public regional endpoints, allowing the SDK to resolve the correct endpoints automatically.
    • Enhanced compatibility with varied AWS environments and endpoint configurations.

@openshift-ci-robot openshift-ci-robot added jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Jul 24, 2026
@openshift-ci-robot

Copy link
Copy Markdown

@tthvo: This pull request references Jira Issue OCPBUGS-99755, which is invalid:

  • expected the bug to target the "5.0.0" version, but no target version was set

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

Description

NLB Security Group OTEs are failing on the installer's e2e-aws-eusc-techpreview presubmit (see example) due to invalid API endpoint resolution. For EUSC, the endpoints have .eu top-level domain instead of .com.

I0724 01:09:00.576463  158635 helper.go:78] transient error describing load balancers (will retry): operation error Elastic Load Balancing v2: DescribeLoadBalancers, https response error StatusCode: 0, RequestID: , request send failed, Post "https://elasticloadbalancing.eusc-de-east-1.amazonaws.com/": dial tcp: lookup elasticloadbalancing.eusc-de-east-1.amazonaws.com on 172.30.0.10:53: no such host

This PR bumps github.com/aws/aws-sdk-go-v2/service/elasticloadbalancingv2 to get EUSC endpoint support.

Notes

Claude analysis shows that EUSC support is available since version:

┌──────────────────────────────────────┬─────────────────────────┬─────────────────────────────────────┐
│               Package                │ First version with EUSC │                Date                 │
├──────────────────────────────────────┼─────────────────────────┼─────────────────────────────────────┤
│ aws-sdk-go-v2 (core partitions.json) │ v1.41.2                 │ 2026-02-23 (commit from 2026-01-15) │
├──────────────────────────────────────┼─────────────────────────┼─────────────────────────────────────┤
│ service/ec2                          │ v1.211.1                │ 2025-04-03                          │
├──────────────────────────────────────┼─────────────────────────┼─────────────────────────────────────┤
│ service/elasticloadbalancingv2       │ v1.54.5                 │ 2025-12-09                          │
└──────────────────────────────────────┴─────────────────────────┴─────────────────────────────────────┘

I took this chance to upgrade service/elasticloadbalancingv2 to latest v1.58.1 while keeping service/ec2 "as-is" to minimize the vendor change.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci
openshift-ci Bot requested a review from damdo July 24, 2026 20:01
@openshift-ci

openshift-ci Bot commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign mdbooth for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci
openshift-ci Bot requested a review from theobarberbany July 24, 2026 20:01
@coderabbitai

coderabbitai Bot commented Jul 24, 2026

Copy link
Copy Markdown

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

Walkthrough

AWS load balancer and EC2 test clients no longer override SDK endpoint resolution with hardcoded regional hostnames.

Changes

AWS endpoint selection

Layer / File(s) Summary
Remove hardcoded AWS endpoints
openshift-tests/ccm-aws-tests/e2e/aws/helper.go
ELBv2 and EC2 clients are created without BaseEndpoint overrides, allowing the AWS SDK to resolve endpoints from configuration and region.

Estimated code review effort: 1 (Trivial) | ~3 minutes

Suggested reviewers: mtulio, damdo, theobarberbany

🚥 Pre-merge checks | ✅ 15
✅ Passed checks (15 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed Only helper client configuration changed; no Ginkgo It/Describe/Context/When titles were added or modified.
Test Structure And Quality ✅ Passed No Ginkgo test code was changed; the PR only edits helper client endpoint wiring in aws/helper.go.
Microshift Test Compatibility ✅ Passed No new Ginkgo e2e tests were added; the PR only changes AWS client helper code in helper.go.
Single Node Openshift (Sno) Test Compatibility ✅ Passed Only helper functions changed; helper.go contains no Ginkgo It/Describe/Context/When tests or SNO-sensitive cluster assumptions.
Topology-Aware Scheduling Compatibility ✅ Passed Only AWS test client endpoint setup changed in helper.go; no manifests/controllers/scheduling constraints were introduced.
Ote Binary Stdout Contract ✅ Passed The PR only removes AWS BaseEndpoint overrides in helper.go; it adds no stdout writes in main/suite setup or any other process-level code.
Ipv6 And Disconnected Network Test Compatibility ✅ Passed Patch only changes AWS client helper wiring; no new Ginkgo tests or IPv4/external-connectivity assumptions were added.
No-Weak-Crypto ✅ Passed helper.go only removes AWS endpoint overrides; no weak crypto, custom crypto, or secret comparisons are present.
Container-Privileges ✅ Passed PR only changes helper.go AWS client endpoint config; no manifest or securityContext privilege fields were modified.
No-Sensitive-Data-In-Logs ✅ Passed Changed lines only remove AWS BaseEndpoint overrides; no logging statements or sensitive fields were added.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main change: fixing AWS endpoint resolution for non-standard partitions.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@tthvo

tthvo commented Jul 24, 2026

Copy link
Copy Markdown
Member Author

/jira refresh

@openshift-ci-robot openshift-ci-robot added jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. and removed jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Jul 24, 2026
@openshift-ci-robot

Copy link
Copy Markdown

@tthvo: This pull request references Jira Issue OCPBUGS-99755, which is valid. The bug has been moved to the POST state.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (5.0.0) matches configured target version for branch (5.0.0)
  • bug is in the state New, which is one of the valid states (NEW, ASSIGNED, POST)
Details

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@tthvo

tthvo commented Jul 24, 2026

Copy link
Copy Markdown
Member Author

/cc @mtulio @mfbonfigli

@openshift-ci
openshift-ci Bot requested review from mfbonfigli and mtulio July 24, 2026 20:02
@openshift-ci-robot

Copy link
Copy Markdown

@tthvo: This pull request references Jira Issue OCPBUGS-99755, which is valid.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (5.0.0) matches configured target version for branch (5.0.0)
  • bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)
Details

In response to this:

Description

NLB Security Group OTEs are failing on the installer's e2e-aws-eusc-techpreview presubmit (see example) due to invalid API endpoint resolution. For EUSC, the endpoints have .eu top-level domain instead of .com.

I0724 01:09:00.576463  158635 helper.go:78] transient error describing load balancers (will retry): operation error Elastic Load Balancing v2: DescribeLoadBalancers, https response error StatusCode: 0, RequestID: , request send failed, Post "https://elasticloadbalancing.eusc-de-east-1.amazonaws.com/": dial tcp: lookup elasticloadbalancing.eusc-de-east-1.amazonaws.com on 172.30.0.10:53: no such host

This PR bumps github.com/aws/aws-sdk-go-v2/service/elasticloadbalancingv2 to get EUSC endpoint support.

Notes

Claude analysis shows that EUSC support is available since version:

┌──────────────────────────────────────┬─────────────────────────┬─────────────────────────────────────┐
│               Package                │ First version with EUSC │                Date                 │
├──────────────────────────────────────┼─────────────────────────┼─────────────────────────────────────┤
│ aws-sdk-go-v2 (core partitions.json) │ v1.41.2                 │ 2026-02-23 (commit from 2026-01-15) │
├──────────────────────────────────────┼─────────────────────────┼─────────────────────────────────────┤
│ service/ec2                          │ v1.211.1                │ 2025-04-03                          │
├──────────────────────────────────────┼─────────────────────────┼─────────────────────────────────────┤
│ service/elasticloadbalancingv2       │ v1.54.5                 │ 2025-12-09                          │
└──────────────────────────────────────┴─────────────────────────┴─────────────────────────────────────┘

I took this chance to upgrade service/elasticloadbalancingv2 to latest v1.58.1 while keeping service/ec2 "as-is" to minimize the vendor change.

Summary by CodeRabbit

  • Chores
  • Updated AWS SDK components used by the AWS integration tests.
  • Upgraded supporting AWS configuration, endpoint, and request-handling libraries for improved compatibility.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@tthvo

tthvo commented Jul 24, 2026

Copy link
Copy Markdown
Member Author

/testwith openshift/installer/main/e2e-aws-eusc-techpreview openshift/installer#10672

@tthvo

tthvo commented Jul 24, 2026

Copy link
Copy Markdown
Member Author

/payload-job periodic-ci-openshift-hypershift-release-5.0-periodics-e2e-aws-ovn-conformance

@openshift-ci

openshift-ci Bot commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

@tthvo: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-hypershift-release-5.0-periodics-e2e-aws-ovn-conformance

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/840867a0-879b-11f1-8fa2-ca425d91c641-0

@tthvo tthvo changed the title OCPBUGS-99755: OTE: bump elasticloadbalancingv2 to support AWS EUSC OCPBUGS-99755: OTE: fix AWS endpoint resolution for non-standard partitions Jul 24, 2026
@openshift-ci-robot

Copy link
Copy Markdown

@tthvo: This pull request references Jira Issue OCPBUGS-99755, which is valid.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (5.0.0) matches configured target version for branch (5.0.0)
  • bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)

The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

Description

NLB Security Group OTEs are failing on the installer's e2e-aws-eusc-techpreview presubmit (see example) due to invalid API endpoint resolution. For EUSC, the endpoints have .eu top-level domain instead of .com.

I0724 01:09:00.576463  158635 helper.go:78] transient error describing load balancers (will retry): operation error Elastic Load Balancing v2: DescribeLoadBalancers, https response error StatusCode: 0, RequestID: , request send failed, Post "https://elasticloadbalancing.eusc-de-east-1.amazonaws.com/": dial tcp: lookup elasticloadbalancing.eusc-de-east-1.amazonaws.com on 172.30.0.10:53: no such host

This PR bumps github.com/aws/aws-sdk-go-v2/service/elasticloadbalancingv2 to get EUSC endpoint support.

Notes

Claude analysis shows that EUSC support is available since version:

┌──────────────────────────────────────┬─────────────────────────┬─────────────────────────────────────┐
│               Package                │ First version with EUSC │                Date                 │
├──────────────────────────────────────┼─────────────────────────┼─────────────────────────────────────┤
│ aws-sdk-go-v2 (core partitions.json) │ v1.41.2                 │ 2026-02-23 (commit from 2026-01-15) │
├──────────────────────────────────────┼─────────────────────────┼─────────────────────────────────────┤
│ service/ec2                          │ v1.211.1                │ 2025-04-03                          │
├──────────────────────────────────────┼─────────────────────────┼─────────────────────────────────────┤
│ service/elasticloadbalancingv2       │ v1.54.5                 │ 2025-12-09                          │
└──────────────────────────────────────┴─────────────────────────┴─────────────────────────────────────┘

I took this chance to upgrade service/elasticloadbalancingv2 to latest v1.58.1 while keeping service/ec2 "as-is" to minimize the vendor change.

Summary by CodeRabbit

  • Bug Fixes
  • Improved AWS endpoint handling across regional partitions.
  • Ensured connections use the correct DNS suffix for standard, China, European Sovereign, and isolated AWS regions.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
openshift-tests/ccm-aws-tests/e2e/aws/helper.go (1)

200-220: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

Add table-driven tests for every partition suffix.

This helper is shared by both ELBv2 and EC2 clients; test cn-, eusc-, each ISO prefix, us-gov-, and the default case to prevent silent endpoint regressions.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@openshift-tests/ccm-aws-tests/e2e/aws/helper.go` around lines 200 - 220, Add
table-driven unit tests for dnsSuffixForRegion covering cn-, eusc-, us-iso-,
us-isob-, eu-isoe-, us-isof-, us-gov-, and an unmatched region. Assert each
input returns its expected partition suffix, including the default amazonaws.com
case.

Source: MCP tools

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@openshift-tests/ccm-aws-tests/e2e/aws/helper.go`:
- Around line 200-220: Add table-driven unit tests for dnsSuffixForRegion
covering cn-, eusc-, us-iso-, us-isob-, eu-isoe-, us-isof-, us-gov-, and an
unmatched region. Assert each input returns its expected partition suffix,
including the default amazonaws.com case.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 002f9648-ade4-4030-9da3-1446579091d8

📥 Commits

Reviewing files that changed from the base of the PR and between f3d4f2b and 12feedf.

📒 Files selected for processing (1)
  • openshift-tests/ccm-aws-tests/e2e/aws/helper.go

@tthvo

tthvo commented Jul 24, 2026

Copy link
Copy Markdown
Member Author

/payload-job periodic-ci-openshift-hypershift-release-5.0-periodics-e2e-aws-ovn-conformance

@openshift-ci

openshift-ci Bot commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

@tthvo: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-hypershift-release-5.0-periodics-e2e-aws-ovn-conformance

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/6e13f750-87a2-11f1-8221-50acf096a3f1-0

@tthvo

tthvo commented Jul 24, 2026

Copy link
Copy Markdown
Member Author

/testwith openshift/installer/main/e2e-aws-eusc-techpreview openshift/installer#10672

Comment thread openshift-tests/ccm-aws-tests/e2e/aws/helper.go
Comment thread openshift-tests/ccm-aws-tests/e2e/aws/helper.go Outdated
@tthvo
tthvo force-pushed the OCPBUGS-99755 branch 2 times, most recently from de8f63c to f48822d Compare July 28, 2026 04:37
@tthvo

tthvo commented Jul 28, 2026

Copy link
Copy Markdown
Member Author

/payload-job periodic-ci-openshift-hypershift-release-5.0-e2e-aws-ovn-conformance-ccm-techpreview
/payload-job periodic-ci-openshift-hypershift-release-5.0-periodics-e2e-aws-ovn-conformance

@openshift-ci

openshift-ci Bot commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

@tthvo: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-hypershift-release-5.0-periodics-e2e-aws-ovn-conformance

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/84ce8b10-8a3e-11f1-960a-7f6a1726f82c-0

@tthvo

tthvo commented Jul 28, 2026

Copy link
Copy Markdown
Member Author

/payload-job periodic-ci-openshift-hypershift-release-5.0-periodics-e2e-aws-ovn-conformance-techpreview

@openshift-ci

openshift-ci Bot commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

@tthvo: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-hypershift-release-5.0-periodics-e2e-aws-ovn-conformance-techpreview

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/fb874df0-8a3e-11f1-9674-dba2d98d43cc-0

@tthvo

tthvo commented Jul 28, 2026

Copy link
Copy Markdown
Member Author

/testwith openshift/installer/main/e2e-aws-eusc-techpreview openshift/installer#10672

@tthvo

tthvo commented Jul 28, 2026

Copy link
Copy Markdown
Member Author

/testwith openshift/installer/main/e2e-aws-eusc-techpreview openshift/installer#10717

Need to use an installer without merge conflicts 😅

@tthvo

tthvo commented Jul 28, 2026

Copy link
Copy Markdown
Member Author

/test e2e-aws-ovn-upgrade

We can rely on the default endpoint resolver of the SDK to construct the
correct partition-aware public regional endpoint.
@tthvo

tthvo commented Jul 28, 2026

Copy link
Copy Markdown
Member Author

/payload-job periodic-ci-openshift-hypershift-release-5.0-e2e-aws-ovn-conformance-ccm-techpreview
/payload-job periodic-ci-openshift-hypershift-release-5.0-periodics-e2e-aws-ovn-conformance-techpreview

@openshift-ci

openshift-ci Bot commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

@tthvo: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-hypershift-release-5.0-periodics-e2e-aws-ovn-conformance-techpreview

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/72623570-8a58-11f1-9ae3-326786f070e2-0

@tthvo

tthvo commented Jul 28, 2026

Copy link
Copy Markdown
Member Author

/testwith openshift/installer/main/e2e-aws-eusc-techpreview openshift/installer#10717

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants