CIHealth Run Log

↗ Open Failure patterns for this day

Environment: PROD

Time (UTC)	Job	Failed at	Result	PR	Failed tests	Details
11:18:47	periodic-ci-Azure-ARO-HCP-main-periodic-prod-e2e-parallel post-goodmerged PR No additional run metadata captured	e2e	failed	n/a	17	e2e (17) CreateNodePoolFromParam timed out after <minutes> minutes; context deadline exceeded 11:18:47 UTC · failed at e2e · test Customer should be able to create an HCP cluster · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create.go:118]: Unexpected error: <fmt.wrapErrors \| 0xc001195ce0>: failed to create NodePool np-1, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group basic-cluster-5jpdsk, error: failed waiting for nodepool="np-1" for cluster "basic-hcp-cluster" in resourcegroup="basic-cluster-5jpdsk" to finish creating: context deadline exceeded { msg: "failed to create NodePool np-1, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group basic-cluster-5jpdsk, error: failed waiting for nodepool=\"np-1\" for cluster \"basic-hcp-cluster\" in resourcegroup=\"basic-cluster-5jpdsk\" to finish creating: context deadline exceeded", errs: [ <errors.errorString \| 0xc000112ea0>{ s: "timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group basic-cluster-5jpdsk", }, <fmt.wrapError \| 0xc00075b4e0>{ msg: "failed waiting for nodepool=\"np-1\" for cluster \"basic-hcp-cluster\" in resourcegroup=\"basic-cluster-5jpdsk\" to finish creating: context deadline exceeded", err: <context.deadlineExceededError>{}, }, ], } occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create.go:118]: Unexpected error: <fmt.wrapErrors \| 0xc001195ce0>: failed to create NodePool np-1, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group basic-cluster-5jpdsk, error: failed waiting for nodepool="np-1" for cluster "basic-hcp-cluster" in resourcegroup="basic-cluster-5jpdsk" to finish creating: context deadline exceeded { msg: "failed to create NodePool np-1, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group basic-cluster-5jpdsk, error: failed waiting for nodepool=\"np-1\" for cluster \"basic-hcp-cluster\" in resourcegroup=\"basic-cluster-5jpdsk\" to finish creating: context deadline exceeded", errs: [ <errors.errorString \| 0xc000112ea0>{ s: "timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group basic-cluster-5jpdsk", }, <fmt.wrapError \| 0xc00075b4e0>{ msg: "failed waiting for nodepool=\"np-1\" for cluster \"basic-hcp-cluster\" in resourcegroup=\"basic-cluster-5jpdsk\" to finish creating: context deadline exceeded", err: <context.deadlineExceededError>{}, }, ], } occurred ERROR CODE: InternalServerError; detail message insufficient public IP address quota: required <count>, available <count>; provider Microsoft.RedHatOpenShift 11:18:47 UTC · failed at e2e · test Customer should be able to list HCP clusters without node pools at both subscription and resource group levels · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_list_no_infra.go:86]: Unexpected error: <fmt.wrapError \| 0xc000f06a60>: failed to create HCP cluster list-test-cluster-p6lb7l: failed waiting for cluster="list-test-cluster-p6lb7l" in resourcegroup="cluster-listing-dlgx6v" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4019f702-1223-4f14-b6db-2e256f1f2a87 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4019f702-1223-4f14-b6db-2e256f1f2a87", "name": "4019f702-1223-4f14-b6db-2e256f1f2a87", "status": "Failed", "startTime": "2026-03-04T11:40:19.135177985Z", "endTime": "2026-03-04T11:45:00.015034663Z", "error": { "code": "InternalServerError", "message": "insufficient public IP address quota: required 2, available 0" } } -------------------------------------------------------------------------------- { msg: "failed to create HCP cluster list-test-cluster-p6lb7l: failed waiting for cluster=\"list-test-cluster-p6lb7l\" in resourcegroup=\"cluster-listing-dlgx6v\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4019f702-1223-4f14-b6db-2e256f1f2a87\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4019f702-1223-4f14-b6db-2e256f1f2a87\",\n \"name\": \"4019f702-1223-4f14-b6db-2e256f1f2a87\",\n \"status\": \"Failed\",\n \"startTime\": \"2026-03-04T11:40:19.135177985Z\",\n \"endTime\": \"2026-03-04T11:45:00.015034663Z\",\n \"error\": {\n \"code\": \"InternalServerError\",\n \"message\": \"insufficient public IP address quota: required 2, available 0\"\n }\n}\n--------------------------------------------------------------------------------\n", err: <fmt.wrapError \| 0xc000f06a40>{ msg: "failed waiting for cluster=\"list-test-cluster-p6lb7l\" in resourcegroup=\"cluster-listing-dlgx6v\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4019f702-1223-4f14-b6db-2e256f1f2a87\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4019f702-1223-4f14-b6db-2e256f1f2a87\",\n \"name\": \"4019f702-1223-4f14-b6db-2e256f1f2a87\",\n \"status\": \"Failed\",\n \"startTime\": \"2026-03-04T11:40:19.135177985Z\",\n \"endTime\": \"2026-03-04T11:45:00.015034663Z\",\n \"error\": {\n \"code\": \"InternalServerError\",\n \"message\": \"insufficient public IP address quota: required 2, available 0\"\n }\n}\n--------------------------------------------------------------------------------\n", err: <exported.ResponseError \| 0xc00122cd20>{ ErrorCode: "InternalServerError", StatusCode: 200, RawResponse: { Status: "200 OK", StatusCode: 200, Proto: "HTTP/2.0", ProtoMajor: 2, ProtoMinor: 0, Header: { "X-Envoy-Upstream-Service-Time": ["19"], "X-Ms-Operation-Identifier": [ "appId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,tenantId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,objectId=0e96dbb7-17a2-4393-b2d3-d84524e39c32/canadacentral/8ffb5ba3-48ab-446c-a59c-3f9d09ee97a7", ], "X-Ms-Routing-Request-Id": [ "CANADACENTRAL:20260304T114501Z:4d14fc79-c15b-43f1-b5fc-ec2ab314d07f", ], "X-Content-Type-Options": ["nosniff"], "X-Cache": ["CONFIG_NOCACHE"], "Cache-Control": ["no-cache"], "X-Ms-Request-Id": [ "a63da353-17cb-482c-9907-e48691e47dad", ], "Pragma": ["no-cache"], "Content-Length": ["499"], "Content-Type": ["application/json"], "Expires": ["-1"], "X-Ms-Ratelimit-Remaining-Subscription-Reads": ["249"], "X-Msedge-Ref": [ ... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_list_no_infra.go:86]: Unexpected error: <fmt.wrapError \| 0xc000f06a60>: failed to create HCP cluster list-test-cluster-p6lb7l: failed waiting for cluster="list-test-cluster-p6lb7l" in resourcegroup="cluster-listing-dlgx6v" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4019f702-1223-4f14-b6db-2e256f1f2a87 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4019f702-1223-4f14-b6db-2e256f1f2a87", "name": "4019f702-1223-4f14-b6db-2e256f1f2a87", "status": "Failed", "startTime": "2026-03-04T11:40:19.135177985Z", "endTime": "2026-03-04T11:45:00.015034663Z", "error": { "code": "InternalServerError", "message": "insufficient public IP address quota: required 2, available 0" } } -------------------------------------------------------------------------------- { msg: "failed to create HCP cluster list-test-cluster-p6lb7l: failed waiting for cluster=\"list-test-cluster-p6lb7l\" in resourcegroup=\"cluster-listing-dlgx6v\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4019f702-1223-4f14-b6db-2e256f1f2a87\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4019f702-1223-4f14-b6db-2e256f1f2a87\",\n \"name\": \"4019f702-1223-4f14-b6db-2e256f1f2a87\",\n \"status\": \"Failed\",\n \"startTime\": \"2026-03-04T11:40:19.135177985Z\",\n \"endTime\": \"2026-03-04T11:45:00.015034663Z\",\n \"error\": {\n \"code\": \"InternalServerError\",\n \"message\": \"insufficient public IP address quota: required 2, available 0\"\n }\n}\n--------------------------------------------------------------------------------\n", err: <fmt.wrapError \| 0xc000f06a40>{ msg: "failed waiting for cluster=\"list-test-cluster-p6lb7l\" in resourcegroup=\"cluster-listing-dlgx6v\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4019f702-1223-4f14-b6db-2e256f1f2a87\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4019f702-1223-4f14-b6db-2e256f1f2a87\",\n \"name\": \"4019f702-1223-4f14-b6db-2e256f1f2a87\",\n \"status\": \"Failed\",\n \"startTime\": \"2026-03-04T11:40:19.135177985Z\",\n \"endTime\": \"2026-03-04T11:45:00.015034663Z\",\n \"error\": {\n \"code\": \"InternalServerError\",\n \"message\": \"insufficient public IP address quota: required 2, available 0\"\n }\n}\n--------------------------------------------------------------------------------\n", err: <exported.ResponseError \| 0xc00122cd20>{ ErrorCode: "InternalServerError", StatusCode: 200, RawResponse: { Status: "200 OK", StatusCode: 200, Proto: "HTTP/2.0", ProtoMajor: 2, ProtoMinor: 0, Header: { "X-Envoy-Upstream-Service-Time": ["19"], "X-Ms-Operation-Identifier": [ "appId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,tenantId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,objectId=0e96dbb7-17a2-4393-b2d3-d84524e39c32/canadacentral/8ffb5ba3-48ab-446c-a59c-3f9d09ee97a7", ], "X-Ms-Routing-Request-Id": [ "CANADACENTRAL:20260304T114501Z:4d14fc79-c15b-43f1-b5fc-ec2ab314d07f", ], "X-Content-Type-Options": ["nosniff"], "X-Cache": ["CONFIG_NOCACHE"], "Cache-Control": ["no-cache"], "X-Ms-Request-Id": [ "a63da353-17cb-482c-9907-e48691e47dad", ], "Pragma": ["no-cache"], "Content-Length": ["499"], "Content-Type": ["application/json"], "Expires": ["-1"], "X-Ms-Ratelimit-Remaining-Subscription-Reads": ["249"], "X-Msedge-Ref": [ ... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output occurred CreateNodePoolFromParam timed out after <minutes> minutes; context deadline exceeded 11:18:47 UTC · failed at e2e · test Customer should be able to create an HCP cluster and manage pull secrets · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_pullsecret.go:112]: Unexpected error: <fmt.wrapErrors \| 0xc0009e6630>: failed to create NodePool np-1, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group pullsecret-test-f6kzk8, error: failed waiting for nodepool="np-1" for cluster "pullsecret-hcp-cluster" in resourcegroup="pullsecret-test-f6kzk8" to finish creating: context deadline exceeded { msg: "failed to create NodePool np-1, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group pullsecret-test-f6kzk8, error: failed waiting for nodepool=\"np-1\" for cluster \"pullsecret-hcp-cluster\" in resourcegroup=\"pullsecret-test-f6kzk8\" to finish creating: context deadline exceeded", errs: [ <errors.errorString \| 0xc000a36550>{ s: "timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group pullsecret-test-f6kzk8", }, <fmt.wrapError \| 0xc0009dc4e0>{ msg: "failed waiting for nodepool=\"np-1\" for cluster \"pullsecret-hcp-cluster\" in resourcegroup=\"pullsecret-test-f6kzk8\" to finish creating: context deadline exceeded", err: <context.deadlineExceededError>{}, }, ], } occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_pullsecret.go:112]: Unexpected error: <fmt.wrapErrors \| 0xc0009e6630>: failed to create NodePool np-1, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group pullsecret-test-f6kzk8, error: failed waiting for nodepool="np-1" for cluster "pullsecret-hcp-cluster" in resourcegroup="pullsecret-test-f6kzk8" to finish creating: context deadline exceeded { msg: "failed to create NodePool np-1, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group pullsecret-test-f6kzk8, error: failed waiting for nodepool=\"np-1\" for cluster \"pullsecret-hcp-cluster\" in resourcegroup=\"pullsecret-test-f6kzk8\" to finish creating: context deadline exceeded", errs: [ <errors.errorString \| 0xc000a36550>{ s: "timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group pullsecret-test-f6kzk8", }, <fmt.wrapError \| 0xc0009dc4e0>{ msg: "failed waiting for nodepool=\"np-1\" for cluster \"pullsecret-hcp-cluster\" in resourcegroup=\"pullsecret-test-f6kzk8\" to finish creating: context deadline exceeded", err: <context.deadlineExceededError>{}, }, ], } occurred CreateOrUpdateExternalAuthAndWait timed out after <minutes> minutes; context deadline exceeded 11:18:47 UTC · failed at e2e · test Customer should be able to lifecycle and confirm external auth on a cluster · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_list_and_verify.go:118]: Unexpected error: <fmt.wrapErrors \| 0xc001057380>: failed waiting for external auth "ea-list" in resourcegroup="ea-list-8crjbf" for cluster="ea-listr26d5r" to finish, caused by: timeout '15.000000' minutes exceeded during CreateOrUpdateExternalAuthAndWait for external auth ea-list in cluster ea-listr26d5r in resource group ea-list-8crjbf, error: context deadline exceeded { msg: "failed waiting for external auth \"ea-list\" in resourcegroup=\"ea-list-8crjbf\" for cluster=\"ea-listr26d5r\" to finish, caused by: timeout '15.000000' minutes exceeded during CreateOrUpdateExternalAuthAndWait for external auth ea-list in cluster ea-listr26d5r in resource group ea-list-8crjbf, error: context deadline exceeded", errs: [ <errors.errorString \| 0xc000717760>{ s: "timeout '15.000000' minutes exceeded during CreateOrUpdateExternalAuthAndWait for external auth ea-list in cluster ea-listr26d5r in resource group ea-list-8crjbf", }, <context.deadlineExceededError>{}, ], } occurred fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_list_and_verify.go:118]: Unexpected error: <fmt.wrapErrors \| 0xc001057380>: failed waiting for external auth "ea-list" in resourcegroup="ea-list-8crjbf" for cluster="ea-listr26d5r" to finish, caused by: timeout '15.000000' minutes exceeded during CreateOrUpdateExternalAuthAndWait for external auth ea-list in cluster ea-listr26d5r in resource group ea-list-8crjbf, error: context deadline exceeded { msg: "failed waiting for external auth \"ea-list\" in resourcegroup=\"ea-list-8crjbf\" for cluster=\"ea-listr26d5r\" to finish, caused by: timeout '15.000000' minutes exceeded during CreateOrUpdateExternalAuthAndWait for external auth ea-list in cluster ea-listr26d5r in resource group ea-list-8crjbf, error: context deadline exceeded", errs: [ <errors.errorString \| 0xc000717760>{ s: "timeout '15.000000' minutes exceeded during CreateOrUpdateExternalAuthAndWait for external auth ea-list in cluster ea-listr26d5r in resource group ea-list-8crjbf", }, <context.deadlineExceededError>{}, ], } occurred ERROR CODE: Conflict; detail message Cannot request credential while resource is "provisioning"; provider Microsoft.RedHatOpenShift 11:18:47 UTC · failed at e2e · test Update HCPOpenShiftCluster Positive creates a cluster and updates tags with a PATCH request · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:170]: Unexpected error: <fmt.wrapError \| 0xc000d461a0>: failed to start credential request: POST https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-tags-l4v8j8/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-tags-cluster/requestAdminCredential -------------------------------------------------------------------------------- RESPONSE 409: 409 Conflict ERROR CODE: Conflict -------------------------------------------------------------------------------- { "error": { "code": "Conflict", "message": "Cannot request credential while resource is \"provisioning\"", "target": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-tags-l4v8j8/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-tags-cluster" } } -------------------------------------------------------------------------------- { msg: "failed to start credential request: POST https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-tags-l4v8j8/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-tags-cluster/requestAdminCredential\n--------------------------------------------------------------------------------\nRESPONSE 409: 409 Conflict\nERROR CODE: Conflict\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"Conflict\",\n \"message\": \"Cannot request credential while resource is \\\"provisioning\\\"\",\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-tags-l4v8j8/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-tags-cluster\"\n }\n}\n--------------------------------------------------------------------------------\n", err: <exported.ResponseError \| 0xc0014c8630>{ ErrorCode: "Conflict", StatusCode: 409, RawResponse: { Status: "409 Conflict", StatusCode: 409, Proto: "HTTP/2.0", ProtoMajor: 2, ProtoMinor: 0, Header: { "Pragma": ["no-cache"], "X-Ms-Error-Code": ["Conflict"], "X-Envoy-Upstream-Service-Time": ["24"], "X-Ms-Correlation-Request-Id": [ "04a3d0b4-7658-4960-bd57-a56b1404d40f", ], "X-Ms-Routing-Request-Id": [ "UKSOUTH:20260304T114425Z:04a3d0b4-7658-4960-bd57-a56b1404d40f", ], "X-Cache": ["CONFIG_NOCACHE"], "Cache-Control": ["no-cache"], "Content-Length": ["316"], "Expires": ["-1"], "X-Ms-Operation-Identifier": [ "appId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,tenantId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,objectId=0e96dbb7-17a2-4393-b2d3-d84524e39c32/uksouth/75e212f8-cdc9-4ef9-961a-651c91f9ecd9", ], "X-Ms-Ratelimit-Remaining-Subscription-Writes": ["199"], "Strict-Transport-Security": [ "max-age=31536000; includeSubDomains", ], "X-Content-Type-Options": ["nosniff"], "X-Ms-Request-Id": [ "51931baa-6371-4efa-a708-4f0f312f1ca0", ], "Date": [ "Wed, 04 Mar 2026 11:44:25 GMT", ], "Content-Type": ["application/json"], "X-Ms-Ratelimit-Remaining-Subscription-Global-Writes": ["2999"], "X-Msedge-Ref": [ "Ref A: 8417EFDD14874F8BB0E5237BA546D57F Ref B: CHI261140512040 Ref C: 2026-03-04T11:44:25Z", ], }, Body: <exported.nopClosingBytesReader \| 0xc000d46000>{ s: [123, 10, 32, 32, 32, 32, 34, 101, 114, 114, 111, 114, 34, 58, 32, 123, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 99, 111, 100, 101, 34, 58, 32, 34, 67, 111, 110, 102, 108, 105, 99, 116, 34, 44, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 109, 101, 115, 115, 97, 103, 101, 34, 58, 32, 34, 67, 97, 110, 110, 111, 116, 32, 114, 101, 113, 117, 101, 115, 116, 32, 99, 114, 101, 100, 101, 110, 116, 105, 97, 108, 32, 119, 104, 105, 108, 101, 32, 114, 101, 115, 111, 117, 114, 99, 101, 32, 105, 115, 32, 92, 34, 112, 114, 111, 118, 105, 115, 105, 111, 110, 105, 110, 103, 92, 34, 34, 44, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 116, 97, 114, 103, 101, 116, 34, 58, 32, 34, 47, 115, 117, 98, 115, 99, 114, 105, 112, 116, 105, 111, 110, 115, 47, 52, 48, 51, 100, 57, 100, 101, 57, 45, 49, 51, 50, 98, 45, 52, 57, 55, 52, 45, 57, 52, 97, 53, 45, 53, 98, 55, 56, 98, 100, 102, 97, 49, 57, 49, 101, 47, 11... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:170]: Unexpected error: <fmt.wrapError \| 0xc000d461a0>: failed to start credential request: POST https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-tags-l4v8j8/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-tags-cluster/requestAdminCredential -------------------------------------------------------------------------------- RESPONSE 409: 409 Conflict ERROR CODE: Conflict -------------------------------------------------------------------------------- { "error": { "code": "Conflict", "message": "Cannot request credential while resource is \"provisioning\"", "target": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-tags-l4v8j8/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-tags-cluster" } } -------------------------------------------------------------------------------- { msg: "failed to start credential request: POST https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-tags-l4v8j8/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-tags-cluster/requestAdminCredential\n--------------------------------------------------------------------------------\nRESPONSE 409: 409 Conflict\nERROR CODE: Conflict\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"Conflict\",\n \"message\": \"Cannot request credential while resource is \\\"provisioning\\\"\",\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-tags-l4v8j8/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-tags-cluster\"\n }\n}\n--------------------------------------------------------------------------------\n", err: <exported.ResponseError \| 0xc0014c8630>{ ErrorCode: "Conflict", StatusCode: 409, RawResponse: { Status: "409 Conflict", StatusCode: 409, Proto: "HTTP/2.0", ProtoMajor: 2, ProtoMinor: 0, Header: { "Pragma": ["no-cache"], "X-Ms-Error-Code": ["Conflict"], "X-Envoy-Upstream-Service-Time": ["24"], "X-Ms-Correlation-Request-Id": [ "04a3d0b4-7658-4960-bd57-a56b1404d40f", ], "X-Ms-Routing-Request-Id": [ "UKSOUTH:20260304T114425Z:04a3d0b4-7658-4960-bd57-a56b1404d40f", ], "X-Cache": ["CONFIG_NOCACHE"], "Cache-Control": ["no-cache"], "Content-Length": ["316"], "Expires": ["-1"], "X-Ms-Operation-Identifier": [ "appId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,tenantId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,objectId=0e96dbb7-17a2-4393-b2d3-d84524e39c32/uksouth/75e212f8-cdc9-4ef9-961a-651c91f9ecd9", ], "X-Ms-Ratelimit-Remaining-Subscription-Writes": ["199"], "Strict-Transport-Security": [ "max-age=31536000; includeSubDomains", ], "X-Content-Type-Options": ["nosniff"], "X-Ms-Request-Id": [ "51931baa-6371-4efa-a708-4f0f312f1ca0", ], "Date": [ "Wed, 04 Mar 2026 11:44:25 GMT", ], "Content-Type": ["application/json"], "X-Ms-Ratelimit-Remaining-Subscription-Global-Writes": ["2999"], "X-Msedge-Ref": [ "Ref A: 8417EFDD14874F8BB0E5237BA546D57F Ref B: CHI261140512040 Ref C: 2026-03-04T11:44:25Z", ], }, Body: <exported.nopClosingBytesReader \| 0xc000d46000>{ s: [123, 10, 32, 32, 32, 32, 34, 101, 114, 114, 111, 114, 34, 58, 32, 123, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 99, 111, 100, 101, 34, 58, 32, 34, 67, 111, 110, 102, 108, 105, 99, 116, 34, 44, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 109, 101, 115, 115, 97, 103, 101, 34, 58, 32, 34, 67, 97, 110, 110, 111, 116, 32, 114, 101, 113, 117, 101, 115, 116, 32, 99, 114, 101, 100, 101, 110, 116, 105, 97, 108, 32, 119, 104, 105, 108, 101, 32, 114, 101, 115, 111, 117, 114, 99, 101, 32, 105, 115, 32, 92, 34, 112, 114, 111, 118, 105, 115, 105, 111, 110, 105, 110, 103, 92, 34, 34, 44, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 116, 97, 114, 103, 101, 116, 34, 58, 32, 34, 47, 115, 117, 98, 115, 99, 114, 105, 112, 116, 105, 111, 110, 115, 47, 52, 48, 51, 100, 57, 100, 101, 57, 45, 49, 51, 50, 98, 45, 52, 57, 55, 52, 45, 57, 52, 97, 53, 45, 53, 98, 55, 56, 98, 100, 102, 97, 49, 57, 49, 101, 47, 11... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output occurred ERROR CODE: InternalServerError; detail message insufficient public IP address quota: required <count>, available <count>; provider Microsoft.RedHatOpenShift 11:18:47 UTC · failed at e2e · test Customer should be able to create an HCP cluster with custom autoscaling · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:97]: Unexpected error: <fmt.wrapError \| 0xc0007439c0>: failed to create HCP cluster autoscaling-hcp-cluster: failed waiting for cluster="autoscaling-hcp-cluster" in resourcegroup="autoscaling-cluster-rggf4c" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b1c8c857-485d-4a3c-b764-cd54b5d212a2 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b1c8c857-485d-4a3c-b764-cd54b5d212a2", "name": "b1c8c857-485d-4a3c-b764-cd54b5d212a2", "status": "Failed", "startTime": "2026-03-04T11:42:32.91171511Z", "endTime": "2026-03-04T11:45:01.618825129Z", "error": { "code": "InternalServerError", "message": "insufficient public IP address quota: required 2, available 1" } } -------------------------------------------------------------------------------- { msg: "failed to create HCP cluster autoscaling-hcp-cluster: failed waiting for cluster=\"autoscaling-hcp-cluster\" in resourcegroup=\"autoscaling-cluster-rggf4c\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b1c8c857-485d-4a3c-b764-cd54b5d212a2\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b1c8c857-485d-4a3c-b764-cd54b5d212a2\",\n \"name\": \"b1c8c857-485d-4a3c-b764-cd54b5d212a2\",\n \"status\": \"Failed\",\n \"startTime\": \"2026-03-04T11:42:32.91171511Z\",\n \"endTime\": \"2026-03-04T11:45:01.618825129Z\",\n \"error\": {\n \"code\": \"InternalServerError\",\n \"message\": \"insufficient public IP address quota: required 2, available 1\"\n }\n}\n--------------------------------------------------------------------------------\n", err: <fmt.wrapError \| 0xc0007439a0>{ msg: "failed waiting for cluster=\"autoscaling-hcp-cluster\" in resourcegroup=\"autoscaling-cluster-rggf4c\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b1c8c857-485d-4a3c-b764-cd54b5d212a2\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b1c8c857-485d-4a3c-b764-cd54b5d212a2\",\n \"name\": \"b1c8c857-485d-4a3c-b764-cd54b5d212a2\",\n \"status\": \"Failed\",\n \"startTime\": \"2026-03-04T11:42:32.91171511Z\",\n \"endTime\": \"2026-03-04T11:45:01.618825129Z\",\n \"error\": {\n \"code\": \"InternalServerError\",\n \"message\": \"insufficient public IP address quota: required 2, available 1\"\n }\n}\n--------------------------------------------------------------------------------\n", err: <exported.ResponseError \| 0xc000679c80>{ ErrorCode: "InternalServerError", StatusCode: 200, RawResponse: { Status: "200 OK", StatusCode: 200, Proto: "HTTP/2.0", ProtoMajor: 2, ProtoMinor: 0, Header: { "Cache-Control": ["no-cache"], "Expires": ["-1"], "X-Ms-Routing-Request-Id": [ "CANADACENTRAL:20260304T114509Z:d921fb07-cad8-47f3-beb6-14b898dd9e31", ], "X-Msedge-Ref": [ "Ref A: C1EE60EB4B804177AABFC80DC74FA33E Ref B: CHI261140501054 Ref C: 2026-03-04T11:45:08Z", ], "X-Ms-Correlation-Request-Id": [ "d921fb07-cad8-47f3-beb6-14b898dd9e31", ], "X-Content-Type-Options": ["nosniff"], "Pragma": ["no-cache"], "Content-Length": ["498"], "X-Envoy-Upstream-Service-Time": ["11"], "X-Ms-Operation-Identifier": [ "appId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,tenantId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,objectId=0e96dbb7-17a2-4393-b2d3-d84524e39c32/canadacentral/78cb4115-38f4-400a-a373-de90cb3f6447", ], "X-Ms-Ratelimit-Remaining... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:97]: Unexpected error: <fmt.wrapError \| 0xc0007439c0>: failed to create HCP cluster autoscaling-hcp-cluster: failed waiting for cluster="autoscaling-hcp-cluster" in resourcegroup="autoscaling-cluster-rggf4c" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b1c8c857-485d-4a3c-b764-cd54b5d212a2 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b1c8c857-485d-4a3c-b764-cd54b5d212a2", "name": "b1c8c857-485d-4a3c-b764-cd54b5d212a2", "status": "Failed", "startTime": "2026-03-04T11:42:32.91171511Z", "endTime": "2026-03-04T11:45:01.618825129Z", "error": { "code": "InternalServerError", "message": "insufficient public IP address quota: required 2, available 1" } } -------------------------------------------------------------------------------- { msg: "failed to create HCP cluster autoscaling-hcp-cluster: failed waiting for cluster=\"autoscaling-hcp-cluster\" in resourcegroup=\"autoscaling-cluster-rggf4c\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b1c8c857-485d-4a3c-b764-cd54b5d212a2\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b1c8c857-485d-4a3c-b764-cd54b5d212a2\",\n \"name\": \"b1c8c857-485d-4a3c-b764-cd54b5d212a2\",\n \"status\": \"Failed\",\n \"startTime\": \"2026-03-04T11:42:32.91171511Z\",\n \"endTime\": \"2026-03-04T11:45:01.618825129Z\",\n \"error\": {\n \"code\": \"InternalServerError\",\n \"message\": \"insufficient public IP address quota: required 2, available 1\"\n }\n}\n--------------------------------------------------------------------------------\n", err: <fmt.wrapError \| 0xc0007439a0>{ msg: "failed waiting for cluster=\"autoscaling-hcp-cluster\" in resourcegroup=\"autoscaling-cluster-rggf4c\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b1c8c857-485d-4a3c-b764-cd54b5d212a2\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b1c8c857-485d-4a3c-b764-cd54b5d212a2\",\n \"name\": \"b1c8c857-485d-4a3c-b764-cd54b5d212a2\",\n \"status\": \"Failed\",\n \"startTime\": \"2026-03-04T11:42:32.91171511Z\",\n \"endTime\": \"2026-03-04T11:45:01.618825129Z\",\n \"error\": {\n \"code\": \"InternalServerError\",\n \"message\": \"insufficient public IP address quota: required 2, available 1\"\n }\n}\n--------------------------------------------------------------------------------\n", err: <exported.ResponseError \| 0xc000679c80>{ ErrorCode: "InternalServerError", StatusCode: 200, RawResponse: { Status: "200 OK", StatusCode: 200, Proto: "HTTP/2.0", ProtoMajor: 2, ProtoMinor: 0, Header: { "Cache-Control": ["no-cache"], "Expires": ["-1"], "X-Ms-Routing-Request-Id": [ "CANADACENTRAL:20260304T114509Z:d921fb07-cad8-47f3-beb6-14b898dd9e31", ], "X-Msedge-Ref": [ "Ref A: C1EE60EB4B804177AABFC80DC74FA33E Ref B: CHI261140501054 Ref C: 2026-03-04T11:45:08Z", ], "X-Ms-Correlation-Request-Id": [ "d921fb07-cad8-47f3-beb6-14b898dd9e31", ], "X-Content-Type-Options": ["nosniff"], "Pragma": ["no-cache"], "Content-Length": ["498"], "X-Envoy-Upstream-Service-Time": ["11"], "X-Ms-Operation-Identifier": [ "appId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,tenantId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,objectId=0e96dbb7-17a2-4393-b2d3-d84524e39c32/canadacentral/78cb4115-38f4-400a-a373-de90cb3f6447", ], "X-Ms-Ratelimit-Remaining... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output occurred Cluster provisioning failed 11:18:47 UTC · failed at e2e · test Customer should be able to test admin credentials before cluster ready, then full admin credential lifecycle · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:153]: Cluster provisioning failed fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:153]: Cluster provisioning failed ERROR CODE: InternalServerError; detail code PublicIPCountLimitReached; detail message Cannot create more than 300 public IP addresses for this subscription in this region.; provider Microsoft.Network 11:18:47 UTC · failed at e2e · test Customer should respect cluster-wide node limits with nodepool autoscaling · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:166]: Unexpected error: <fmt.wrapError \| 0xc000c090a0>: failed to create HCP cluster node-limit-cluster: failed waiting for cluster="node-limit-cluster" in resourcegroup="node-limit-test-v554wg" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b6c96910-d9fe-4ee6-be6a-ddfe7e0aebc3 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b6c96910-d9fe-4ee6-be6a-ddfe7e0aebc3", "name": "b6c96910-d9fe-4ee6-be6a-ddfe7e0aebc3", "status": "Failed", "startTime": "2026-03-04T11:29:07.030229354Z", "endTime": "2026-03-04T11:33:56.440551062Z", "error": { "code": "InternalServerError", "message": "PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/node-limit-test-v554wg--managed/providers/Microsoft.Network/publicIPAddresses/2oqsj0l8tte65mugqrulb0nrcn5o984k-0\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"PublicIPCountLimitReached\",\n \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n \"details\": []\n }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/node-limit-test-v554wg--managed/providers/Microsoft.Network/publicIPAddresses/2oqsj0l8tte65mugqrulb0nrcn5o984k-1\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"PublicIPCountLimitReached\",\n \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n \"details\": []\n }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/node-limit-test-v554wg--managed/providers/Microsoft.Network/publicIPAddresses/2oqsj0l8tte65mugqrulb0nrcn5o984k-2\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"PublicIPCountLimitReached\",\n \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n \"details\": []\n }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/node-limit-test-v554wg--managed/providers/Microsoft.Network/publicIPAddresses/2oqsj0l8tte65mugqrulb0nrcn5o984k-3\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"PublicIPCountLimitReached\",\n \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n \"details\": []\n }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/node-limit-test-v554wg--managed/providers/Microsoft.Network/publicIPAddresses/2oqsj0l8tte65mugqrulb0nrcn5o984k-4\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"PublicIPCountLimitReached\",\n \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n \"details\": []\n }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/node-limit-test-v554wg--managed/providers/Microsoft.Network/publicIPAddresses/2oqsj0l8tte65mugqrulb0nrcn5o984k-5\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"PublicIPCountLimitReached\",\n \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n \"details\": []\n }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/node-limit-test-v554wg--managed/providers/Microsoft.Network/publicIPAddresses/2oqsj0l8tte65mugqrulb0nrcn5o984k-6\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"PublicIPCountLimitReached\",\n \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n \"details\": []\n }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/node-limit-test-v554wg--managed/providers/Microsoft.Network/publicIPAddresses/2oqsj0l8tte65mugqrulb0nrcn5o984k-7\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"PublicIPCountLimitReached\",\n \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n \"details\": []\n }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/node-limit-test-v554wg--managed/providers/Microsoft.Network/publicIPAddresses/2oqsj0l8tte65mugqrulb0nrcn5o984k-8\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"PublicIPCountLimitReached\",\n \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n \"details\": []\n }\n}\n--------------------------------------------------------------------------------\n" } } -------------------------------------------------------------------------------- { msg: "failed to create HCP cluster node-limit-cluster: failed waiting for cluster=\"node-limit-cluster\" in resourcegroup=\"node-limit-test-v554wg\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b6c96910-d9fe-4ee6-be6a-ddfe7e0aebc3\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b6c96910-d9fe-4ee6-be6a-ddfe7e0aebc3\",\n \"name\": \"b6c96910-d9fe-4ee6-be6a-ddfe7e0aebc3\",\n \"status\": \"Failed\",\n \"startTime\": \"2026-03-04T11:29:07.030229354Z\",\n \"endTime\": \"2026-03-04T11:33:56.440551062Z\",\n \"error\": {\n \"code\": \"InternalServerError\",\n \"message\": \"PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/node-limit-test-v554wg--managed/providers/Microsoft.Network/publicIPAddresses/2oqsj0l8tte65mugqrulb0nrcn5o984k-0\\n--------------------------------------------------------------------------------\\nRESPONSE 400: 400 Bad Request\\nERROR CODE: PublicIPCountLimitReached\\n--------------------------------------------------------------------------------\\n{\\n \\\"error\\\": {\\n \\\"code\\\": \\\"PublicIPCountLimitReached\\\",\\n \\\"message\\\": \\\"Cannot create more than 300 public IP addresses for this subscription in this region.\\\",\\n \\\"details\\\": []\\n }\\n}\\n--------------------------------------------------------------------------------\\n\\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/node-limit-test-v554wg--managed/providers/Microsoft.Network/publicIPAddresses/2oqsj0l8tte65mugqrulb0nrcn5o984k-1\\n--------------------------------------------------------------------------------\\nRESPONSE 400: 400 Bad Request\\nERROR CODE: PublicIPCountLimitReached\\n--------------------------------------------------------------------------------\\n{\\n \\\"error\\\": {\\n \\\"code\\\": \\\"PublicIPCountLimitReached\\\",\\n \\\"message\\\": \\\"Cannot create more than 300 public IP addresses for this subscription in this region.\\\",\\n \\\"details\\\": []\\n }\\n}\\n--------------------------------------------------------------------------------\\n\\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/node-limit-test-v554wg--managed/providers/Microsoft.Network/publicIPAddresses/2oqsj0l8tte65mugqrulb0nrcn5o984k-2\\n--------------------------------------------------------------------------------\\nRESPONSE 400: 400 Bad Request\\nERROR CODE: PublicIPCountLimitReached\\n--------------------------------------------------------------------------------\\n{\\n \\\"error\\\": {\\n \\\"code\\\": \\\"PublicIPCountLimitReached\\\",\\n \\\"message\\\": \\\"Cannot create more than 300 public IP addresses for this subscription in this region.\\\",\\n \\\"details\\\": []\\n }\\n}\\n--------------------------------------------------------------------------------\\n\\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/node-limit-test-v554wg--managed/providers/Microsoft.Network/publicIPAddresses/2oqsj0l8tte65mugqrulb0nrcn5o984k-3\\n--------------------------------------------------------------------------------\\nRESPONSE 400: 400 Bad Request\\nERROR CODE: PublicIPCountLimitReached\\n--------------------------------------------------------------------------------\\n{\\n \\\"error\\\": {\\n \\\"code\\\": \\\"PublicIPCountLimitReached\\\",\\n \\\"message\\\": \\\"Cannot create more than 300 public IP addresses for this subscription in this region.\... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:166]: Unexpected error: <fmt.wrapError \| 0xc000c090a0>: failed to create HCP cluster node-limit-cluster: failed waiting for cluster="node-limit-cluster" in resourcegroup="node-limit-test-v554wg" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b6c96910-d9fe-4ee6-be6a-ddfe7e0aebc3 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b6c96910-d9fe-4ee6-be6a-ddfe7e0aebc3", "name": "b6c96910-d9fe-4ee6-be6a-ddfe7e0aebc3", "status": "Failed", "startTime": "2026-03-04T11:29:07.030229354Z", "endTime": "2026-03-04T11:33:56.440551062Z", "error": { "code": "InternalServerError", "message": "PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/node-limit-test-v554wg--managed/providers/Microsoft.Network/publicIPAddresses/2oqsj0l8tte65mugqrulb0nrcn5o984k-0\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"PublicIPCountLimitReached\",\n \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n \"details\": []\n }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/node-limit-test-v554wg--managed/providers/Microsoft.Network/publicIPAddresses/2oqsj0l8tte65mugqrulb0nrcn5o984k-1\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"PublicIPCountLimitReached\",\n \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n \"details\": []\n }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/node-limit-test-v554wg--managed/providers/Microsoft.Network/publicIPAddresses/2oqsj0l8tte65mugqrulb0nrcn5o984k-2\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"PublicIPCountLimitReached\",\n \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n \"details\": []\n }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/node-limit-test-v554wg--managed/providers/Microsoft.Network/publicIPAddresses/2oqsj0l8tte65mugqrulb0nrcn5o984k-3\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"PublicIPCountLimitReached\",\n \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n \"details\": []\n }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/node-limit-test-v554wg--managed/providers/Microsoft.Network/publicIPAddresses/2oqsj0l8tte65mugqrulb0nrcn5o984k-4\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"PublicIPCountLimitReached\",\n \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n \"details\": []\n }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/node-limit-test-v554wg--managed/providers/Microsoft.Network/publicIPAddresses/2oqsj0l8tte65mugqrulb0nrcn5o984k-5\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"PublicIPCountLimitReached\",\n \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n \"details\": []\n }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/node-limit-test-v554wg--managed/providers/Microsoft.Network/publicIPAddresses/2oqsj0l8tte65mugqrulb0nrcn5o984k-6\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"PublicIPCountLimitReached\",\n \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n \"details\": []\n }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/node-limit-test-v554wg--managed/providers/Microsoft.Network/publicIPAddresses/2oqsj0l8tte65mugqrulb0nrcn5o984k-7\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"PublicIPCountLimitReached\",\n \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n \"details\": []\n }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/node-limit-test-v554wg--managed/providers/Microsoft.Network/publicIPAddresses/2oqsj0l8tte65mugqrulb0nrcn5o984k-8\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"PublicIPCountLimitReached\",\n \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n \"details\": []\n }\n}\n--------------------------------------------------------------------------------\n" } } -------------------------------------------------------------------------------- { msg: "failed to create HCP cluster node-limit-cluster: failed waiting for cluster=\"node-limit-cluster\" in resourcegroup=\"node-limit-test-v554wg\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b6c96910-d9fe-4ee6-be6a-ddfe7e0aebc3\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b6c96910-d9fe-4ee6-be6a-ddfe7e0aebc3\",\n \"name\": \"b6c96910-d9fe-4ee6-be6a-ddfe7e0aebc3\",\n \"status\": \"Failed\",\n \"startTime\": \"2026-03-04T11:29:07.030229354Z\",\n \"endTime\": \"2026-03-04T11:33:56.440551062Z\",\n \"error\": {\n \"code\": \"InternalServerError\",\n \"message\": \"PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/node-limit-test-v554wg--managed/providers/Microsoft.Network/publicIPAddresses/2oqsj0l8tte65mugqrulb0nrcn5o984k-0\\n--------------------------------------------------------------------------------\\nRESPONSE 400: 400 Bad Request\\nERROR CODE: PublicIPCountLimitReached\\n--------------------------------------------------------------------------------\\n{\\n \\\"error\\\": {\\n \\\"code\\\": \\\"PublicIPCountLimitReached\\\",\\n \\\"message\\\": \\\"Cannot create more than 300 public IP addresses for this subscription in this region.\\\",\\n \\\"details\\\": []\\n }\\n}\\n--------------------------------------------------------------------------------\\n\\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/node-limit-test-v554wg--managed/providers/Microsoft.Network/publicIPAddresses/2oqsj0l8tte65mugqrulb0nrcn5o984k-1\\n--------------------------------------------------------------------------------\\nRESPONSE 400: 400 Bad Request\\nERROR CODE: PublicIPCountLimitReached\\n--------------------------------------------------------------------------------\\n{\\n \\\"error\\\": {\\n \\\"code\\\": \\\"PublicIPCountLimitReached\\\",\\n \\\"message\\\": \\\"Cannot create more than 300 public IP addresses for this subscription in this region.\\\",\\n \\\"details\\\": []\\n }\\n}\\n--------------------------------------------------------------------------------\\n\\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/node-limit-test-v554wg--managed/providers/Microsoft.Network/publicIPAddresses/2oqsj0l8tte65mugqrulb0nrcn5o984k-2\\n--------------------------------------------------------------------------------\\nRESPONSE 400: 400 Bad Request\\nERROR CODE: PublicIPCountLimitReached\\n--------------------------------------------------------------------------------\\n{\\n \\\"error\\\": {\\n \\\"code\\\": \\\"PublicIPCountLimitReached\\\",\\n \\\"message\\\": \\\"Cannot create more than 300 public IP addresses for this subscription in this region.\\\",\\n \\\"details\\\": []\\n }\\n}\\n--------------------------------------------------------------------------------\\n\\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/node-limit-test-v554wg--managed/providers/Microsoft.Network/publicIPAddresses/2oqsj0l8tte65mugqrulb0nrcn5o984k-3\\n--------------------------------------------------------------------------------\\nRESPONSE 400: 400 Bad Request\\nERROR CODE: PublicIPCountLimitReached\\n--------------------------------------------------------------------------------\\n{\\n \\\"error\\\": {\\n \\\"code\\\": \\\"PublicIPCountLimitReached\\\",\\n \\\"message\\\": \\\"Cannot create more than 300 public IP addresses for this subscription in this region.\... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output occurred ERROR CODE: InternalServerError; detail code PublicIPCountLimitReached; detail message Cannot create more than 300 public IP addresses for this subscription in this region.; provider Microsoft.Network 11:18:47 UTC · failed at e2e · test Customer should be able to create an HCP cluster with Image Registry not present · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/image_registry_cluster_create.go:89]: Unexpected error: <fmt.wrapError \| 0xc0014a86a0>: failed to create HCP cluster disabled-image-registry-hcp-cluster: failed waiting for cluster="disabled-image-registry-hcp-cluster" in resourcegroup="disabled-image-registry-cqrzqm" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/ee06715b-b440-41cc-8866-ae10dbcc67f8 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/ee06715b-b440-41cc-8866-ae10dbcc67f8", "name": "ee06715b-b440-41cc-8866-ae10dbcc67f8", "status": "Failed", "startTime": "2026-03-04T11:52:51.02692019Z", "endTime": "2026-03-04T11:56:11.562177046Z", "error": { "code": "InternalServerError", "message": "PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/disabled-image-registry-cqrzqm--managed/providers/Microsoft.Network/publicIPAddresses/2oqsu4n7eod01mi46n8r90vsr5m1qqfp-8\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"PublicIPCountLimitReached\",\n \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n \"details\": []\n }\n}\n--------------------------------------------------------------------------------\n" } } -------------------------------------------------------------------------------- { msg: "failed to create HCP cluster disabled-image-registry-hcp-cluster: failed waiting for cluster=\"disabled-image-registry-hcp-cluster\" in resourcegroup=\"disabled-image-registry-cqrzqm\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/ee06715b-b440-41cc-8866-ae10dbcc67f8\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/ee06715b-b440-41cc-8866-ae10dbcc67f8\",\n \"name\": \"ee06715b-b440-41cc-8866-ae10dbcc67f8\",\n \"status\": \"Failed\",\n \"startTime\": \"2026-03-04T11:52:51.02692019Z\",\n \"endTime\": \"2026-03-04T11:56:11.562177046Z\",\n \"error\": {\n \"code\": \"InternalServerError\",\n \"message\": \"PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/disabled-image-registry-cqrzqm--managed/providers/Microsoft.Network/publicIPAddresses/2oqsu4n7eod01mi46n8r90vsr5m1qqfp-8\\n--------------------------------------------------------------------------------\\nRESPONSE 400: 400 Bad Request\\nERROR CODE: PublicIPCountLimitReached\\n--------------------------------------------------------------------------------\\n{\\n \\\"error\\\": {\\n \\\"code\\\": \\\"PublicIPCountLimitReached\\\",\\n \\\"message\\\": \\\"Cannot create more than 300 public IP addresses for this subscription in this region.\\\",\\n \\\"details\\\": []\\n }\\n}\\n--------------------------------------------------------------------------------\\n\"\n }\n}\n--------------------------------------------------------------------------------\n", err: <fmt.wrapError \| 0xc0014a8680>{ msg: "failed waiting for cluster=\"disabled-image-registry-hcp-cluster\" in resourcegroup=\"disabled-image-registry-cqrzqm\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/ee06715b-b440-41cc-8866-ae10dbcc67f8\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/ee06715b-b440-41cc-8866-ae10dbcc67f8\",\n \"name\": \"ee06715b-b440-41cc-8866-ae10dbcc67f8\",\n \"status\": \"Failed\",\n \"startTime\": \"2026-03-04T11:52:51.02692019Z\",\n \"endTime\": \"2026-03-04T11:56:11.562177046Z\",\n \"error\": {\n \"code\": \"InternalServerError\",\n \"message\": \"PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/disabled-image-registry-cqrzqm--managed/providers/Microsoft.Network/publicIPAddresses/2oqsu4n7eod01mi46n8r90vsr5m1qqfp-8\\n--------------------------------------------------------------------------------\\nRESPONSE 400: 400 Bad Request\\nERROR CODE: PublicIPCountLimitReached\\n--------------------------------------------------------------------------------\\n{\\n \\\"error\\\": {\\n \\\"code\\\": \\\"PublicIPCountLimitReached\\\",\\n \\\"message\\\": \\\"Cannot create more than 300 public IP addresses for this subscription in this region.\\\",\\n \\\"details\\\": []\\n }\\n}\\n--------------------------------------------------------------------------------\\n\"\n }\n}\n--------------------------------------------------------------------------------\n", err: <exported.ResponseError \| 0xc000df4d20>{ ErrorCode: "InternalServerError... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output occurred fail [github.com/Azure/ARO-HCP/test/e2e/image_registry_cluster_create.go:89]: Unexpected error: <fmt.wrapError \| 0xc0014a86a0>: failed to create HCP cluster disabled-image-registry-hcp-cluster: failed waiting for cluster="disabled-image-registry-hcp-cluster" in resourcegroup="disabled-image-registry-cqrzqm" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/ee06715b-b440-41cc-8866-ae10dbcc67f8 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/ee06715b-b440-41cc-8866-ae10dbcc67f8", "name": "ee06715b-b440-41cc-8866-ae10dbcc67f8", "status": "Failed", "startTime": "2026-03-04T11:52:51.02692019Z", "endTime": "2026-03-04T11:56:11.562177046Z", "error": { "code": "InternalServerError", "message": "PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/disabled-image-registry-cqrzqm--managed/providers/Microsoft.Network/publicIPAddresses/2oqsu4n7eod01mi46n8r90vsr5m1qqfp-8\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"PublicIPCountLimitReached\",\n \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n \"details\": []\n }\n}\n--------------------------------------------------------------------------------\n" } } -------------------------------------------------------------------------------- { msg: "failed to create HCP cluster disabled-image-registry-hcp-cluster: failed waiting for cluster=\"disabled-image-registry-hcp-cluster\" in resourcegroup=\"disabled-image-registry-cqrzqm\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/ee06715b-b440-41cc-8866-ae10dbcc67f8\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/ee06715b-b440-41cc-8866-ae10dbcc67f8\",\n \"name\": \"ee06715b-b440-41cc-8866-ae10dbcc67f8\",\n \"status\": \"Failed\",\n \"startTime\": \"2026-03-04T11:52:51.02692019Z\",\n \"endTime\": \"2026-03-04T11:56:11.562177046Z\",\n \"error\": {\n \"code\": \"InternalServerError\",\n \"message\": \"PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/disabled-image-registry-cqrzqm--managed/providers/Microsoft.Network/publicIPAddresses/2oqsu4n7eod01mi46n8r90vsr5m1qqfp-8\\n--------------------------------------------------------------------------------\\nRESPONSE 400: 400 Bad Request\\nERROR CODE: PublicIPCountLimitReached\\n--------------------------------------------------------------------------------\\n{\\n \\\"error\\\": {\\n \\\"code\\\": \\\"PublicIPCountLimitReached\\\",\\n \\\"message\\\": \\\"Cannot create more than 300 public IP addresses for this subscription in this region.\\\",\\n \\\"details\\\": []\\n }\\n}\\n--------------------------------------------------------------------------------\\n\"\n }\n}\n--------------------------------------------------------------------------------\n", err: <fmt.wrapError \| 0xc0014a8680>{ msg: "failed waiting for cluster=\"disabled-image-registry-hcp-cluster\" in resourcegroup=\"disabled-image-registry-cqrzqm\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/ee06715b-b440-41cc-8866-ae10dbcc67f8\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/ee06715b-b440-41cc-8866-ae10dbcc67f8\",\n \"name\": \"ee06715b-b440-41cc-8866-ae10dbcc67f8\",\n \"status\": \"Failed\",\n \"startTime\": \"2026-03-04T11:52:51.02692019Z\",\n \"endTime\": \"2026-03-04T11:56:11.562177046Z\",\n \"error\": {\n \"code\": \"InternalServerError\",\n \"message\": \"PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/disabled-image-registry-cqrzqm--managed/providers/Microsoft.Network/publicIPAddresses/2oqsu4n7eod01mi46n8r90vsr5m1qqfp-8\\n--------------------------------------------------------------------------------\\nRESPONSE 400: 400 Bad Request\\nERROR CODE: PublicIPCountLimitReached\\n--------------------------------------------------------------------------------\\n{\\n \\\"error\\\": {\\n \\\"code\\\": \\\"PublicIPCountLimitReached\\\",\\n \\\"message\\\": \\\"Cannot create more than 300 public IP addresses for this subscription in this region.\\\",\\n \\\"details\\\": []\\n }\\n}\\n--------------------------------------------------------------------------------\\n\"\n }\n}\n--------------------------------------------------------------------------------\n", err: <exported.ResponseError \| 0xc000df4d20>{ ErrorCode: "InternalServerError... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output occurred CreateNodePoolFromParam timed out after <minutes> minutes; context deadline exceeded 11:18:47 UTC · failed at e2e · test Customer should create an HCP cluster and validate TLS certificates · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:114]: Unexpected error: <fmt.wrapErrors \| 0xc0013f4bd0>: failed to create NodePool np-1, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group tls-endpoint-cluster-tjpbzb, error: failed waiting for nodepool="np-1" for cluster "tls-endpoint-hcp-cluster" in resourcegroup="tls-endpoint-cluster-tjpbzb" to finish creating: context deadline exceeded { msg: "failed to create NodePool np-1, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group tls-endpoint-cluster-tjpbzb, error: failed waiting for nodepool=\"np-1\" for cluster \"tls-endpoint-hcp-cluster\" in resourcegroup=\"tls-endpoint-cluster-tjpbzb\" to finish creating: context deadline exceeded", errs: [ <errors.errorString \| 0xc0009ed300>{ s: "timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group tls-endpoint-cluster-tjpbzb", }, <fmt.wrapError \| 0xc000f4a800>{ msg: "failed waiting for nodepool=\"np-1\" for cluster \"tls-endpoint-hcp-cluster\" in resourcegroup=\"tls-endpoint-cluster-tjpbzb\" to finish creating: context deadline exceeded", err: <context.deadlineExceededError>{}, }, ], } occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:114]: Unexpected error: <fmt.wrapErrors \| 0xc0013f4bd0>: failed to create NodePool np-1, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group tls-endpoint-cluster-tjpbzb, error: failed waiting for nodepool="np-1" for cluster "tls-endpoint-hcp-cluster" in resourcegroup="tls-endpoint-cluster-tjpbzb" to finish creating: context deadline exceeded { msg: "failed to create NodePool np-1, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group tls-endpoint-cluster-tjpbzb, error: failed waiting for nodepool=\"np-1\" for cluster \"tls-endpoint-hcp-cluster\" in resourcegroup=\"tls-endpoint-cluster-tjpbzb\" to finish creating: context deadline exceeded", errs: [ <errors.errorString \| 0xc0009ed300>{ s: "timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group tls-endpoint-cluster-tjpbzb", }, <fmt.wrapError \| 0xc000f4a800>{ msg: "failed waiting for nodepool=\"np-1\" for cluster \"tls-endpoint-hcp-cluster\" in resourcegroup=\"tls-endpoint-cluster-tjpbzb\" to finish creating: context deadline exceeded", err: <context.deadlineExceededError>{}, }, ], } occurred ERROR CODE: InternalServerError; detail message insufficient public IP address quota: required <count>, available <count>; provider Microsoft.RedHatOpenShift 11:18:47 UTC · failed at e2e · test Customer should be able to create a cluster with an external auth config and get the external auth config · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:93]: Unexpected error: <fmt.wrapError \| 0xc000a802a0>: failed to create HCP cluster ea-cluster: failed waiting for cluster="ea-cluster" in resourcegroup="external-auth-cluster-66kqnv" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/dca31cdc-d199-43d3-b94c-109078e85b87 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/dca31cdc-d199-43d3-b94c-109078e85b87", "name": "dca31cdc-d199-43d3-b94c-109078e85b87", "status": "Failed", "startTime": "2026-03-04T11:44:31.085504793Z", "endTime": "2026-03-04T11:49:48.850191595Z", "error": { "code": "InternalServerError", "message": "insufficient public IP address quota: required 2, available 0" } } -------------------------------------------------------------------------------- { msg: "failed to create HCP cluster ea-cluster: failed waiting for cluster=\"ea-cluster\" in resourcegroup=\"external-auth-cluster-66kqnv\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/dca31cdc-d199-43d3-b94c-109078e85b87\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/dca31cdc-d199-43d3-b94c-109078e85b87\",\n \"name\": \"dca31cdc-d199-43d3-b94c-109078e85b87\",\n \"status\": \"Failed\",\n \"startTime\": \"2026-03-04T11:44:31.085504793Z\",\n \"endTime\": \"2026-03-04T11:49:48.850191595Z\",\n \"error\": {\n \"code\": \"InternalServerError\",\n \"message\": \"insufficient public IP address quota: required 2, available 0\"\n }\n}\n--------------------------------------------------------------------------------\n", err: <fmt.wrapError \| 0xc000a80280>{ msg: "failed waiting for cluster=\"ea-cluster\" in resourcegroup=\"external-auth-cluster-66kqnv\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/dca31cdc-d199-43d3-b94c-109078e85b87\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/dca31cdc-d199-43d3-b94c-109078e85b87\",\n \"name\": \"dca31cdc-d199-43d3-b94c-109078e85b87\",\n \"status\": \"Failed\",\n \"startTime\": \"2026-03-04T11:44:31.085504793Z\",\n \"endTime\": \"2026-03-04T11:49:48.850191595Z\",\n \"error\": {\n \"code\": \"InternalServerError\",\n \"message\": \"insufficient public IP address quota: required 2, available 0\"\n }\n}\n--------------------------------------------------------------------------------\n", err: <exported.ResponseError \| 0xc000e9a120>{ ErrorCode: "InternalServerError", StatusCode: 200, RawResponse: { Status: "200 OK", StatusCode: 200, Proto: "HTTP/2.0", ProtoMajor: 2, ProtoMinor: 0, Header: { "Content-Length": ["499"], "X-Ms-Ratelimit-Remaining-Subscription-Global-Reads": ["3749"], "X-Ms-Correlation-Request-Id": [ "f84a4c0c-f1d0-4ea2-9df8-e7ed7e57b882", ], "X-Ms-Routing-Request-Id": [ "CANADACENTRAL:20260304T114953Z:f84a4c0c-f1d0-4ea2-9df8-e7ed7e57b882", ], "Date": [ "Wed, 04 Mar 2026 11:49:52 GMT", ], "Pragma": ["no-cache"], "Content-Type": ["application/json"], "X-Ms-Operation-Identifier": [ "appId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,tenantId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,objectId=0e96dbb7-17a2-4393-b2d3-d84524e39c32/canadacentral/55274095-8948-4add-bada-7f6aff2a7024", ], "X-Content-Type-Options": ["nosniff"], "X-Cache": ["CONFIG_NOCACHE"], "Cache-Control": ["no-cache"], "Expires": ["-1"], ... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output occurred fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:93]: Unexpected error: <fmt.wrapError \| 0xc000a802a0>: failed to create HCP cluster ea-cluster: failed waiting for cluster="ea-cluster" in resourcegroup="external-auth-cluster-66kqnv" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/dca31cdc-d199-43d3-b94c-109078e85b87 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/dca31cdc-d199-43d3-b94c-109078e85b87", "name": "dca31cdc-d199-43d3-b94c-109078e85b87", "status": "Failed", "startTime": "2026-03-04T11:44:31.085504793Z", "endTime": "2026-03-04T11:49:48.850191595Z", "error": { "code": "InternalServerError", "message": "insufficient public IP address quota: required 2, available 0" } } -------------------------------------------------------------------------------- { msg: "failed to create HCP cluster ea-cluster: failed waiting for cluster=\"ea-cluster\" in resourcegroup=\"external-auth-cluster-66kqnv\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/dca31cdc-d199-43d3-b94c-109078e85b87\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/dca31cdc-d199-43d3-b94c-109078e85b87\",\n \"name\": \"dca31cdc-d199-43d3-b94c-109078e85b87\",\n \"status\": \"Failed\",\n \"startTime\": \"2026-03-04T11:44:31.085504793Z\",\n \"endTime\": \"2026-03-04T11:49:48.850191595Z\",\n \"error\": {\n \"code\": \"InternalServerError\",\n \"message\": \"insufficient public IP address quota: required 2, available 0\"\n }\n}\n--------------------------------------------------------------------------------\n", err: <fmt.wrapError \| 0xc000a80280>{ msg: "failed waiting for cluster=\"ea-cluster\" in resourcegroup=\"external-auth-cluster-66kqnv\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/dca31cdc-d199-43d3-b94c-109078e85b87\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/dca31cdc-d199-43d3-b94c-109078e85b87\",\n \"name\": \"dca31cdc-d199-43d3-b94c-109078e85b87\",\n \"status\": \"Failed\",\n \"startTime\": \"2026-03-04T11:44:31.085504793Z\",\n \"endTime\": \"2026-03-04T11:49:48.850191595Z\",\n \"error\": {\n \"code\": \"InternalServerError\",\n \"message\": \"insufficient public IP address quota: required 2, available 0\"\n }\n}\n--------------------------------------------------------------------------------\n", err: <exported.ResponseError \| 0xc000e9a120>{ ErrorCode: "InternalServerError", StatusCode: 200, RawResponse: { Status: "200 OK", StatusCode: 200, Proto: "HTTP/2.0", ProtoMajor: 2, ProtoMinor: 0, Header: { "Content-Length": ["499"], "X-Ms-Ratelimit-Remaining-Subscription-Global-Reads": ["3749"], "X-Ms-Correlation-Request-Id": [ "f84a4c0c-f1d0-4ea2-9df8-e7ed7e57b882", ], "X-Ms-Routing-Request-Id": [ "CANADACENTRAL:20260304T114953Z:f84a4c0c-f1d0-4ea2-9df8-e7ed7e57b882", ], "Date": [ "Wed, 04 Mar 2026 11:49:52 GMT", ], "Pragma": ["no-cache"], "Content-Type": ["application/json"], "X-Ms-Operation-Identifier": [ "appId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,tenantId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,objectId=0e96dbb7-17a2-4393-b2d3-d84524e39c32/canadacentral/55274095-8948-4add-bada-7f6aff2a7024", ], "X-Content-Type-Options": ["nosniff"], "X-Cache": ["CONFIG_NOCACHE"], "Cache-Control": ["no-cache"], "Expires": ["-1"], ... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output occurred CreateNodePoolFromParam timed out after <minutes> minutes; context deadline exceeded 11:18:47 UTC · failed at e2e · test Customer should be able to create an HCP cluster and custom node pool osDisk size · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_nodepool_osdisk.go:94]: Unexpected error: <fmt.wrapErrors \| 0xc001079320>: failed to create NodePool nodepool-128GiB, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool nodepool-128GiB in resource group clusternp128-59thg6, error: failed waiting for nodepool="nodepool-128GiB" for cluster "hcp-cluster-np-128" in resourcegroup="clusternp128-59thg6" to finish creating: context deadline exceeded { msg: "failed to create NodePool nodepool-128GiB, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool nodepool-128GiB in resource group clusternp128-59thg6, error: failed waiting for nodepool=\"nodepool-128GiB\" for cluster \"hcp-cluster-np-128\" in resourcegroup=\"clusternp128-59thg6\" to finish creating: context deadline exceeded", errs: [ <errors.errorString \| 0xc0006a1820>{ s: "timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool nodepool-128GiB in resource group clusternp128-59thg6", }, <fmt.wrapError \| 0xc00079f180>{ msg: "failed waiting for nodepool=\"nodepool-128GiB\" for cluster \"hcp-cluster-np-128\" in resourcegroup=\"clusternp128-59thg6\" to finish creating: context deadline exceeded", err: <context.deadlineExceededError>{}, }, ], } occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_nodepool_osdisk.go:94]: Unexpected error: <fmt.wrapErrors \| 0xc001079320>: failed to create NodePool nodepool-128GiB, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool nodepool-128GiB in resource group clusternp128-59thg6, error: failed waiting for nodepool="nodepool-128GiB" for cluster "hcp-cluster-np-128" in resourcegroup="clusternp128-59thg6" to finish creating: context deadline exceeded { msg: "failed to create NodePool nodepool-128GiB, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool nodepool-128GiB in resource group clusternp128-59thg6, error: failed waiting for nodepool=\"nodepool-128GiB\" for cluster \"hcp-cluster-np-128\" in resourcegroup=\"clusternp128-59thg6\" to finish creating: context deadline exceeded", errs: [ <errors.errorString \| 0xc0006a1820>{ s: "timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool nodepool-128GiB in resource group clusternp128-59thg6", }, <fmt.wrapError \| 0xc00079f180>{ msg: "failed waiting for nodepool=\"nodepool-128GiB\" for cluster \"hcp-cluster-np-128\" in resourcegroup=\"clusternp128-59thg6\" to finish creating: context deadline exceeded", err: <context.deadlineExceededError>{}, }, ], } occurred CreateNodePoolFromParam timed out after <minutes> minutes; context deadline exceeded 11:18:47 UTC · failed at e2e · test Customer should be able to create a cluster with default autoscaling and a nodepool with autoscaling enabled · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:102]: Unexpected error: <fmt.wrapErrors \| 0xc000b345d0>: failed to create NodePool autoscale-np, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool autoscale-np in resource group np-autoscaling-8clfgs, error: failed waiting for nodepool="autoscale-np" for cluster "np-autoscale-cluster" in resourcegroup="np-autoscaling-8clfgs" to finish creating: context deadline exceeded { msg: "failed to create NodePool autoscale-np, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool autoscale-np in resource group np-autoscaling-8clfgs, error: failed waiting for nodepool=\"autoscale-np\" for cluster \"np-autoscale-cluster\" in resourcegroup=\"np-autoscaling-8clfgs\" to finish creating: context deadline exceeded", errs: [ <errors.errorString \| 0xc000b2d2e0>{ s: "timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool autoscale-np in resource group np-autoscaling-8clfgs", }, <fmt.wrapError \| 0xc0008b49c0>{ msg: "failed waiting for nodepool=\"autoscale-np\" for cluster \"np-autoscale-cluster\" in resourcegroup=\"np-autoscaling-8clfgs\" to finish creating: context deadline exceeded", err: <context.deadlineExceededError>{}, }, ], } occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:102]: Unexpected error: <fmt.wrapErrors \| 0xc000b345d0>: failed to create NodePool autoscale-np, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool autoscale-np in resource group np-autoscaling-8clfgs, error: failed waiting for nodepool="autoscale-np" for cluster "np-autoscale-cluster" in resourcegroup="np-autoscaling-8clfgs" to finish creating: context deadline exceeded { msg: "failed to create NodePool autoscale-np, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool autoscale-np in resource group np-autoscaling-8clfgs, error: failed waiting for nodepool=\"autoscale-np\" for cluster \"np-autoscale-cluster\" in resourcegroup=\"np-autoscaling-8clfgs\" to finish creating: context deadline exceeded", errs: [ <errors.errorString \| 0xc000b2d2e0>{ s: "timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool autoscale-np in resource group np-autoscaling-8clfgs", }, <fmt.wrapError \| 0xc0008b49c0>{ msg: "failed waiting for nodepool=\"autoscale-np\" for cluster \"np-autoscale-cluster\" in resourcegroup=\"np-autoscaling-8clfgs\" to finish creating: context deadline exceeded", err: <context.deadlineExceededError>{}, }, ], } occurred ERROR CODE: InternalServerError; detail message insufficient public IP address quota: required <count>, available <count>; provider Microsoft.RedHatOpenShift 11:18:47 UTC · failed at e2e · test Customer should be able to create an HCP cluster with back-level version 4.19 · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:147]: Unexpected error: <fmt.wrapError \| 0xc0011e4a80>: failed waiting for cluster="cluster-ver-4-19" in resourcegroup="rg-cluster-back-version-8xpmw9" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0407f0ef-72df-4f2b-93de-b19410d40d6d -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0407f0ef-72df-4f2b-93de-b19410d40d6d", "name": "0407f0ef-72df-4f2b-93de-b19410d40d6d", "status": "Failed", "startTime": "2026-03-04T11:42:21.423884671Z", "endTime": "2026-03-04T11:45:00.069078308Z", "error": { "code": "InternalServerError", "message": "insufficient public IP address quota: required 2, available 1" } } -------------------------------------------------------------------------------- { msg: "failed waiting for cluster=\"cluster-ver-4-19\" in resourcegroup=\"rg-cluster-back-version-8xpmw9\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0407f0ef-72df-4f2b-93de-b19410d40d6d\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0407f0ef-72df-4f2b-93de-b19410d40d6d\",\n \"name\": \"0407f0ef-72df-4f2b-93de-b19410d40d6d\",\n \"status\": \"Failed\",\n \"startTime\": \"2026-03-04T11:42:21.423884671Z\",\n \"endTime\": \"2026-03-04T11:45:00.069078308Z\",\n \"error\": {\n \"code\": \"InternalServerError\",\n \"message\": \"insufficient public IP address quota: required 2, available 1\"\n }\n}\n--------------------------------------------------------------------------------\n", err: <exported.ResponseError \| 0xc000ed1320>{ ErrorCode: "InternalServerError", StatusCode: 200, RawResponse: { Status: "200 OK", StatusCode: 200, Proto: "HTTP/2.0", ProtoMajor: 2, ProtoMinor: 0, Header: { "Pragma": ["no-cache"], "X-Ms-Request-Id": [ "1f116b65-75f8-4195-a4e7-62fd014a5842", ], "X-Ms-Ratelimit-Remaining-Subscription-Global-Reads": ["3749"], "X-Ms-Correlation-Request-Id": [ "6fb9a3eb-33d5-49fd-9ee8-9145d49e0a4e", ], "X-Content-Type-Options": ["nosniff"], "X-Ms-Operation-Identifier": [ "appId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,tenantId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,objectId=0e96dbb7-17a2-4393-b2d3-d84524e39c32/canadacentral/f0a897d5-d1ee-4de5-9c39-3e46359eafd8", ], "X-Ms-Ratelimit-Remaining-Subscription-Reads": ["249"], "X-Ms-Routing-Request-Id": [ "CANADACENTRAL:20260304T114506Z:6fb9a3eb-33d5-49fd-9ee8-9145d49e0a4e", ], "X-Cache": ["CONFIG_NOCACHE"], "Cache-Control": ["no-cache"], "Content-Type": ["application/json"], "Strict-Transport-Security": [ "max-age=31536000; includeSubDomains", ], "X-Msedge-Ref": [ "Ref A: 38725E90972A49A387B79ED183CDDF6F Ref B: CHI261140501034 Ref C: 2026-03-04T11:45:06Z", ], "Content-Length": ["499"], "Expires": ["-1"], "X-Envoy-Upstream-Service-Time": ["9"], "Date": [ "Wed, 04 Mar 2026 11:45:06 GMT", ], }, Body: <exported.nopClosingBytesReader \| 0xc0011e4960>{ s: [123, 10, 32, 32, 32, 32, 34, 105, 100, 34, 58, 32, 34, 47, 115, 117, 98, 115, 99, 114, 105, 112, 116, 105, 111, 110, 115, 47, 52, 48, 51, 100, 57, 100, 101, 57, 45, 49, 51, 50, 98, 45, 52, 57, 55, 52, 45, 57, 52, 97, 53, 45, 53, 98, 55, 56, 98, 100, 102, 97, 49, 57, 49, 101, 47, 112, 114, 111, 118, 105, 100, 101, 114, 115, 47, 77, 105, 99, 114, 111, 115, 111, 102, 116, 46, 82, 101, 100, 72, 97, 116, 79, 112, 101, 110, 83, 104, 105, 102, 116, 47, 108, 111, 99, 97, 116, 105, 111, 110, 115, 47, 117, 107, 115, 111, 117, 116, 104, 47, 104, 99, 112, 79, 112, 101, 114, 97, 116, 105, 111, 110, 83, 116, 97, 116, 117, 115, 101, 115, 47, 48, 52, 48, 55, 102, 48, 101, ... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:147]: Unexpected error: <fmt.wrapError \| 0xc0011e4a80>: failed waiting for cluster="cluster-ver-4-19" in resourcegroup="rg-cluster-back-version-8xpmw9" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0407f0ef-72df-4f2b-93de-b19410d40d6d -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0407f0ef-72df-4f2b-93de-b19410d40d6d", "name": "0407f0ef-72df-4f2b-93de-b19410d40d6d", "status": "Failed", "startTime": "2026-03-04T11:42:21.423884671Z", "endTime": "2026-03-04T11:45:00.069078308Z", "error": { "code": "InternalServerError", "message": "insufficient public IP address quota: required 2, available 1" } } -------------------------------------------------------------------------------- { msg: "failed waiting for cluster=\"cluster-ver-4-19\" in resourcegroup=\"rg-cluster-back-version-8xpmw9\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0407f0ef-72df-4f2b-93de-b19410d40d6d\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0407f0ef-72df-4f2b-93de-b19410d40d6d\",\n \"name\": \"0407f0ef-72df-4f2b-93de-b19410d40d6d\",\n \"status\": \"Failed\",\n \"startTime\": \"2026-03-04T11:42:21.423884671Z\",\n \"endTime\": \"2026-03-04T11:45:00.069078308Z\",\n \"error\": {\n \"code\": \"InternalServerError\",\n \"message\": \"insufficient public IP address quota: required 2, available 1\"\n }\n}\n--------------------------------------------------------------------------------\n", err: <exported.ResponseError \| 0xc000ed1320>{ ErrorCode: "InternalServerError", StatusCode: 200, RawResponse: { Status: "200 OK", StatusCode: 200, Proto: "HTTP/2.0", ProtoMajor: 2, ProtoMinor: 0, Header: { "Pragma": ["no-cache"], "X-Ms-Request-Id": [ "1f116b65-75f8-4195-a4e7-62fd014a5842", ], "X-Ms-Ratelimit-Remaining-Subscription-Global-Reads": ["3749"], "X-Ms-Correlation-Request-Id": [ "6fb9a3eb-33d5-49fd-9ee8-9145d49e0a4e", ], "X-Content-Type-Options": ["nosniff"], "X-Ms-Operation-Identifier": [ "appId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,tenantId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,objectId=0e96dbb7-17a2-4393-b2d3-d84524e39c32/canadacentral/f0a897d5-d1ee-4de5-9c39-3e46359eafd8", ], "X-Ms-Ratelimit-Remaining-Subscription-Reads": ["249"], "X-Ms-Routing-Request-Id": [ "CANADACENTRAL:20260304T114506Z:6fb9a3eb-33d5-49fd-9ee8-9145d49e0a4e", ], "X-Cache": ["CONFIG_NOCACHE"], "Cache-Control": ["no-cache"], "Content-Type": ["application/json"], "Strict-Transport-Security": [ "max-age=31536000; includeSubDomains", ], "X-Msedge-Ref": [ "Ref A: 38725E90972A49A387B79ED183CDDF6F Ref B: CHI261140501034 Ref C: 2026-03-04T11:45:06Z", ], "Content-Length": ["499"], "Expires": ["-1"], "X-Envoy-Upstream-Service-Time": ["9"], "Date": [ "Wed, 04 Mar 2026 11:45:06 GMT", ], }, Body: <exported.nopClosingBytesReader \| 0xc0011e4960>{ s: [123, 10, 32, 32, 32, 32, 34, 105, 100, 34, 58, 32, 34, 47, 115, 117, 98, 115, 99, 114, 105, 112, 116, 105, 111, 110, 115, 47, 52, 48, 51, 100, 57, 100, 101, 57, 45, 49, 51, 50, 98, 45, 52, 57, 55, 52, 45, 57, 52, 97, 53, 45, 53, 98, 55, 56, 98, 100, 102, 97, 49, 57, 49, 101, 47, 112, 114, 111, 118, 105, 100, 101, 114, 115, 47, 77, 105, 99, 114, 111, 115, 111, 102, 116, 46, 82, 101, 100, 72, 97, 116, 79, 112, 101, 110, 83, 104, 105, 102, 116, 47, 108, 111, 99, 97, 116, 105, 111, 110, 115, 47, 117, 107, 115, 111, 117, 116, 104, 47, 104, 99, 112, 79, 112, 101, 114, 97, 116, 105, 111, 110, 83, 116, 97, 116, 117, 115, 101, 115, 47, 48, 52, 48, 55, 102, 48, 101, ... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output occurred ERROR CODE: InternalServerError; detail message insufficient public IP address quota: required <count>, available <count>; provider Microsoft.RedHatOpenShift 11:18:47 UTC · failed at e2e · test Customer should be able to create node pools with ARM64-based VMs · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/arm64_nodepool.go:90]: Unexpected error: <fmt.wrapError \| 0xc0008963e0>: failed to create HCP cluster arm64-vm-hcp-cluster: failed waiting for cluster="arm64-vm-hcp-cluster" in resourcegroup="arm64-vm-cluster-4ljw7f" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/8e2e6926-ad32-404f-9710-045e248d82cd -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/8e2e6926-ad32-404f-9710-045e248d82cd", "name": "8e2e6926-ad32-404f-9710-045e248d82cd", "status": "Failed", "startTime": "2026-03-04T11:42:34.945511327Z", "endTime": "2026-03-04T11:49:48.715948579Z", "error": { "code": "InternalServerError", "message": "insufficient public IP address quota: required 2, available 1" } } -------------------------------------------------------------------------------- { msg: "failed to create HCP cluster arm64-vm-hcp-cluster: failed waiting for cluster=\"arm64-vm-hcp-cluster\" in resourcegroup=\"arm64-vm-cluster-4ljw7f\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/8e2e6926-ad32-404f-9710-045e248d82cd\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/8e2e6926-ad32-404f-9710-045e248d82cd\",\n \"name\": \"8e2e6926-ad32-404f-9710-045e248d82cd\",\n \"status\": \"Failed\",\n \"startTime\": \"2026-03-04T11:42:34.945511327Z\",\n \"endTime\": \"2026-03-04T11:49:48.715948579Z\",\n \"error\": {\n \"code\": \"InternalServerError\",\n \"message\": \"insufficient public IP address quota: required 2, available 1\"\n }\n}\n--------------------------------------------------------------------------------\n", err: <fmt.wrapError \| 0xc0008963c0>{ msg: "failed waiting for cluster=\"arm64-vm-hcp-cluster\" in resourcegroup=\"arm64-vm-cluster-4ljw7f\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/8e2e6926-ad32-404f-9710-045e248d82cd\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/8e2e6926-ad32-404f-9710-045e248d82cd\",\n \"name\": \"8e2e6926-ad32-404f-9710-045e248d82cd\",\n \"status\": \"Failed\",\n \"startTime\": \"2026-03-04T11:42:34.945511327Z\",\n \"endTime\": \"2026-03-04T11:49:48.715948579Z\",\n \"error\": {\n \"code\": \"InternalServerError\",\n \"message\": \"insufficient public IP address quota: required 2, available 1\"\n }\n}\n--------------------------------------------------------------------------------\n", err: <exported.ResponseError \| 0xc00128c5a0>{ ErrorCode: "InternalServerError", StatusCode: 200, RawResponse: { Status: "200 OK", StatusCode: 200, Proto: "HTTP/2.0", ProtoMajor: 2, ProtoMinor: 0, Header: { "Content-Length": ["499"], "X-Ms-Correlation-Request-Id": [ "8649fcef-a89d-4ebe-a831-d5aa547b7859", ], "X-Ms-Routing-Request-Id": [ "CANADACENTRAL:20260304T114951Z:8649fcef-a89d-4ebe-a831-d5aa547b7859", ], "Strict-Transport-Security": [ "max-age=31536000; includeSubDomains", ], "Date": [ "Wed, 04 Mar 2026 11:49:50 GMT", ], "Content-Type": ["application/json"], "X-Ms-Operation-Identifier": [ "appId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,tenantId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,objectId=0e96dbb7-17a2-4393-b2d3-d84524e39c32/canadacentral/43fae7b6-ce0d-421e-b423-059eca2efbf5", ], "X-Ms-Ratelimit-Remaining-Subscription-Reads": ["249"], "X-Cache": ["CONFIG_NOCACHE"], "Cache-Control": ["no-cache"], ... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output occurred fail [github.com/Azure/ARO-HCP/test/e2e/arm64_nodepool.go:90]: Unexpected error: <fmt.wrapError \| 0xc0008963e0>: failed to create HCP cluster arm64-vm-hcp-cluster: failed waiting for cluster="arm64-vm-hcp-cluster" in resourcegroup="arm64-vm-cluster-4ljw7f" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/8e2e6926-ad32-404f-9710-045e248d82cd -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/8e2e6926-ad32-404f-9710-045e248d82cd", "name": "8e2e6926-ad32-404f-9710-045e248d82cd", "status": "Failed", "startTime": "2026-03-04T11:42:34.945511327Z", "endTime": "2026-03-04T11:49:48.715948579Z", "error": { "code": "InternalServerError", "message": "insufficient public IP address quota: required 2, available 1" } } -------------------------------------------------------------------------------- { msg: "failed to create HCP cluster arm64-vm-hcp-cluster: failed waiting for cluster=\"arm64-vm-hcp-cluster\" in resourcegroup=\"arm64-vm-cluster-4ljw7f\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/8e2e6926-ad32-404f-9710-045e248d82cd\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/8e2e6926-ad32-404f-9710-045e248d82cd\",\n \"name\": \"8e2e6926-ad32-404f-9710-045e248d82cd\",\n \"status\": \"Failed\",\n \"startTime\": \"2026-03-04T11:42:34.945511327Z\",\n \"endTime\": \"2026-03-04T11:49:48.715948579Z\",\n \"error\": {\n \"code\": \"InternalServerError\",\n \"message\": \"insufficient public IP address quota: required 2, available 1\"\n }\n}\n--------------------------------------------------------------------------------\n", err: <fmt.wrapError \| 0xc0008963c0>{ msg: "failed waiting for cluster=\"arm64-vm-hcp-cluster\" in resourcegroup=\"arm64-vm-cluster-4ljw7f\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/8e2e6926-ad32-404f-9710-045e248d82cd\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/8e2e6926-ad32-404f-9710-045e248d82cd\",\n \"name\": \"8e2e6926-ad32-404f-9710-045e248d82cd\",\n \"status\": \"Failed\",\n \"startTime\": \"2026-03-04T11:42:34.945511327Z\",\n \"endTime\": \"2026-03-04T11:49:48.715948579Z\",\n \"error\": {\n \"code\": \"InternalServerError\",\n \"message\": \"insufficient public IP address quota: required 2, available 1\"\n }\n}\n--------------------------------------------------------------------------------\n", err: <exported.ResponseError \| 0xc00128c5a0>{ ErrorCode: "InternalServerError", StatusCode: 200, RawResponse: { Status: "200 OK", StatusCode: 200, Proto: "HTTP/2.0", ProtoMajor: 2, ProtoMinor: 0, Header: { "Content-Length": ["499"], "X-Ms-Correlation-Request-Id": [ "8649fcef-a89d-4ebe-a831-d5aa547b7859", ], "X-Ms-Routing-Request-Id": [ "CANADACENTRAL:20260304T114951Z:8649fcef-a89d-4ebe-a831-d5aa547b7859", ], "Strict-Transport-Security": [ "max-age=31536000; includeSubDomains", ], "Date": [ "Wed, 04 Mar 2026 11:49:50 GMT", ], "Content-Type": ["application/json"], "X-Ms-Operation-Identifier": [ "appId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,tenantId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,objectId=0e96dbb7-17a2-4393-b2d3-d84524e39c32/canadacentral/43fae7b6-ce0d-421e-b423-059eca2efbf5", ], "X-Ms-Ratelimit-Remaining-Subscription-Reads": ["249"], "X-Cache": ["CONFIG_NOCACHE"], "Cache-Control": ["no-cache"], ... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output occurred ERROR CODE: InternalServerError; detail code PublicIPCountLimitReached; detail message Cannot create more than 300 public IP addresses for this subscription in this region.; provider Microsoft.Network 11:18:47 UTC · failed at e2e · test Authorized CIDRs Connectivity should allow API access only from authorized VM IP · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:133]: Unexpected error: <fmt.wrapError \| 0xc001050900>: failed to create HCP cluster cidr-connectivity-test: failed waiting for cluster="cidr-connectivity-test" in resourcegroup="e2e-cidr-connectivity-42hshj" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/6ee8ec73-61b8-4344-ad00-bba98e2702a7 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/6ee8ec73-61b8-4344-ad00-bba98e2702a7", "name": "6ee8ec73-61b8-4344-ad00-bba98e2702a7", "status": "Failed", "startTime": "2026-03-04T11:27:28.76644337Z", "endTime": "2026-03-04T11:33:55.189840009Z", "error": { "code": "InternalServerError", "message": "PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-42hshj--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi6tet62ut5t7l4015hb57ec6maei-3\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"PublicIPCountLimitReached\",\n \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n \"details\": []\n }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-42hshj--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi6tet62ut5t7l4015hb57ec6maei-4\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"PublicIPCountLimitReached\",\n \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n \"details\": []\n }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-42hshj--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi6tet62ut5t7l4015hb57ec6maei-5\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"PublicIPCountLimitReached\",\n \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n \"details\": []\n }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-42hshj--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi6tet62ut5t7l4015hb57ec6maei-6\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"PublicIPCountLimitReached\",\n \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n \"details\": []\n }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-42hshj--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi6tet62ut5t7l4015hb57ec6maei-7\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"PublicIPCountLimitReached\",\n \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n \"details\": []\n }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-42hshj--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi6tet62ut5t7l4015hb57ec6maei-8\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"PublicIPCountLimitReached\",\n \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n \"details\": []\n }\n}\n--------------------------------------------------------------------------------\n" } } -------------------------------------------------------------------------------- { msg: "failed to create HCP cluster cidr-connectivity-test: failed waiting for cluster=\"cidr-connectivity-test\" in resourcegroup=\"e2e-cidr-connectivity-42hshj\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/6ee8ec73-61b8-4344-ad00-bba98e2702a7\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/6ee8ec73-61b8-4344-ad00-bba98e2702a7\",\n \"name\": \"6ee8ec73-61b8-4344-ad00-bba98e2702a7\",\n \"status\": \"Failed\",\n \"startTime\": \"2026-03-04T11:27:28.76644337Z\",\n \"endTime\": \"2026-03-04T11:33:55.189840009Z\",\n \"error\": {\n \"code\": \"InternalServerError\",\n \"message\": \"PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-42hshj--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi6tet62ut5t7l4015hb57ec6maei-3\\n--------------------------------------------------------------------------------\\nRESPONSE 400: 400 Bad Request\\nERROR CODE: PublicIPCountLimitReached\\n--------------------------------------------------------------------------------\\n{\\n \\\"error\\\": {\\n \\\"code\\\": \\\"PublicIPCountLimitReached\\\",\\n \\\"message\\\": \\\"Cannot create more than 300 public IP addresses for this subscription in this region.\\\",\\n \\\"details\\\": []\\n }\\n}\\n--------------------------------------------------------------------------------\\n\\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-42hshj--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi6tet62ut5t7l4015hb57ec6maei-4\\n--------------------------------------------------------------------------------\\nRESPONSE 400: 400 Bad Request\\nERROR CODE: PublicIPCountLimitReached\\n--------------------------------------------------------------------------------\\n{\\n \\\"error\\\": {\\n \\\"code\\\": \\\"PublicIPCountLimitReached\\\",\\n \\\"message\\\": \\\"Cannot create more than 300 public IP addresses for this subscription in this region.\\\",\\n \\\"details\\\": []\\n }\\n}\\n--------------------------------------------------------------------------------\\n\\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-42hshj--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi6tet62ut5t7l4015hb57ec6maei-5\\n--------------------------------------------------------------------------------\\nRESPONSE 400: 400 Bad Request\\nERROR CODE: PublicIPCountLimitReached\\n--------------------------------------------------------------------------------\\n{\\n \\\"error\\\": {\\n \\\"code\\\": \\\"PublicIPCountLimitReached\\\",\\n \\\"message\\\": \\\"Cannot create more than 300 public IP addresses for this subscription in this region.\\\",\\n \\\"details\\\": []\\n }\\n}\\n--------------------------------------------------------------------------------\\n\\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-42hshj--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi6tet62ut5t7l4015hb57ec6maei-6\\n--------------------------------------------------------------------------------\\nRESPONSE 400: 400 Bad Request\\nERROR CODE: PublicIPCountLimitReached\\n--------------------------------------------------------------------------------\\n{\\n \\\"error\\\": {\\n \\\"code\\\": \\\"PublicIPCountLimitReached\\\",\\n \\\"message\\\": \\\"Cannot create more than 300 public IP addresses f... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:133]: Unexpected error: <fmt.wrapError \| 0xc001050900>: failed to create HCP cluster cidr-connectivity-test: failed waiting for cluster="cidr-connectivity-test" in resourcegroup="e2e-cidr-connectivity-42hshj" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/6ee8ec73-61b8-4344-ad00-bba98e2702a7 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/6ee8ec73-61b8-4344-ad00-bba98e2702a7", "name": "6ee8ec73-61b8-4344-ad00-bba98e2702a7", "status": "Failed", "startTime": "2026-03-04T11:27:28.76644337Z", "endTime": "2026-03-04T11:33:55.189840009Z", "error": { "code": "InternalServerError", "message": "PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-42hshj--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi6tet62ut5t7l4015hb57ec6maei-3\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"PublicIPCountLimitReached\",\n \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n \"details\": []\n }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-42hshj--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi6tet62ut5t7l4015hb57ec6maei-4\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"PublicIPCountLimitReached\",\n \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n \"details\": []\n }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-42hshj--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi6tet62ut5t7l4015hb57ec6maei-5\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"PublicIPCountLimitReached\",\n \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n \"details\": []\n }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-42hshj--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi6tet62ut5t7l4015hb57ec6maei-6\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"PublicIPCountLimitReached\",\n \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n \"details\": []\n }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-42hshj--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi6tet62ut5t7l4015hb57ec6maei-7\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"PublicIPCountLimitReached\",\n \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n \"details\": []\n }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-42hshj--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi6tet62ut5t7l4015hb57ec6maei-8\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"PublicIPCountLimitReached\",\n \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n \"details\": []\n }\n}\n--------------------------------------------------------------------------------\n" } } -------------------------------------------------------------------------------- { msg: "failed to create HCP cluster cidr-connectivity-test: failed waiting for cluster=\"cidr-connectivity-test\" in resourcegroup=\"e2e-cidr-connectivity-42hshj\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/6ee8ec73-61b8-4344-ad00-bba98e2702a7\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/6ee8ec73-61b8-4344-ad00-bba98e2702a7\",\n \"name\": \"6ee8ec73-61b8-4344-ad00-bba98e2702a7\",\n \"status\": \"Failed\",\n \"startTime\": \"2026-03-04T11:27:28.76644337Z\",\n \"endTime\": \"2026-03-04T11:33:55.189840009Z\",\n \"error\": {\n \"code\": \"InternalServerError\",\n \"message\": \"PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-42hshj--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi6tet62ut5t7l4015hb57ec6maei-3\\n--------------------------------------------------------------------------------\\nRESPONSE 400: 400 Bad Request\\nERROR CODE: PublicIPCountLimitReached\\n--------------------------------------------------------------------------------\\n{\\n \\\"error\\\": {\\n \\\"code\\\": \\\"PublicIPCountLimitReached\\\",\\n \\\"message\\\": \\\"Cannot create more than 300 public IP addresses for this subscription in this region.\\\",\\n \\\"details\\\": []\\n }\\n}\\n--------------------------------------------------------------------------------\\n\\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-42hshj--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi6tet62ut5t7l4015hb57ec6maei-4\\n--------------------------------------------------------------------------------\\nRESPONSE 400: 400 Bad Request\\nERROR CODE: PublicIPCountLimitReached\\n--------------------------------------------------------------------------------\\n{\\n \\\"error\\\": {\\n \\\"code\\\": \\\"PublicIPCountLimitReached\\\",\\n \\\"message\\\": \\\"Cannot create more than 300 public IP addresses for this subscription in this region.\\\",\\n \\\"details\\\": []\\n }\\n}\\n--------------------------------------------------------------------------------\\n\\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-42hshj--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi6tet62ut5t7l4015hb57ec6maei-5\\n--------------------------------------------------------------------------------\\nRESPONSE 400: 400 Bad Request\\nERROR CODE: PublicIPCountLimitReached\\n--------------------------------------------------------------------------------\\n{\\n \\\"error\\\": {\\n \\\"code\\\": \\\"PublicIPCountLimitReached\\\",\\n \\\"message\\\": \\\"Cannot create more than 300 public IP addresses for this subscription in this region.\\\",\\n \\\"details\\\": []\\n }\\n}\\n--------------------------------------------------------------------------------\\n\\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-42hshj--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi6tet62ut5t7l4015hb57ec6maei-6\\n--------------------------------------------------------------------------------\\nRESPONSE 400: 400 Bad Request\\nERROR CODE: PublicIPCountLimitReached\\n--------------------------------------------------------------------------------\\n{\\n \\\"error\\\": {\\n \\\"code\\\": \\\"PublicIPCountLimitReached\\\",\\n \\\"message\\\": \\\"Cannot create more than 300 public IP addresses f... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output occurred ERROR CODE: InternalServerError; detail code PublicIPCountLimitReached; detail message Cannot create more than 300 public IP addresses for this subscription in this region.; provider Microsoft.Network 11:18:47 UTC · failed at e2e · test Customer should be able to create several HCP clusters in their customer resource group, but not in the same managed resource group · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:129]: Unexpected error: <exported.ResponseError \| 0xc001696270>: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0b35155f-1c87-4a27-b264-b5ffaf2df592 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0b35155f-1c87-4a27-b264-b5ffaf2df592", "name": "0b35155f-1c87-4a27-b264-b5ffaf2df592", "status": "Failed", "startTime": "2026-03-04T11:27:10.141510238Z", "endTime": "2026-03-04T11:33:01.507880583Z", "error": { "code": "InternalServerError", "message": "PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/customer-rg-42hl9p--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi3dcu806c95qg7cc9io5migfbgq6-0\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"PublicIPCountLimitReached\",\n \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n \"details\": []\n }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/customer-rg-42hl9p--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi3dcu806c95qg7cc9io5migfbgq6-1\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"PublicIPCountLimitReached\",\n \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n \"details\": []\n }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/customer-rg-42hl9p--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi3dcu806c95qg7cc9io5migfbgq6-2\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"PublicIPCountLimitReached\",\n \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n \"details\": []\n }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/customer-rg-42hl9p--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi3dcu806c95qg7cc9io5migfbgq6-3\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"PublicIPCountLimitReached\",\n \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n \"details\": []\n }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/customer-rg-42hl9p--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi3dcu806c95qg7cc9io5migfbgq6-4\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"PublicIPCountLimitReached\",\n \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n \"details\": []\n }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/customer-rg-42hl9p--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi3dcu806c95qg7cc9io5migfbgq6-5\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"PublicIPCountLimitReached\",\n \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n \"details\": []\n }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/customer-rg-42hl9p--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi3dcu806c95qg7cc9io5migfbgq6-6\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"PublicIPCountLimitReached\",\n \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n \"details\": []\n }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/customer-rg-42hl9p--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi3dcu806c95qg7cc9io5migfbgq6-7\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"PublicIPCountLimitReached\",\n \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n \"details\": []\n }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/customer-rg-42hl9p--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi3dcu806c95qg7cc9io5migfbgq6-8\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"PublicIPCountLimitReached\",\n \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n \"details\": []\n }\n}\n--------------------------------------------------------------------------------\n" } } -------------------------------------------------------------------------------- { ErrorCode: "InternalServerError", StatusCode: 200, RawResponse: { Status: "200 OK", StatusCode: 200, Proto: "HTTP/2.0", ProtoMajor: 2, ProtoMinor: 0, Header: { "Cache-Control": ["no-cache"], "X-Ms-Ratelimit-Remaining-Subscription-Reads": ["249"], "Content-Length": ["7006"], "X-Ms-Request-Id": [ "8f5597b5-16fa-4df6-bd73-65ab0a1bb005", ], "X-Envoy-Upstream-Service-Time": ["10"], "X-Ms-Ratelimit-Remaining-Subscription-Global-Reads": ["3749"], "X-Ms-Correlation-Request-Id": [ "bd0a002f-0f30-4cf8-91cd-38e2956ccb93", ], "Content-Type": ["application/json"], "X-Ms-Operation-Identifier": [ "appId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,tenantId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,objectId=0e96dbb7-17a2-4393-b2d3-d84524e39c32/canadacentral/eec24277-b57a-425a-a5e4-071fef365907", ], "X-Ms-Routing-Request-Id": [ "CANADACENTRAL:20260304T113304Z:bd0a002f-0f30-4cf8-91cd-38e2956ccb93", ], "Strict-Transport-Security": [ "max-age=31536000; includeSubDomains", ], "X-Content-Type-Options": ["nosniff"], "X-Msedge-Ref": [ "Ref A: 7BD958C08EC847A19E41507ED24FBCAF Ref B: CHI261140510034 Ref C: 2026-03-04T11:33:03Z", ], "Pragma": ["no-cache"], "Expires": ["-1"], "X-Cache": ["CONFIG_NOCACHE"], "Date": [ "Wed, 04 Mar 2026 11:33:03 GMT", ], }, Body: <exported.nopClosingBytesReader \| 0xc000e12000>{ s: [123, 10, 32, 32, 32, 32, 34, 105, 100, 34, 58, 32, 34, 47, 115, 117, 98, 115, 99, 114, 105, 112, 116, 105, 111, 110, 115, 47, 52, 48, 51, 100, 57, 100, 101, 57, 45, 49, 51, 50, 98, 45, 52, 57, 55, 52, 45, 57, 52, 97, 53, 45, 53, 98, 55, 56, 98, 100, 102, 97, 49, 57, 49, 101, 47, 112, 114, 111, 118, 105, 100, 101, 114, 115, 47, 77, 105, 99, 114, 111, 115, 111, 102, 116, 46, 82, 101, 100, 72, 97, 116, 79, 112, 101, 110, 83, 104, 105, 102, 116, 47, 108, 111, 99, 97, 116, 105, 111, 110, 115, 47, 117, 107, 115, 111, 117, 116, 104, 47, 104, 99, 112, 79, 112, 101, 114, 97, 116, 105, 111, 110, 83, 116, 97, 116, 117, 115, 101, 115, 47, 48, 98, 51, 53, 49, 53, 53, 102, 45, 49, 99, 56, 55, 45, 52, 97, 50, 55, 45, 98, 50, 54, 52, 45, 98, 53, 102, 102, 97, 102, 50, 100, 102, 53, 57, 50, 34, 44, 10, 32, 32, 32, 32, 34, 110, 97, 109, 101, 34, 58, 32, 34, 48, 98, 51, 53, 49, 53, 53, 102, 45, 49, 99, 56, 55, 45, 52, 97, 50, 55, 45, 98, 50, 54, 52, 45, 98, 53, 102, 102, 97, 102, 50, 100, 102, 53, 57, 50, 34, 44, 10, 32, 32, 32, 32, 34, 115, 116, 97, 116, 117, 115, 34, 58, 32, 34, 70, 97, 105, 108, 101, 100, 34, 44, 10, 32, 32, 32, 32, 34, 115, 116, 97, 114, 116, 84, 105, 109, 101, 34, 58, 32, 34, 50, 48, 50, 54, 45, 48, 51, 45, 48, 52, 84, 49, 49, 58, 50, 55, 58, 49, 48, 46, 49, 52, 49, 53, 49, 48, 50, 51, 56, 90, 34, 44, 10, 32, 32, 32, 32, 34, 101, 110, 100, 84, 105, 109, 101, 34, 58, 32, 34, 50, 48, 50, 54, 45, 48, 51, 45, 48, 52, 84, 49, 49, 58, 51, 51, 58, 48, 49, 46, 53, 48, 55, 56, 56, 48, 53, 56, 51, 90, 34, 44, 10, 32, 32, 32, 32, 34, 101, 114, 114, 111, 114, 34, 58, 32, 123, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 99, 111, 100, 101, 34, 58, 32, 34, 73, 110, 116, 101, 114, 110, 97, 108, 83, 101, 114, 118, 101, 114, 69, 114, 114, 111, 114, 34, 44, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 109, 101, 115, 115, 97, 103, 101, 34, 58, 32, 34, 80, 85, 84, 32, 104, 116, 116, 112, 115, 58, 47, 47, 109, 97, 110, 97, 103, 101, 109, 101, 110, 116, 46, 97, 122, 117, 114, 101, 46, 99, 111, 109, 47, 115, 117, 98, 115, 99, 114, 105, 11... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output occurred fail [github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:129]: Unexpected error: <exported.ResponseError \| 0xc001696270>: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0b35155f-1c87-4a27-b264-b5ffaf2df592 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0b35155f-1c87-4a27-b264-b5ffaf2df592", "name": "0b35155f-1c87-4a27-b264-b5ffaf2df592", "status": "Failed", "startTime": "2026-03-04T11:27:10.141510238Z", "endTime": "2026-03-04T11:33:01.507880583Z", "error": { "code": "InternalServerError", "message": "PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/customer-rg-42hl9p--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi3dcu806c95qg7cc9io5migfbgq6-0\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"PublicIPCountLimitReached\",\n \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n \"details\": []\n }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/customer-rg-42hl9p--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi3dcu806c95qg7cc9io5migfbgq6-1\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"PublicIPCountLimitReached\",\n \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n \"details\": []\n }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/customer-rg-42hl9p--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi3dcu806c95qg7cc9io5migfbgq6-2\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"PublicIPCountLimitReached\",\n \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n \"details\": []\n }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/customer-rg-42hl9p--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi3dcu806c95qg7cc9io5migfbgq6-3\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"PublicIPCountLimitReached\",\n \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n \"details\": []\n }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/customer-rg-42hl9p--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi3dcu806c95qg7cc9io5migfbgq6-4\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"PublicIPCountLimitReached\",\n \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n \"details\": []\n }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/customer-rg-42hl9p--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi3dcu806c95qg7cc9io5migfbgq6-5\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"PublicIPCountLimitReached\",\n \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n \"details\": []\n }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/customer-rg-42hl9p--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi3dcu806c95qg7cc9io5migfbgq6-6\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"PublicIPCountLimitReached\",\n \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n \"details\": []\n }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/customer-rg-42hl9p--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi3dcu806c95qg7cc9io5migfbgq6-7\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"PublicIPCountLimitReached\",\n \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n \"details\": []\n }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/customer-rg-42hl9p--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi3dcu806c95qg7cc9io5migfbgq6-8\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"PublicIPCountLimitReached\",\n \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n \"details\": []\n }\n}\n--------------------------------------------------------------------------------\n" } } -------------------------------------------------------------------------------- { ErrorCode: "InternalServerError", StatusCode: 200, RawResponse: { Status: "200 OK", StatusCode: 200, Proto: "HTTP/2.0", ProtoMajor: 2, ProtoMinor: 0, Header: { "Cache-Control": ["no-cache"], "X-Ms-Ratelimit-Remaining-Subscription-Reads": ["249"], "Content-Length": ["7006"], "X-Ms-Request-Id": [ "8f5597b5-16fa-4df6-bd73-65ab0a1bb005", ], "X-Envoy-Upstream-Service-Time": ["10"], "X-Ms-Ratelimit-Remaining-Subscription-Global-Reads": ["3749"], "X-Ms-Correlation-Request-Id": [ "bd0a002f-0f30-4cf8-91cd-38e2956ccb93", ], "Content-Type": ["application/json"], "X-Ms-Operation-Identifier": [ "appId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,tenantId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,objectId=0e96dbb7-17a2-4393-b2d3-d84524e39c32/canadacentral/eec24277-b57a-425a-a5e4-071fef365907", ], "X-Ms-Routing-Request-Id": [ "CANADACENTRAL:20260304T113304Z:bd0a002f-0f30-4cf8-91cd-38e2956ccb93", ], "Strict-Transport-Security": [ "max-age=31536000; includeSubDomains", ], "X-Content-Type-Options": ["nosniff"], "X-Msedge-Ref": [ "Ref A: 7BD958C08EC847A19E41507ED24FBCAF Ref B: CHI261140510034 Ref C: 2026-03-04T11:33:03Z", ], "Pragma": ["no-cache"], "Expires": ["-1"], "X-Cache": ["CONFIG_NOCACHE"], "Date": [ "Wed, 04 Mar 2026 11:33:03 GMT", ], }, Body: <exported.nopClosingBytesReader \| 0xc000e12000>{ s: [123, 10, 32, 32, 32, 32, 34, 105, 100, 34, 58, 32, 34, 47, 115, 117, 98, 115, 99, 114, 105, 112, 116, 105, 111, 110, 115, 47, 52, 48, 51, 100, 57, 100, 101, 57, 45, 49, 51, 50, 98, 45, 52, 57, 55, 52, 45, 57, 52, 97, 53, 45, 53, 98, 55, 56, 98, 100, 102, 97, 49, 57, 49, 101, 47, 112, 114, 111, 118, 105, 100, 101, 114, 115, 47, 77, 105, 99, 114, 111, 115, 111, 102, 116, 46, 82, 101, 100, 72, 97, 116, 79, 112, 101, 110, 83, 104, 105, 102, 116, 47, 108, 111, 99, 97, 116, 105, 111, 110, 115, 47, 117, 107, 115, 111, 117, 116, 104, 47, 104, 99, 112, 79, 112, 101, 114, 97, 116, 105, 111, 110, 83, 116, 97, 116, 117, 115, 101, 115, 47, 48, 98, 51, 53, 49, 53, 53, 102, 45, 49, 99, 56, 55, 45, 52, 97, 50, 55, 45, 98, 50, 54, 52, 45, 98, 53, 102, 102, 97, 102, 50, 100, 102, 53, 57, 50, 34, 44, 10, 32, 32, 32, 32, 34, 110, 97, 109, 101, 34, 58, 32, 34, 48, 98, 51, 53, 49, 53, 53, 102, 45, 49, 99, 56, 55, 45, 52, 97, 50, 55, 45, 98, 50, 54, 52, 45, 98, 53, 102, 102, 97, 102, 50, 100, 102, 53, 57, 50, 34, 44, 10, 32, 32, 32, 32, 34, 115, 116, 97, 116, 117, 115, 34, 58, 32, 34, 70, 97, 105, 108, 101, 100, 34, 44, 10, 32, 32, 32, 32, 34, 115, 116, 97, 114, 116, 84, 105, 109, 101, 34, 58, 32, 34, 50, 48, 50, 54, 45, 48, 51, 45, 48, 52, 84, 49, 49, 58, 50, 55, 58, 49, 48, 46, 49, 52, 49, 53, 49, 48, 50, 51, 56, 90, 34, 44, 10, 32, 32, 32, 32, 34, 101, 110, 100, 84, 105, 109, 101, 34, 58, 32, 34, 50, 48, 50, 54, 45, 48, 51, 45, 48, 52, 84, 49, 49, 58, 51, 51, 58, 48, 49, 46, 53, 48, 55, 56, 56, 48, 53, 56, 51, 90, 34, 44, 10, 32, 32, 32, 32, 34, 101, 114, 114, 111, 114, 34, 58, 32, 123, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 99, 111, 100, 101, 34, 58, 32, 34, 73, 110, 116, 101, 114, 110, 97, 108, 83, 101, 114, 118, 101, 114, 69, 114, 114, 111, 114, 34, 44, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 109, 101, 115, 115, 97, 103, 101, 34, 58, 32, 34, 80, 85, 84, 32, 104, 116, 116, 112, 115, 58, 47, 47, 109, 97, 110, 97, 103, 101, 109, 101, 110, 116, 46, 97, 122, 117, 114, 101, 46, 99, 111, 109, 47, 115, 117, 98, 115, 99, 114, 105, 11... Gomega truncated this representation as it exceeds 'format.MaxLength'. Consider having the object provide a custom 'GomegaStringer' representation or adjust the parameters in Gomega's 'format' package. Learn more here: https://onsi.github.io/gomega/#adjusting-output occurred
02:56:34	periodic-ci-Azure-ARO-HCP-main-periodic-prod-e2e-parallel post-goodmerged PR No additional run metadata captured	e2e	failed	n/a	21	e2e (21) Test Panicked 02:56:34 UTC · failed at e2e · test Customer should be able to test admin credentials before cluster ready, then full admin credential lifecycle · suite prod/parallel Show raw failure github.com/Azure/ARO-HCP/test/util/framework.Must[...](...) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000aca550?, {0x7f5094d33060?, 0xc000e80c90?}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).NewResourceGroup(0xc0008ab0e0, {0x7f5094d33060, 0xc000e80c90}, {0x784e4b5, 0x1f}, {0xc0000140b9, 0x7}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5 github.com/Azure/ARO-HCP/test/e2e.init.func3.1({0x7f5094d33060, 0xc000e80c90}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:66 +0x13f github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000e80c90?}) /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e github.com/onsi/ginkgo/v2/internal.(Suite).runNode.func3() /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b created by github.com/onsi/ginkgo/v2/internal.(Suite).runNode in goroutine 62 /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50 fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked github.com/Azure/ARO-HCP/test/util/framework.Must[...](...) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000aca550?, {0x7f5094d33060?, 0xc000e80c90?}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).NewResourceGroup(0xc0008ab0e0, {0x7f5094d33060, 0xc000e80c90}, {0x784e4b5, 0x1f}, {0xc0000140b9, 0x7}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5 github.com/Azure/ARO-HCP/test/e2e.init.func3.1({0x7f5094d33060, 0xc000e80c90}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:66 +0x13f github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000e80c90?}) /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e github.com/onsi/ginkgo/v2/internal.(Suite).runNode.func3() /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b created by github.com/onsi/ginkgo/v2/internal.(Suite).runNode in goroutine 62 /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50 fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked Interrupted by User 02:56:34 UTC · failed at e2e · test Customer should be able to create a cluster with default autoscaling and a nodepool with autoscaling enabled · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:36]: Interrupted by User fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:36]: Interrupted by User Test Panicked 02:56:34 UTC · failed at e2e · test Customer should not be able to reuse subnets and NSGs between clusters · suite prod/parallel Show raw failure github.com/Azure/ARO-HCP/test/util/framework.Must[...](...) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000e23cc0?, {0x7f88300fd488?, 0xc000e6bb60?}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).NewResourceGroup(0xc0006fbb00, {0x7f88300fd488, 0xc000e6bb60}, {0x783a7e5, 0x1b}, {0xc0000140b9, 0x7}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5 github.com/Azure/ARO-HCP/test/e2e.init.func11.1({0x7f88300fd488, 0xc000e6bb60}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_nsg_subnet_reuse.go:49 +0xe9 github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000e6bb60?}) /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e github.com/onsi/ginkgo/v2/internal.(Suite).runNode.func3() /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b created by github.com/onsi/ginkgo/v2/internal.(Suite).runNode in goroutine 83 /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50 fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked github.com/Azure/ARO-HCP/test/util/framework.Must[...](...) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000e23cc0?, {0x7f88300fd488?, 0xc000e6bb60?}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).NewResourceGroup(0xc0006fbb00, {0x7f88300fd488, 0xc000e6bb60}, {0x783a7e5, 0x1b}, {0xc0000140b9, 0x7}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5 github.com/Azure/ARO-HCP/test/e2e.init.func11.1({0x7f88300fd488, 0xc000e6bb60}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_nsg_subnet_reuse.go:49 +0xe9 github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000e6bb60?}) /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e github.com/onsi/ginkgo/v2/internal.(Suite).runNode.func3() /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b created by github.com/onsi/ginkgo/v2/internal.(Suite).runNode in goroutine 83 /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50 fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked Test Panicked 02:56:34 UTC · failed at e2e · test Customer should be able to create an HCP cluster and manage pull secrets · suite prod/parallel Show raw failure github.com/Azure/ARO-HCP/test/util/framework.Must[...](...) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000291f80?, {0x7f122ca56100?, 0xc000a47aa0?}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).NewResourceGroup(0xc000a8efc0, {0x7f122ca56100, 0xc000a47aa0}, {0x77ff021, 0xf}, {0xc0000140b9, 0x7}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5 github.com/Azure/ARO-HCP/test/e2e.init.func12.2({0x7f122ca56100, 0xc000a47aa0}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_pullsecret.go:74 +0x20f github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000a47aa0?}) /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e github.com/onsi/ginkgo/v2/internal.(Suite).runNode.func3() /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b created by github.com/onsi/ginkgo/v2/internal.(Suite).runNode in goroutine 74 /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50 fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked github.com/Azure/ARO-HCP/test/util/framework.Must[...](...) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000291f80?, {0x7f122ca56100?, 0xc000a47aa0?}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).NewResourceGroup(0xc000a8efc0, {0x7f122ca56100, 0xc000a47aa0}, {0x77ff021, 0xf}, {0xc0000140b9, 0x7}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5 github.com/Azure/ARO-HCP/test/e2e.init.func12.2({0x7f122ca56100, 0xc000a47aa0}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_pullsecret.go:74 +0x20f github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000a47aa0?}) /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e github.com/onsi/ginkgo/v2/internal.(Suite).runNode.func3() /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b created by github.com/onsi/ginkgo/v2/internal.(Suite).runNode in goroutine 74 /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50 fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked Test Panicked 02:56:34 UTC · failed at e2e · test Customer should be able to create several HCP clusters in their customer resource group, but not in the same managed resource group · suite prod/parallel Show raw failure github.com/Azure/ARO-HCP/test/util/framework.Must[...](...) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000a33e00?, {0x7f2b24450c60?, 0xc0009a2ea0?}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).NewResourceGroup(0xc0006a2900, {0x7f2b24450c60, 0xc0009a2ea0}, {0x77ee286, 0xb}, {0xc0000140b9, 0x7}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5 github.com/Azure/ARO-HCP/test/e2e.init.func16.2({0x7f2b24450c60, 0xc0009a2ea0}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:65 +0xe9 github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc0009a2ea0?}) /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e github.com/onsi/ginkgo/v2/internal.(Suite).runNode.func3() /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b created by github.com/onsi/ginkgo/v2/internal.(Suite).runNode in goroutine 93 /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50 fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked github.com/Azure/ARO-HCP/test/util/framework.Must[...](...) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000a33e00?, {0x7f2b24450c60?, 0xc0009a2ea0?}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).NewResourceGroup(0xc0006a2900, {0x7f2b24450c60, 0xc0009a2ea0}, {0x77ee286, 0xb}, {0xc0000140b9, 0x7}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5 github.com/Azure/ARO-HCP/test/e2e.init.func16.2({0x7f2b24450c60, 0xc0009a2ea0}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:65 +0xe9 github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc0009a2ea0?}) /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e github.com/onsi/ginkgo/v2/internal.(Suite).runNode.func3() /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b created by github.com/onsi/ginkgo/v2/internal.(Suite).runNode in goroutine 93 /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50 fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked Interrupted by User 02:56:34 UTC · failed at e2e · test Customer should be able to create an HCP cluster with back-level version 4.19 · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:48]: Interrupted by User fail [github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:48]: Interrupted by User Interrupted by User 02:56:34 UTC · failed at e2e · test Customer should be able to create an HCP cluster with custom autoscaling · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:36]: Interrupted by User fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:36]: Interrupted by User Interrupted by User 02:56:34 UTC · failed at e2e · test Customer should be able to create node pools with ARM64-based VMs · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/arm64_nodepool.go:35]: Interrupted by User fail [github.com/Azure/ARO-HCP/test/e2e/arm64_nodepool.go:35]: Interrupted by User Test Panicked 02:56:34 UTC · failed at e2e · test Customer should be able to lifecycle and confirm external auth on a cluster · suite prod/parallel Show raw failure github.com/Azure/ARO-HCP/test/util/framework.Must[...](...) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc0008e6680?, {0x7f29796b7660?, 0xc000e00840?}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).NewResourceGroup(0xc0008eefc0, {0x7f29796b7660, 0xc000e00840}, {0x77d9d2e, 0x7}, {0xc0000140b9, 0x7}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5 github.com/Azure/ARO-HCP/test/e2e.init.func19.2({0x7f29796b7660, 0xc000e00840}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/external_auth_list_and_verify.go:58 +0x155 github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000e00840?}) /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e github.com/onsi/ginkgo/v2/internal.(Suite).runNode.func3() /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b created by github.com/onsi/ginkgo/v2/internal.(Suite).runNode in goroutine 87 /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50 fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked github.com/Azure/ARO-HCP/test/util/framework.Must[...](...) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc0008e6680?, {0x7f29796b7660?, 0xc000e00840?}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).NewResourceGroup(0xc0008eefc0, {0x7f29796b7660, 0xc000e00840}, {0x77d9d2e, 0x7}, {0xc0000140b9, 0x7}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5 github.com/Azure/ARO-HCP/test/e2e.init.func19.2({0x7f29796b7660, 0xc000e00840}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/external_auth_list_and_verify.go:58 +0x155 github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000e00840?}) /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e github.com/onsi/ginkgo/v2/internal.(Suite).runNode.func3() /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b created by github.com/onsi/ginkgo/v2/internal.(Suite).runNode in goroutine 87 /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50 fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked Interrupted by User 02:56:34 UTC · failed at e2e · test Customer should be able to list HCP clusters without node pools at both subscription and resource group levels · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_list_no_infra.go:35]: Interrupted by User fail [github.com/Azure/ARO-HCP/test/e2e/cluster_list_no_infra.go:35]: Interrupted by User Interrupted by User 02:56:34 UTC · failed at e2e · test Customer should be able to create an HCP cluster with Image Registry not present · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/image_registry_cluster_create.go:37]: Interrupted by User fail [github.com/Azure/ARO-HCP/test/e2e/image_registry_cluster_create.go:37]: Interrupted by User Test Panicked 02:56:34 UTC · failed at e2e · test Customer should be able to create an HCP cluster and custom node pool osDisk size · suite prod/parallel Show raw failure github.com/Azure/ARO-HCP/test/util/framework.Must[...](...) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc0009801c0?, {0x7f138152c460?, 0xc000bf77a0?}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).NewResourceGroup(0xc000a20900, {0x7f138152c460, 0xc000bf77a0}, {0x77f1edf, 0xc}, {0xc0000140b9, 0x7}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5 github.com/Azure/ARO-HCP/test/e2e.init.func9.2({0x7f138152c460, 0xc000bf77a0}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_create_nodepool_osdisk.go:54 +0xe9 github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000bf77a0?}) /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e github.com/onsi/ginkgo/v2/internal.(Suite).runNode.func3() /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b created by github.com/onsi/ginkgo/v2/internal.(Suite).runNode in goroutine 99 /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50 fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked github.com/Azure/ARO-HCP/test/util/framework.Must[...](...) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc0009801c0?, {0x7f138152c460?, 0xc000bf77a0?}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).NewResourceGroup(0xc000a20900, {0x7f138152c460, 0xc000bf77a0}, {0x77f1edf, 0xc}, {0xc0000140b9, 0x7}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5 github.com/Azure/ARO-HCP/test/e2e.init.func9.2({0x7f138152c460, 0xc000bf77a0}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_create_nodepool_osdisk.go:54 +0xe9 github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000bf77a0?}) /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e github.com/onsi/ginkgo/v2/internal.(Suite).runNode.func3() /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b created by github.com/onsi/ginkgo/v2/internal.(Suite).runNode in goroutine 99 /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50 fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked Interrupted by User 02:56:34 UTC · failed at e2e · test Customer should not be able to create a 4.18 HCP cluster · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_missing_info.go:37]: Interrupted by User fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_missing_info.go:37]: Interrupted by User Deserializaion Error: no output from command 02:56:34 UTC · failed at e2e · test Customer should respect cluster-wide node limits with nodepool autoscaling · suite prod/parallel Show raw failure p=0xc000cf3900 sp=0xc000cf38e0 pc=0x2abd28e runtime.selectgo(0xc000cf49a0, 0xc000cf3cf4, 0x0?, 0x0, 0x0?, 0x1) /usr/local/go/src/runtime/select.go:351 +0x8b7 fp=0xc000cf3a40 sp=0xc000cf3900 pc=0x2a98137 github.com/onsi/ginkgo/v2/internal.(Suite).runNode(_, {0x42, 0x4, {0x78ef949, 0x41}, 0xc000d855e0, {{0x968cd56, 0x4b}, 0x7a, {0x0, ...}, ...}, ...}, ...) /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:923 +0xf65 fp=0xc000cf4d38 sp=0xc000cf3a40 pc=0x2f00005 github.com/onsi/ginkgo/v2/internal.(group).attemptSpec(0xc000cf6b98, 0x1, {{0xc000a9ea88?, 0xc000598bc0?, 0x1?}, 0x0?}) /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/group.go:199 +0xbd3 fp=0xc000cf5d58 sp=0xc000cf4d38 pc=0x2ee5453 github.com/onsi/ginkgo/v2/internal.(group).run(0xc000cf6b98, {0xc000159100?, 0x0?, 0x0?}) /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/group.go:349 +0x905 fp=0xc000cf6960 sp=0xc000cf5d58 pc=0x2ee79e5 github.com/onsi/ginkgo/v2/internal.(Suite).runSpecs(0xc00085d508, {0x0, 0x0}, {0xb319ac0, 0x0, 0x0}, {0xc00007a064, 0x2a}, 0x0, {0xc0001590e0, ...}) /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:496 +0xa9b fp=0xc000cf6e10 sp=0xc000cf6960 pc=0x2efb4db github.com/onsi/ginkgo/v2/internal.(Suite).RunSpec(_, {_, _}, {_, _, _}, {_, _}, {0xc00007a064, 0x2a}, ...) /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite_patch.go:68 +0x3c5 fp=0xc000cf6eb8 sp=0xc000cf6e10 pc=0x2f02845 github.com/openshift-eng/openshift-tests-extension/pkg/ginkgo.BuildExtensionTestSpecsFromOpenShiftGinkgoSuite.func1.1({0x2a4ca65?, 0xc000302300?}) /go/pkg/mod/github.com/openshift-eng/openshift-tests-extension@v0.0.0-20260127124016-0fed2b824818/pkg/ginkgo/util.go:85 +0x2ba fp=0xc000cf7b88 sp=0xc000cf6eb8 pc=0x521c71a github.com/openshift-eng/openshift-tests-extension/pkg/extension/extensiontests.runSpec({0x820bbb8, 0xc000b9f0e0}, 0xc000861680, 0x1) /go/pkg/mod/github.com/openshift-eng/openshift-tests-extension@v0.0.0-20260127124016-0fed2b824818/pkg/extension/extensiontests/spec.go:596 +0xee fp=0xc000cf7c30 sp=0xc000cf7b88 pc=0x5212a8e github.com/openshift-eng/openshift-tests-extension/pkg/extension/extensiontests.ExtensionTestSpecs.Run.func2() /go/pkg/mod/github.com/openshift-eng/openshift-tests-extension@v0.0.0-20260127124016-0fed2b824818/pkg/extension/extensiontests/spec.go:237 +0x196 fp=0xc000cf7fe0 sp=0xc000cf7c30 pc=0x520e816 runtime.goexit({}) /usr/local/go/src/runtime/asm_amd64.s:1693 +0x1 fp=0xc000cf7fe8 sp=0xc000cf7fe0 pc=0x2ac5921 created by github.com/openshift-eng/openshift-tests-extension/pkg/extension/extensiontests.ExtensionTestSpecs.Run in goroutine 1 /go/pkg/mod/github.com/openshift-eng/openshift-tests-extension@v0.0.0-20260127124016-0fed2b824818/pkg/extension/extensiontests/spec.go:230 +0x1e5 goroutine 86 gp=0xc000b32e00 m=nil [select, 2 minutes]: runtime.gopark(0xc000b3d868?, 0x2?, 0xc8?, 0x2d?, 0xc000b3d834?) /usr/local/go/src/runtime/proc.go:460 +0xce fp=0xc000b3d6b8 sp=0xc000b3d698 pc=0x2abd28e runtime.selectgo(0xc000b3d868, 0xc000b3d830, 0x816e580?, 0x0, 0x0?, 0x1) /usr/local/go/src/runtime/select.go:351 +0x8b7 fp=0xc000b3d7f8 sp=0xc000b3d6b8 pc=0x2a98137 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).AssignIdentityContainers(0xc0008ae120, {0x7f0b68c01c60, 0xc000811dd0}, 0x1, 0xdf8475800) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/identities_helper.go:271 +0x218 fp=0xc000b3d8e0 sp=0xc000b3d7f8 pc=0x4e997f8 github.com/Azure/ARO-HCP/test/e2e.init.func23.3({0x7f0b68c01c60, 0xc000811dd0}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:135 +0x6a fp=0xc000b3df58 sp=0xc000b3d8e0 pc=0x501aaea github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000811dd0?}) /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e fp=0xc000b3df88 sp=0xc000b3df58 pc=0x2eed25e github.com/onsi/ginkgo/v2/internal.(Suite).runNode.func3() /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b fp=0xc000b3dfe0 sp=0xc000b3df88 pc=0x2f0197b runtime.goexit({}) /usr/local/go/src/runtime/asm_amd64.s:1693 +0x1 fp=0xc000b3dfe8 sp=0xc000b3dfe0 pc=0x2ac5921 created by github.com/onsi/ginkgo/v2/internal.(Suite).runNode in goroutine 83 /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50 rax 0xca rbx 0x0 rcx 0x2ac7743 rdx 0x0 rdi 0xb2ef450 rsi 0x80 rbp 0x7fff3f6424d0 rsp 0x7fff3f642488 r8 0x0 r9 0x0 r10 0x0 r11 0x286 r12 0x2 r13 0x1 r14 0xb2ed360 r15 0x7 rip 0x2ac7741 rflags 0x286 cs 0x33 fs 0x0 gs 0x0 Command Error: exit status 2 Deserializaion Error: no output from command p=0xc000cf3900 sp=0xc000cf38e0 pc=0x2abd28e runtime.selectgo(0xc000cf49a0, 0xc000cf3cf4, 0x0?, 0x0, 0x0?, 0x1) /usr/local/go/src/runtime/select.go:351 +0x8b7 fp=0xc000cf3a40 sp=0xc000cf3900 pc=0x2a98137 github.com/onsi/ginkgo/v2/internal.(Suite).runNode(_, {0x42, 0x4, {0x78ef949, 0x41}, 0xc000d855e0, {{0x968cd56, 0x4b}, 0x7a, {0x0, ...}, ...}, ...}, ...) /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:923 +0xf65 fp=0xc000cf4d38 sp=0xc000cf3a40 pc=0x2f00005 github.com/onsi/ginkgo/v2/internal.(group).attemptSpec(0xc000cf6b98, 0x1, {{0xc000a9ea88?, 0xc000598bc0?, 0x1?}, 0x0?}) /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/group.go:199 +0xbd3 fp=0xc000cf5d58 sp=0xc000cf4d38 pc=0x2ee5453 github.com/onsi/ginkgo/v2/internal.(group).run(0xc000cf6b98, {0xc000159100?, 0x0?, 0x0?}) /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/group.go:349 +0x905 fp=0xc000cf6960 sp=0xc000cf5d58 pc=0x2ee79e5 github.com/onsi/ginkgo/v2/internal.(Suite).runSpecs(0xc00085d508, {0x0, 0x0}, {0xb319ac0, 0x0, 0x0}, {0xc00007a064, 0x2a}, 0x0, {0xc0001590e0, ...}) /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:496 +0xa9b fp=0xc000cf6e10 sp=0xc000cf6960 pc=0x2efb4db github.com/onsi/ginkgo/v2/internal.(Suite).RunSpec(_, {_, _}, {_, _, _}, {_, _}, {0xc00007a064, 0x2a}, ...) /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite_patch.go:68 +0x3c5 fp=0xc000cf6eb8 sp=0xc000cf6e10 pc=0x2f02845 github.com/openshift-eng/openshift-tests-extension/pkg/ginkgo.BuildExtensionTestSpecsFromOpenShiftGinkgoSuite.func1.1({0x2a4ca65?, 0xc000302300?}) /go/pkg/mod/github.com/openshift-eng/openshift-tests-extension@v0.0.0-20260127124016-0fed2b824818/pkg/ginkgo/util.go:85 +0x2ba fp=0xc000cf7b88 sp=0xc000cf6eb8 pc=0x521c71a github.com/openshift-eng/openshift-tests-extension/pkg/extension/extensiontests.runSpec({0x820bbb8, 0xc000b9f0e0}, 0xc000861680, 0x1) /go/pkg/mod/github.com/openshift-eng/openshift-tests-extension@v0.0.0-20260127124016-0fed2b824818/pkg/extension/extensiontests/spec.go:596 +0xee fp=0xc000cf7c30 sp=0xc000cf7b88 pc=0x5212a8e github.com/openshift-eng/openshift-tests-extension/pkg/extension/extensiontests.ExtensionTestSpecs.Run.func2() /go/pkg/mod/github.com/openshift-eng/openshift-tests-extension@v0.0.0-20260127124016-0fed2b824818/pkg/extension/extensiontests/spec.go:237 +0x196 fp=0xc000cf7fe0 sp=0xc000cf7c30 pc=0x520e816 runtime.goexit({}) /usr/local/go/src/runtime/asm_amd64.s:1693 +0x1 fp=0xc000cf7fe8 sp=0xc000cf7fe0 pc=0x2ac5921 created by github.com/openshift-eng/openshift-tests-extension/pkg/extension/extensiontests.ExtensionTestSpecs.Run in goroutine 1 /go/pkg/mod/github.com/openshift-eng/openshift-tests-extension@v0.0.0-20260127124016-0fed2b824818/pkg/extension/extensiontests/spec.go:230 +0x1e5 goroutine 86 gp=0xc000b32e00 m=nil [select, 2 minutes]: runtime.gopark(0xc000b3d868?, 0x2?, 0xc8?, 0x2d?, 0xc000b3d834?) /usr/local/go/src/runtime/proc.go:460 +0xce fp=0xc000b3d6b8 sp=0xc000b3d698 pc=0x2abd28e runtime.selectgo(0xc000b3d868, 0xc000b3d830, 0x816e580?, 0x0, 0x0?, 0x1) /usr/local/go/src/runtime/select.go:351 +0x8b7 fp=0xc000b3d7f8 sp=0xc000b3d6b8 pc=0x2a98137 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).AssignIdentityContainers(0xc0008ae120, {0x7f0b68c01c60, 0xc000811dd0}, 0x1, 0xdf8475800) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/identities_helper.go:271 +0x218 fp=0xc000b3d8e0 sp=0xc000b3d7f8 pc=0x4e997f8 github.com/Azure/ARO-HCP/test/e2e.init.func23.3({0x7f0b68c01c60, 0xc000811dd0}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:135 +0x6a fp=0xc000b3df58 sp=0xc000b3d8e0 pc=0x501aaea github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000811dd0?}) /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e fp=0xc000b3df88 sp=0xc000b3df58 pc=0x2eed25e github.com/onsi/ginkgo/v2/internal.(Suite).runNode.func3() /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b fp=0xc000b3dfe0 sp=0xc000b3df88 pc=0x2f0197b runtime.goexit({}) /usr/local/go/src/runtime/asm_amd64.s:1693 +0x1 fp=0xc000b3dfe8 sp=0xc000b3dfe0 pc=0x2ac5921 created by github.com/onsi/ginkgo/v2/internal.(Suite).runNode in goroutine 83 /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50 rax 0xca rbx 0x0 rcx 0x2ac7743 rdx 0x0 rdi 0xb2ef450 rsi 0x80 rbp 0x7fff3f6424d0 rsp 0x7fff3f642488 r8 0x0 r9 0x0 r10 0x0 r11 0x286 r12 0x2 r13 0x1 r14 0xb2ed360 r15 0x7 rip 0x2ac7741 rflags 0x286 cs 0x33 fs 0x0 gs 0x0 Command Error: exit status 2 Deserializaion Error: no output from command Test Panicked 02:56:34 UTC · failed at e2e · test Customer should be able to create an HCP cluster · suite prod/parallel Show raw failure github.com/Azure/ARO-HCP/test/util/framework.Must[...](...) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000a6a8c0?, {0x7fb7543ecda0?, 0xc0005e2150?}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).NewResourceGroup(0xc000a2eea0, {0x7fb7543ecda0, 0xc0005e2150}, {0x77f67a9, 0xd}, {0xc0000140b9, 0x7}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5 github.com/Azure/ARO-HCP/test/e2e.init.func17.2({0x7fb7543ecda0, 0xc0005e2150}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create.go:59 +0xe9 github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc0005e2150?}) /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e github.com/onsi/ginkgo/v2/internal.(Suite).runNode.func3() /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b created by github.com/onsi/ginkgo/v2/internal.(Suite).runNode in goroutine 85 /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50 fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked github.com/Azure/ARO-HCP/test/util/framework.Must[...](...) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000a6a8c0?, {0x7fb7543ecda0?, 0xc0005e2150?}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).NewResourceGroup(0xc000a2eea0, {0x7fb7543ecda0, 0xc0005e2150}, {0x77f67a9, 0xd}, {0xc0000140b9, 0x7}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5 github.com/Azure/ARO-HCP/test/e2e.init.func17.2({0x7fb7543ecda0, 0xc0005e2150}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create.go:59 +0xe9 github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc0005e2150?}) /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e github.com/onsi/ginkgo/v2/internal.(Suite).runNode.func3() /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b created by github.com/onsi/ginkgo/v2/internal.(Suite).runNode in goroutine 85 /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50 fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked Test Panicked 02:56:34 UTC · failed at e2e · test Update HCPOpenShiftCluster Positive creates a cluster and updates tags with a PATCH request · suite prod/parallel Show raw failure github.com/Azure/ARO-HCP/test/util/framework.Must[...](...) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000f22f40?, {0x7f38e8424e60?, 0xc0009e5920?}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).NewResourceGroup(0xc000a52ea0, {0x7f38e8424e60, 0xc0009e5920}, {0x77ea6bf, 0xa}, {0xc0000140b9, 0x7}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5 github.com/Azure/ARO-HCP/test/e2e.init.func14.2.1({0x7f38e8424e60, 0xc0009e5920}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:133 +0xe9 github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc0009e5920?}) /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e github.com/onsi/ginkgo/v2/internal.(Suite).runNode.func3() /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b created by github.com/onsi/ginkgo/v2/internal.(Suite).runNode in goroutine 105 /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50 fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked github.com/Azure/ARO-HCP/test/util/framework.Must[...](...) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000f22f40?, {0x7f38e8424e60?, 0xc0009e5920?}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).NewResourceGroup(0xc000a52ea0, {0x7f38e8424e60, 0xc0009e5920}, {0x77ea6bf, 0xa}, {0xc0000140b9, 0x7}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5 github.com/Azure/ARO-HCP/test/e2e.init.func14.2.1({0x7f38e8424e60, 0xc0009e5920}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:133 +0xe9 github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc0009e5920?}) /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e github.com/onsi/ginkgo/v2/internal.(Suite).runNode.func3() /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b created by github.com/onsi/ginkgo/v2/internal.(Suite).runNode in goroutine 105 /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50 fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked Test Panicked 02:56:34 UTC · failed at e2e · test Customer should be able to list available HCP OpenShift versions and validate response content · suite prod/parallel Show raw failure github.com/Azure/ARO-HCP/test/util/framework.Must[...](...) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).Get20240610ClientFactoryOrDie(0x78497cf?, {0x7f81f89bf020?, 0xc000ea21e0?}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:611 +0x37 github.com/Azure/ARO-HCP/test/e2e.init.func1.1({0x7f81f89bf020, 0xc000ea21e0}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_versions.go:38 +0x6f github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000ea21e0?}) /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e github.com/onsi/ginkgo/v2/internal.(Suite).runNode.func3() /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b created by github.com/onsi/ginkgo/v2/internal.(Suite).runNode in goroutine 88 /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50 fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked github.com/Azure/ARO-HCP/test/util/framework.Must[...](...) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).Get20240610ClientFactoryOrDie(0x78497cf?, {0x7f81f89bf020?, 0xc000ea21e0?}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:611 +0x37 github.com/Azure/ARO-HCP/test/e2e.init.func1.1({0x7f81f89bf020, 0xc000ea21e0}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_versions.go:38 +0x6f github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000ea21e0?}) /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e github.com/onsi/ginkgo/v2/internal.(Suite).runNode.func3() /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b created by github.com/onsi/ginkgo/v2/internal.(Suite).runNode in goroutine 88 /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50 fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked Test Panicked 02:56:34 UTC · failed at e2e · test Customer should create an HCP cluster and validate TLS certificates · suite prod/parallel Show raw failure github.com/Azure/ARO-HCP/test/util/framework.Must[...](...) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000d21480?, {0x7f4bdc53e060?, 0xc000aefc50?}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).NewResourceGroup(0xc000ed0120, {0x7f4bdc53e060, 0xc000aefc50}, {0x7818357, 0x14}, {0xc0000140b9, 0x7}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5 github.com/Azure/ARO-HCP/test/e2e.init.func13.2({0x7f4bdc53e060, 0xc000aefc50}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:57 +0xe9 github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000aefc50?}) /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e github.com/onsi/ginkgo/v2/internal.(Suite).runNode.func3() /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b created by github.com/onsi/ginkgo/v2/internal.(Suite).runNode in goroutine 82 /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50 fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked github.com/Azure/ARO-HCP/test/util/framework.Must[...](...) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000d21480?, {0x7f4bdc53e060?, 0xc000aefc50?}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).NewResourceGroup(0xc000ed0120, {0x7f4bdc53e060, 0xc000aefc50}, {0x7818357, 0x14}, {0xc0000140b9, 0x7}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5 github.com/Azure/ARO-HCP/test/e2e.init.func13.2({0x7f4bdc53e060, 0xc000aefc50}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:57 +0xe9 github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000aefc50?}) /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e github.com/onsi/ginkgo/v2/internal.(Suite).runNode.func3() /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b created by github.com/onsi/ginkgo/v2/internal.(Suite).runNode in goroutine 82 /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50 fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked Test Panicked 02:56:34 UTC · failed at e2e · test Customer should be able to create a cluster with an external auth config and get the external auth config · suite prod/parallel Show raw failure github.com/Azure/ARO-HCP/test/util/framework.Must[...](...) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000f32240?, {0x7f3400497460?, 0xc000e9e5d0?}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).NewResourceGroup(0xc000ec6120, {0x7f3400497460, 0xc000e9e5d0}, {0x781d01f, 0x15}, {0xc0000140b9, 0x7}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5 github.com/Azure/ARO-HCP/test/e2e.init.func18.2({0x7f3400497460, 0xc000e9e5d0}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:67 +0xe9 github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000e9e5d0?}) /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e github.com/onsi/ginkgo/v2/internal.(Suite).runNode.func3() /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b created by github.com/onsi/ginkgo/v2/internal.(Suite).runNode in goroutine 98 /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50 fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked github.com/Azure/ARO-HCP/test/util/framework.Must[...](...) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000f32240?, {0x7f3400497460?, 0xc000e9e5d0?}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).NewResourceGroup(0xc000ec6120, {0x7f3400497460, 0xc000e9e5d0}, {0x781d01f, 0x15}, {0xc0000140b9, 0x7}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5 github.com/Azure/ARO-HCP/test/e2e.init.func18.2({0x7f3400497460, 0xc000e9e5d0}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:67 +0xe9 github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000e9e5d0?}) /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e github.com/onsi/ginkgo/v2/internal.(Suite).runNode.func3() /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b created by github.com/onsi/ginkgo/v2/internal.(Suite).runNode in goroutine 98 /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50 fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked Test Panicked 02:56:34 UTC · failed at e2e · test Authorized CIDRs Connectivity should allow API access only from authorized VM IP · suite prod/parallel Show raw failure github.com/Azure/ARO-HCP/test/util/framework.Must[...](...) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000e90140?, {0x7ff6a549ca60?, 0xc000b0d170?}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).NewResourceGroup(0xc000dc4a20, {0x7ff6a549ca60, 0xc000b0d170}, {0x781cf8c, 0x15}, {0xc0000140b9, 0x7}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5 github.com/Azure/ARO-HCP/test/e2e.init.func5.1.1({0x7ff6a549ca60, 0xc000b0d170}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:68 +0xe9 github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000b0d170?}) /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e github.com/onsi/ginkgo/v2/internal.(Suite).runNode.func3() /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b created by github.com/onsi/ginkgo/v2/internal.(Suite).runNode in goroutine 85 /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50 fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked github.com/Azure/ARO-HCP/test/util/framework.Must[...](...) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000e90140?, {0x7ff6a549ca60?, 0xc000b0d170?}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).NewResourceGroup(0xc000dc4a20, {0x7ff6a549ca60, 0xc000b0d170}, {0x781cf8c, 0x15}, {0xc0000140b9, 0x7}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5 github.com/Azure/ARO-HCP/test/e2e.init.func5.1.1({0x7ff6a549ca60, 0xc000b0d170}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:68 +0xe9 github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000b0d170?}) /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e github.com/onsi/ginkgo/v2/internal.(Suite).runNode.func3() /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b created by github.com/onsi/ginkgo/v2/internal.(Suite).runNode in goroutine 85 /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50 fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked Test Panicked 02:56:34 UTC · failed at e2e · test Update HCPOpenShiftCluster Negative creates a cluster and fails to update its name with a PATCH request · suite prod/parallel Show raw failure github.com/Azure/ARO-HCP/test/util/framework.Must[...](...) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000ecf480?, {0x7f3e46332348?, 0xc000de9e00?}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).NewResourceGroup(0xc000e918c0, {0x7f3e46332348, 0xc000de9e00}, {0x77ea6b5, 0xa}, {0xc0000140b9, 0x7}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5 github.com/Azure/ARO-HCP/test/e2e.init.func14.1.1({0x7f3e46332348, 0xc000de9e00}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:57 +0xe9 github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000de9e00?}) /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e github.com/onsi/ginkgo/v2/internal.(Suite).runNode.func3() /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b created by github.com/onsi/ginkgo/v2/internal.(Suite).runNode in goroutine 111 /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50 fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked github.com/Azure/ARO-HCP/test/util/framework.Must[...](...) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000ecf480?, {0x7f3e46332348?, 0xc000de9e00?}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).NewResourceGroup(0xc000e918c0, {0x7f3e46332348, 0xc000de9e00}, {0x77ea6b5, 0xa}, {0xc0000140b9, 0x7}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5 github.com/Azure/ARO-HCP/test/e2e.init.func14.1.1({0x7f3e46332348, 0xc000de9e00}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:57 +0xe9 github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000de9e00?}) /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e github.com/onsi/ginkgo/v2/internal.(Suite).runNode.func3() /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b created by github.com/onsi/ginkgo/v2/internal.(Suite).runNode in goroutine 111 /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50 fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked
02:00:12	periodic-ci-Azure-ARO-HCP-main-periodic-prod-e2e-parallel post-goodmerged PR No additional run metadata captured	e2e	failed	n/a	21	e2e (21) Test Panicked 02:00:12 UTC · failed at e2e · test Customer should be able to create several HCP clusters in their customer resource group, but not in the same managed resource group · suite prod/parallel Show raw failure github.com/Azure/ARO-HCP/test/util/framework.Must[...](...) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000e26280?, {0x7fbe01426860?, 0xc000dbaf30?}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).NewResourceGroup(0xc000dee120, {0x7fbe01426860, 0xc000dbaf30}, {0x77ee286, 0xb}, {0xc0000140b9, 0x7}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5 github.com/Azure/ARO-HCP/test/e2e.init.func16.2({0x7fbe01426860, 0xc000dbaf30}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:65 +0xe9 github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000dbaf30?}) /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e github.com/onsi/ginkgo/v2/internal.(Suite).runNode.func3() /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b created by github.com/onsi/ginkgo/v2/internal.(Suite).runNode in goroutine 91 /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50 fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked github.com/Azure/ARO-HCP/test/util/framework.Must[...](...) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000e26280?, {0x7fbe01426860?, 0xc000dbaf30?}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).NewResourceGroup(0xc000dee120, {0x7fbe01426860, 0xc000dbaf30}, {0x77ee286, 0xb}, {0xc0000140b9, 0x7}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5 github.com/Azure/ARO-HCP/test/e2e.init.func16.2({0x7fbe01426860, 0xc000dbaf30}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:65 +0xe9 github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000dbaf30?}) /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e github.com/onsi/ginkgo/v2/internal.(Suite).runNode.func3() /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b created by github.com/onsi/ginkgo/v2/internal.(Suite).runNode in goroutine 91 /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50 fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked Test Panicked 02:00:12 UTC · failed at e2e · test Customer should not be able to create a 4.18 HCP cluster · suite prod/parallel Show raw failure github.com/Azure/ARO-HCP/test/util/framework.Must[...](...) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000e10900?, {0x7efc940c7260?, 0xc000abae70?}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).NewResourceGroup(0xc000a5c120, {0x7efc940c7260, 0xc000abae70}, {0x7812f30, 0x13}, {0xc0000140b9, 0x7}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5 github.com/Azure/ARO-HCP/test/e2e.init.func7.2({0x7efc940c7260, 0xc000abae70}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_create_missing_info.go:54 +0x105 github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000abae70?}) /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e github.com/onsi/ginkgo/v2/internal.(Suite).runNode.func3() /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b created by github.com/onsi/ginkgo/v2/internal.(Suite).runNode in goroutine 107 /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50 fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked github.com/Azure/ARO-HCP/test/util/framework.Must[...](...) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000e10900?, {0x7efc940c7260?, 0xc000abae70?}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).NewResourceGroup(0xc000a5c120, {0x7efc940c7260, 0xc000abae70}, {0x7812f30, 0x13}, {0xc0000140b9, 0x7}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5 github.com/Azure/ARO-HCP/test/e2e.init.func7.2({0x7efc940c7260, 0xc000abae70}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_create_missing_info.go:54 +0x105 github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000abae70?}) /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e github.com/onsi/ginkgo/v2/internal.(Suite).runNode.func3() /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b created by github.com/onsi/ginkgo/v2/internal.(Suite).runNode in goroutine 107 /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50 fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked Test Panicked 02:00:12 UTC · failed at e2e · test Customer should be able to lifecycle and confirm external auth on a cluster · suite prod/parallel Show raw failure github.com/Azure/ARO-HCP/test/util/framework.Must[...](...) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000e5ab80?, {0x7fa170358460?, 0xc000a04090?}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).NewResourceGroup(0xc000de57a0, {0x7fa170358460, 0xc000a04090}, {0x77d9d2e, 0x7}, {0xc0000140b9, 0x7}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5 github.com/Azure/ARO-HCP/test/e2e.init.func19.2({0x7fa170358460, 0xc000a04090}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/external_auth_list_and_verify.go:58 +0x155 github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000a04090?}) /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e github.com/onsi/ginkgo/v2/internal.(Suite).runNode.func3() /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b created by github.com/onsi/ginkgo/v2/internal.(Suite).runNode in goroutine 101 /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50 fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked github.com/Azure/ARO-HCP/test/util/framework.Must[...](...) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000e5ab80?, {0x7fa170358460?, 0xc000a04090?}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).NewResourceGroup(0xc000de57a0, {0x7fa170358460, 0xc000a04090}, {0x77d9d2e, 0x7}, {0xc0000140b9, 0x7}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5 github.com/Azure/ARO-HCP/test/e2e.init.func19.2({0x7fa170358460, 0xc000a04090}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/external_auth_list_and_verify.go:58 +0x155 github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000a04090?}) /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e github.com/onsi/ginkgo/v2/internal.(Suite).runNode.func3() /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b created by github.com/onsi/ginkgo/v2/internal.(Suite).runNode in goroutine 101 /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50 fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked Test Panicked 02:00:12 UTC · failed at e2e · test Customer should be able to test admin credentials before cluster ready, then full admin credential lifecycle · suite prod/parallel Show raw failure github.com/Azure/ARO-HCP/test/util/framework.Must[...](...) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000d038b0?, {0x7ff65c2bc1a0?, 0xc000da5440?}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).NewResourceGroup(0xc000dd3680, {0x7ff65c2bc1a0, 0xc000da5440}, {0x784e4b5, 0x1f}, {0xc0000140b9, 0x7}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5 github.com/Azure/ARO-HCP/test/e2e.init.func3.1({0x7ff65c2bc1a0, 0xc000da5440}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:66 +0x13f github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000da5440?}) /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e github.com/onsi/ginkgo/v2/internal.(Suite).runNode.func3() /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b created by github.com/onsi/ginkgo/v2/internal.(Suite).runNode in goroutine 109 /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50 fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked github.com/Azure/ARO-HCP/test/util/framework.Must[...](...) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000d038b0?, {0x7ff65c2bc1a0?, 0xc000da5440?}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).NewResourceGroup(0xc000dd3680, {0x7ff65c2bc1a0, 0xc000da5440}, {0x784e4b5, 0x1f}, {0xc0000140b9, 0x7}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5 github.com/Azure/ARO-HCP/test/e2e.init.func3.1({0x7ff65c2bc1a0, 0xc000da5440}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:66 +0x13f github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000da5440?}) /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e github.com/onsi/ginkgo/v2/internal.(Suite).runNode.func3() /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b created by github.com/onsi/ginkgo/v2/internal.(Suite).runNode in goroutine 109 /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50 fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked Test Panicked 02:00:12 UTC · failed at e2e · test Customer should not be able to reuse subnets and NSGs between clusters · suite prod/parallel Show raw failure github.com/Azure/ARO-HCP/test/util/framework.Must[...](...) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000e5e690?, {0x7f6e31d87120?, 0xc000c26210?}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).NewResourceGroup(0xc000c30a20, {0x7f6e31d87120, 0xc000c26210}, {0x783a7e5, 0x1b}, {0xc0000140b9, 0x7}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5 github.com/Azure/ARO-HCP/test/e2e.init.func11.1({0x7f6e31d87120, 0xc000c26210}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_nsg_subnet_reuse.go:49 +0xe9 github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000c26210?}) /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e github.com/onsi/ginkgo/v2/internal.(Suite).runNode.func3() /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b created by github.com/onsi/ginkgo/v2/internal.(Suite).runNode in goroutine 82 /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50 fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked github.com/Azure/ARO-HCP/test/util/framework.Must[...](...) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000e5e690?, {0x7f6e31d87120?, 0xc000c26210?}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).NewResourceGroup(0xc000c30a20, {0x7f6e31d87120, 0xc000c26210}, {0x783a7e5, 0x1b}, {0xc0000140b9, 0x7}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5 github.com/Azure/ARO-HCP/test/e2e.init.func11.1({0x7f6e31d87120, 0xc000c26210}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_nsg_subnet_reuse.go:49 +0xe9 github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000c26210?}) /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e github.com/onsi/ginkgo/v2/internal.(Suite).runNode.func3() /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b created by github.com/onsi/ginkgo/v2/internal.(Suite).runNode in goroutine 82 /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50 fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked Interrupted by User 02:00:12 UTC · failed at e2e · test Customer should be able to create an HCP cluster and manage pull secrets · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_pullsecret.go:46]: Interrupted by User fail [github.com/Azure/ARO-HCP/test/e2e/cluster_pullsecret.go:46]: Interrupted by User Test Panicked 02:00:12 UTC · failed at e2e · test Customer should be able to create a cluster with an external auth config and get the external auth config · suite prod/parallel Show raw failure github.com/Azure/ARO-HCP/test/util/framework.Must[...](...) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000e0be00?, {0x7f4d79470ee0?, 0xc000f620f0?}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).NewResourceGroup(0xc000f07680, {0x7f4d79470ee0, 0xc000f620f0}, {0x781d01f, 0x15}, {0xc0000140b9, 0x7}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5 github.com/Azure/ARO-HCP/test/e2e.init.func18.2({0x7f4d79470ee0, 0xc000f620f0}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:67 +0xe9 github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000f620f0?}) /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e github.com/onsi/ginkgo/v2/internal.(Suite).runNode.func3() /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b created by github.com/onsi/ginkgo/v2/internal.(Suite).runNode in goroutine 103 /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50 fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked github.com/Azure/ARO-HCP/test/util/framework.Must[...](...) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000e0be00?, {0x7f4d79470ee0?, 0xc000f620f0?}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).NewResourceGroup(0xc000f07680, {0x7f4d79470ee0, 0xc000f620f0}, {0x781d01f, 0x15}, {0xc0000140b9, 0x7}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5 github.com/Azure/ARO-HCP/test/e2e.init.func18.2({0x7f4d79470ee0, 0xc000f620f0}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:67 +0xe9 github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000f620f0?}) /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e github.com/onsi/ginkgo/v2/internal.(Suite).runNode.func3() /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b created by github.com/onsi/ginkgo/v2/internal.(Suite).runNode in goroutine 103 /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50 fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked Interrupted by User 02:00:12 UTC · failed at e2e · test Update HCPOpenShiftCluster Positive creates a cluster and updates tags with a PATCH request · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:120]: Interrupted by User fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:120]: Interrupted by User Test Panicked 02:00:12 UTC · failed at e2e · test Customer should be able to list HCP clusters without node pools at both subscription and resource group levels · suite prod/parallel Show raw failure github.com/Azure/ARO-HCP/test/util/framework.Must[...](...) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc00097fa00?, {0x7f5fef765660?, 0xc000f0ca80?}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).NewResourceGroup(0xc000f5ea20, {0x7f5fef765660, 0xc000f0ca80}, {0x77ff012, 0xf}, {0xc0000140b9, 0x7}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5 github.com/Azure/ARO-HCP/test/e2e.init.func10.1({0x7f5fef765660, 0xc000f0ca80}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_list_no_infra.go:55 +0x23e github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000f0ca80?}) /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e github.com/onsi/ginkgo/v2/internal.(Suite).runNode.func3() /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b created by github.com/onsi/ginkgo/v2/internal.(Suite).runNode in goroutine 102 /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50 fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked github.com/Azure/ARO-HCP/test/util/framework.Must[...](...) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc00097fa00?, {0x7f5fef765660?, 0xc000f0ca80?}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).NewResourceGroup(0xc000f5ea20, {0x7f5fef765660, 0xc000f0ca80}, {0x77ff012, 0xf}, {0xc0000140b9, 0x7}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5 github.com/Azure/ARO-HCP/test/e2e.init.func10.1({0x7f5fef765660, 0xc000f0ca80}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_list_no_infra.go:55 +0x23e github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000f0ca80?}) /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e github.com/onsi/ginkgo/v2/internal.(Suite).runNode.func3() /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b created by github.com/onsi/ginkgo/v2/internal.(Suite).runNode in goroutine 102 /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50 fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked Test Panicked 02:00:12 UTC · failed at e2e · test Customer should be able to create an HCP cluster with back-level version 4.19 · suite prod/parallel Show raw failure github.com/Azure/ARO-HCP/test/util/framework.Must[...](...) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc00103a050?, {0x7fb94c3f7c60?, 0xc000f0e720?}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).NewResourceGroup(0xc000f306c0, {0x7fb94c3f7c60, 0xc000f0e720}, {0x782740d, 0x17}, {0xc0000140b9, 0x7}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5 github.com/Azure/ARO-HCP/test/e2e.init.func15.1({0x7fb94c3f7c60, 0xc000f0e720}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:69 +0xf1 github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000f0e720?}) /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e github.com/onsi/ginkgo/v2/internal.(Suite).runNode.func3() /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b created by github.com/onsi/ginkgo/v2/internal.(Suite).runNode in goroutine 106 /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50 fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked github.com/Azure/ARO-HCP/test/util/framework.Must[...](...) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc00103a050?, {0x7fb94c3f7c60?, 0xc000f0e720?}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).NewResourceGroup(0xc000f306c0, {0x7fb94c3f7c60, 0xc000f0e720}, {0x782740d, 0x17}, {0xc0000140b9, 0x7}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5 github.com/Azure/ARO-HCP/test/e2e.init.func15.1({0x7fb94c3f7c60, 0xc000f0e720}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:69 +0xf1 github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000f0e720?}) /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e github.com/onsi/ginkgo/v2/internal.(Suite).runNode.func3() /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b created by github.com/onsi/ginkgo/v2/internal.(Suite).runNode in goroutine 106 /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50 fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked Test Panicked 02:00:12 UTC · failed at e2e · test Customer should create an HCP cluster and validate TLS certificates · suite prod/parallel Show raw failure github.com/Azure/ARO-HCP/test/util/framework.Must[...](...) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000d91780?, {0x7f5edd39dea0?, 0xc000ed19e0?}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).NewResourceGroup(0xc000c82480, {0x7f5edd39dea0, 0xc000ed19e0}, {0x7818357, 0x14}, {0xc0000140b9, 0x7}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5 github.com/Azure/ARO-HCP/test/e2e.init.func13.2({0x7f5edd39dea0, 0xc000ed19e0}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:57 +0xe9 github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000ed19e0?}) /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e github.com/onsi/ginkgo/v2/internal.(Suite).runNode.func3() /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b created by github.com/onsi/ginkgo/v2/internal.(Suite).runNode in goroutine 96 /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50 fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked github.com/Azure/ARO-HCP/test/util/framework.Must[...](...) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000d91780?, {0x7f5edd39dea0?, 0xc000ed19e0?}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).NewResourceGroup(0xc000c82480, {0x7f5edd39dea0, 0xc000ed19e0}, {0x7818357, 0x14}, {0xc0000140b9, 0x7}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5 github.com/Azure/ARO-HCP/test/e2e.init.func13.2({0x7f5edd39dea0, 0xc000ed19e0}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:57 +0xe9 github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000ed19e0?}) /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e github.com/onsi/ginkgo/v2/internal.(Suite).runNode.func3() /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b created by github.com/onsi/ginkgo/v2/internal.(Suite).runNode in goroutine 96 /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50 fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked Interrupted by User 02:00:12 UTC · failed at e2e · test Customer should be able to create an HCP cluster and custom node pool osDisk size · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_nodepool_osdisk.go:35]: Interrupted by User fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_nodepool_osdisk.go:35]: Interrupted by User Test Panicked 02:00:12 UTC · failed at e2e · test Customer should be able to create an HCP cluster · suite prod/parallel Show raw failure github.com/Azure/ARO-HCP/test/util/framework.Must[...](...) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000d17100?, {0x7f0a10091060?, 0xc000be5500?}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).NewResourceGroup(0xc0009c18c0, {0x7f0a10091060, 0xc000be5500}, {0x77f67a9, 0xd}, {0xc0000140b9, 0x7}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5 github.com/Azure/ARO-HCP/test/e2e.init.func17.2({0x7f0a10091060, 0xc000be5500}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create.go:59 +0xe9 github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000be5500?}) /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e github.com/onsi/ginkgo/v2/internal.(Suite).runNode.func3() /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b created by github.com/onsi/ginkgo/v2/internal.(Suite).runNode in goroutine 104 /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50 fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked github.com/Azure/ARO-HCP/test/util/framework.Must[...](...) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000d17100?, {0x7f0a10091060?, 0xc000be5500?}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).NewResourceGroup(0xc0009c18c0, {0x7f0a10091060, 0xc000be5500}, {0x77f67a9, 0xd}, {0xc0000140b9, 0x7}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5 github.com/Azure/ARO-HCP/test/e2e.init.func17.2({0x7f0a10091060, 0xc000be5500}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create.go:59 +0xe9 github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000be5500?}) /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e github.com/onsi/ginkgo/v2/internal.(Suite).runNode.func3() /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b created by github.com/onsi/ginkgo/v2/internal.(Suite).runNode in goroutine 104 /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50 fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked Interrupted by User 02:00:12 UTC · failed at e2e · test Customer should be able to create an HCP cluster with Image Registry not present · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/image_registry_cluster_create.go:37]: Interrupted by User fail [github.com/Azure/ARO-HCP/test/e2e/image_registry_cluster_create.go:37]: Interrupted by User Interrupted by User 02:00:12 UTC · failed at e2e · test Customer should be able to create a cluster with default autoscaling and a nodepool with autoscaling enabled · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:36]: Interrupted by User fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:36]: Interrupted by User Test Panicked 02:00:12 UTC · failed at e2e · test Customer should be able to create node pools with ARM64-based VMs · suite prod/parallel Show raw failure github.com/Azure/ARO-HCP/test/util/framework.Must[...](...) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000df4300?, {0x7fa2da729c60?, 0xc000f1c150?}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).NewResourceGroup(0xc000f26240, {0x7fa2da729c60, 0xc000f1c150}, {0x7803b39, 0x10}, {0xc0000140b9, 0x7}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5 github.com/Azure/ARO-HCP/test/e2e.init.func4.2({0x7fa2da729c60, 0xc000f1c150}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/arm64_nodepool.go:59 +0x105 github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000f1c150?}) /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e github.com/onsi/ginkgo/v2/internal.(Suite).runNode.func3() /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b created by github.com/onsi/ginkgo/v2/internal.(Suite).runNode in goroutine 94 /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50 fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked github.com/Azure/ARO-HCP/test/util/framework.Must[...](...) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000df4300?, {0x7fa2da729c60?, 0xc000f1c150?}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).NewResourceGroup(0xc000f26240, {0x7fa2da729c60, 0xc000f1c150}, {0x7803b39, 0x10}, {0xc0000140b9, 0x7}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5 github.com/Azure/ARO-HCP/test/e2e.init.func4.2({0x7fa2da729c60, 0xc000f1c150}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/arm64_nodepool.go:59 +0x105 github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000f1c150?}) /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e github.com/onsi/ginkgo/v2/internal.(Suite).runNode.func3() /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b created by github.com/onsi/ginkgo/v2/internal.(Suite).runNode in goroutine 94 /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50 fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked Test Panicked 02:00:12 UTC · failed at e2e · test Customer should be able to list available HCP OpenShift versions and validate response content · suite prod/parallel Show raw failure github.com/Azure/ARO-HCP/test/util/framework.Must[...](...) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).Get20240610ClientFactoryOrDie(0x78497cf?, {0x7f3fd40c6060?, 0xc000ebf860?}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:611 +0x37 github.com/Azure/ARO-HCP/test/e2e.init.func1.1({0x7f3fd40c6060, 0xc000ebf860}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_versions.go:38 +0x6f github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000ebf860?}) /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e github.com/onsi/ginkgo/v2/internal.(Suite).runNode.func3() /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b created by github.com/onsi/ginkgo/v2/internal.(Suite).runNode in goroutine 90 /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50 fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked github.com/Azure/ARO-HCP/test/util/framework.Must[...](...) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402 github.com/Azure/ARO-HCP/test/util/framework.(perItOrDescribeTestContext).Get20240610ClientFactoryOrDie(0x78497cf?, {0x7f3fd40c6060?, 0xc000ebf860?}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:611 +0x37 github.com/Azure/ARO-HCP/test/e2e.init.func1.1({0x7f3fd40c6060, 0xc000ebf860}) /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_versions.go:38 +0x6f github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000ebf860?}) /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e github.com/onsi/ginkgo/v2/internal.(Suite).runNode.func3() /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b created by github.com/onsi/ginkgo/v2/internal.(Suite).runNode in goroutine 90 /go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50 fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked Interrupted by User 02:00:12 UTC · failed at e2e · test Authorized CIDRs Connectivity should allow API access only from authorized VM IP · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:45]: Interrupted by User fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:45]: Interrupted by User Interrupted by User 02:00:12 UTC · failed at e2e · test Customer should respect cluster-wide node limits with nodepool autoscaling · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:122]: Interrupted by User fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:122]: Interrupted by User Interrupted by User 02:00:12 UTC · failed at e2e · test Update HCPOpenShiftCluster Negative creates a cluster and fails to update its name with a PATCH request · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:44]: Interrupted by User fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:44]: Interrupted by User Interrupted by User 02:00:12 UTC · failed at e2e · test Customer should be able to create an HCP cluster with custom autoscaling · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:36]: Interrupted by User fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:36]: Interrupted by User

Time (UTC)

Job

Failed at

Result

Failed tests

Details

11:18:47

periodic-ci-Azure-ARO-HCP-main-periodic-prod-e2e-parallel

post-goodmerged PR

No additional run metadata captured

e2e

failed

n/a

e2e (17)

CreateNodePoolFromParam timed out after <minutes> minutes; context deadline exceeded

11:18:47 UTC · failed at e2e · test Customer should be able to create an HCP cluster · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create.go:118]: Unexpected error:
    <*fmt.wrapErrors | 0xc001195ce0>: 
    failed to create NodePool np-1, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group basic-cluster-5jpdsk, error: failed waiting for nodepool="np-1" for cluster "basic-hcp-cluster" in resourcegroup="basic-cluster-5jpdsk" to finish creating: context deadline exceeded
    {
        msg: "failed to create NodePool np-1, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group basic-cluster-5jpdsk, error: failed waiting for nodepool=\"np-1\" for cluster \"basic-hcp-cluster\" in resourcegroup=\"basic-cluster-5jpdsk\" to finish creating: context deadline exceeded",
        errs: [
            <*errors.errorString | 0xc000112ea0>{
                s: "timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group basic-cluster-5jpdsk",
            },
            <*fmt.wrapError | 0xc00075b4e0>{
                msg: "failed waiting for nodepool=\"np-1\" for cluster \"basic-hcp-cluster\" in resourcegroup=\"basic-cluster-5jpdsk\" to finish creating: context deadline exceeded",
                err: <context.deadlineExceededError>{},
            },
        ],
    }
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create.go:118]: Unexpected error:
    <*fmt.wrapErrors | 0xc001195ce0>: 
    failed to create NodePool np-1, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group basic-cluster-5jpdsk, error: failed waiting for nodepool="np-1" for cluster "basic-hcp-cluster" in resourcegroup="basic-cluster-5jpdsk" to finish creating: context deadline exceeded
    {
        msg: "failed to create NodePool np-1, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group basic-cluster-5jpdsk, error: failed waiting for nodepool=\"np-1\" for cluster \"basic-hcp-cluster\" in resourcegroup=\"basic-cluster-5jpdsk\" to finish creating: context deadline exceeded",
        errs: [
            <*errors.errorString | 0xc000112ea0>{
                s: "timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group basic-cluster-5jpdsk",
            },
            <*fmt.wrapError | 0xc00075b4e0>{
                msg: "failed waiting for nodepool=\"np-1\" for cluster \"basic-hcp-cluster\" in resourcegroup=\"basic-cluster-5jpdsk\" to finish creating: context deadline exceeded",
                err: <context.deadlineExceededError>{},
            },
        ],
    }
occurred

ERROR CODE: InternalServerError; detail message insufficient public IP address quota: required <count>, available <count>; provider Microsoft.RedHatOpenShift

11:18:47 UTC · failed at e2e · test Customer should be able to list HCP clusters without node pools at both subscription and resource group levels · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_list_no_infra.go:86]: Unexpected error:
    <*fmt.wrapError | 0xc000f06a60>: 
    failed to create HCP cluster list-test-cluster-p6lb7l: failed waiting for cluster="list-test-cluster-p6lb7l" in resourcegroup="cluster-listing-dlgx6v" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4019f702-1223-4f14-b6db-2e256f1f2a87
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4019f702-1223-4f14-b6db-2e256f1f2a87",
      "name": "4019f702-1223-4f14-b6db-2e256f1f2a87",
      "status": "Failed",
      "startTime": "2026-03-04T11:40:19.135177985Z",
      "endTime": "2026-03-04T11:45:00.015034663Z",
      "error": {
        "code": "InternalServerError",
        "message": "insufficient public IP address quota: required 2, available 0"
      }
    }
    --------------------------------------------------------------------------------
    
    {
        msg: "failed to create HCP cluster list-test-cluster-p6lb7l: failed waiting for cluster=\"list-test-cluster-p6lb7l\" in resourcegroup=\"cluster-listing-dlgx6v\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4019f702-1223-4f14-b6db-2e256f1f2a87\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n  \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4019f702-1223-4f14-b6db-2e256f1f2a87\",\n  \"name\": \"4019f702-1223-4f14-b6db-2e256f1f2a87\",\n  \"status\": \"Failed\",\n  \"startTime\": \"2026-03-04T11:40:19.135177985Z\",\n  \"endTime\": \"2026-03-04T11:45:00.015034663Z\",\n  \"error\": {\n    \"code\": \"InternalServerError\",\n    \"message\": \"insufficient public IP address quota: required 2, available 0\"\n  }\n}\n--------------------------------------------------------------------------------\n",
        err: <*fmt.wrapError | 0xc000f06a40>{
            msg: "failed waiting for cluster=\"list-test-cluster-p6lb7l\" in resourcegroup=\"cluster-listing-dlgx6v\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4019f702-1223-4f14-b6db-2e256f1f2a87\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n  \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4019f702-1223-4f14-b6db-2e256f1f2a87\",\n  \"name\": \"4019f702-1223-4f14-b6db-2e256f1f2a87\",\n  \"status\": \"Failed\",\n  \"startTime\": \"2026-03-04T11:40:19.135177985Z\",\n  \"endTime\": \"2026-03-04T11:45:00.015034663Z\",\n  \"error\": {\n    \"code\": \"InternalServerError\",\n    \"message\": \"insufficient public IP address quota: required 2, available 0\"\n  }\n}\n--------------------------------------------------------------------------------\n",
            err: <*exported.ResponseError | 0xc00122cd20>{
                ErrorCode: "InternalServerError",
                StatusCode: 200,
                RawResponse: {
                    Status: "200 OK",
                    StatusCode: 200,
                    Proto: "HTTP/2.0",
                    ProtoMajor: 2,
                    ProtoMinor: 0,
                    Header: {
                        "X-Envoy-Upstream-Service-Time": ["19"],
                        "X-Ms-Operation-Identifier": [
                            "appId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,tenantId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,objectId=0e96dbb7-17a2-4393-b2d3-d84524e39c32/canadacentral/8ffb5ba3-48ab-446c-a59c-3f9d09ee97a7",
                        ],
                        "X-Ms-Routing-Request-Id": [
                            "CANADACENTRAL:20260304T114501Z:4d14fc79-c15b-43f1-b5fc-ec2ab314d07f",
                        ],
                        "X-Content-Type-Options": ["nosniff"],
                        "X-Cache": ["CONFIG_NOCACHE"],
                        "Cache-Control": ["no-cache"],
                        "X-Ms-Request-Id": [
                            "a63da353-17cb-482c-9907-e48691e47dad",
                        ],
                        "Pragma": ["no-cache"],
                        "Content-Length": ["499"],
                        "Content-Type": ["application/json"],
                        "Expires": ["-1"],
                        "X-Ms-Ratelimit-Remaining-Subscription-Reads": ["249"],
                        "X-Msedge-Ref": [
                ...

Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.

Learn more here: https://onsi.github.io/gomega/#adjusting-output

occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_list_no_infra.go:86]: Unexpected error:
    <*fmt.wrapError | 0xc000f06a60>: 
    failed to create HCP cluster list-test-cluster-p6lb7l: failed waiting for cluster="list-test-cluster-p6lb7l" in resourcegroup="cluster-listing-dlgx6v" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4019f702-1223-4f14-b6db-2e256f1f2a87
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4019f702-1223-4f14-b6db-2e256f1f2a87",
      "name": "4019f702-1223-4f14-b6db-2e256f1f2a87",
      "status": "Failed",
      "startTime": "2026-03-04T11:40:19.135177985Z",
      "endTime": "2026-03-04T11:45:00.015034663Z",
      "error": {
        "code": "InternalServerError",
        "message": "insufficient public IP address quota: required 2, available 0"
      }
    }
    --------------------------------------------------------------------------------
    
    {
        msg: "failed to create HCP cluster list-test-cluster-p6lb7l: failed waiting for cluster=\"list-test-cluster-p6lb7l\" in resourcegroup=\"cluster-listing-dlgx6v\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4019f702-1223-4f14-b6db-2e256f1f2a87\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n  \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4019f702-1223-4f14-b6db-2e256f1f2a87\",\n  \"name\": \"4019f702-1223-4f14-b6db-2e256f1f2a87\",\n  \"status\": \"Failed\",\n  \"startTime\": \"2026-03-04T11:40:19.135177985Z\",\n  \"endTime\": \"2026-03-04T11:45:00.015034663Z\",\n  \"error\": {\n    \"code\": \"InternalServerError\",\n    \"message\": \"insufficient public IP address quota: required 2, available 0\"\n  }\n}\n--------------------------------------------------------------------------------\n",
        err: <*fmt.wrapError | 0xc000f06a40>{
            msg: "failed waiting for cluster=\"list-test-cluster-p6lb7l\" in resourcegroup=\"cluster-listing-dlgx6v\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4019f702-1223-4f14-b6db-2e256f1f2a87\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n  \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4019f702-1223-4f14-b6db-2e256f1f2a87\",\n  \"name\": \"4019f702-1223-4f14-b6db-2e256f1f2a87\",\n  \"status\": \"Failed\",\n  \"startTime\": \"2026-03-04T11:40:19.135177985Z\",\n  \"endTime\": \"2026-03-04T11:45:00.015034663Z\",\n  \"error\": {\n    \"code\": \"InternalServerError\",\n    \"message\": \"insufficient public IP address quota: required 2, available 0\"\n  }\n}\n--------------------------------------------------------------------------------\n",
            err: <*exported.ResponseError | 0xc00122cd20>{
                ErrorCode: "InternalServerError",
                StatusCode: 200,
                RawResponse: {
                    Status: "200 OK",
                    StatusCode: 200,
                    Proto: "HTTP/2.0",
                    ProtoMajor: 2,
                    ProtoMinor: 0,
                    Header: {
                        "X-Envoy-Upstream-Service-Time": ["19"],
                        "X-Ms-Operation-Identifier": [
                            "appId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,tenantId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,objectId=0e96dbb7-17a2-4393-b2d3-d84524e39c32/canadacentral/8ffb5ba3-48ab-446c-a59c-3f9d09ee97a7",
                        ],
                        "X-Ms-Routing-Request-Id": [
                            "CANADACENTRAL:20260304T114501Z:4d14fc79-c15b-43f1-b5fc-ec2ab314d07f",
                        ],
                        "X-Content-Type-Options": ["nosniff"],
                        "X-Cache": ["CONFIG_NOCACHE"],
                        "Cache-Control": ["no-cache"],
                        "X-Ms-Request-Id": [
                            "a63da353-17cb-482c-9907-e48691e47dad",
                        ],
                        "Pragma": ["no-cache"],
                        "Content-Length": ["499"],
                        "Content-Type": ["application/json"],
                        "Expires": ["-1"],
                        "X-Ms-Ratelimit-Remaining-Subscription-Reads": ["249"],
                        "X-Msedge-Ref": [
                ...

Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.

Learn more here: https://onsi.github.io/gomega/#adjusting-output

occurred

CreateNodePoolFromParam timed out after <minutes> minutes; context deadline exceeded

11:18:47 UTC · failed at e2e · test Customer should be able to create an HCP cluster and manage pull secrets · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_pullsecret.go:112]: Unexpected error:
    <*fmt.wrapErrors | 0xc0009e6630>: 
    failed to create NodePool np-1, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group pullsecret-test-f6kzk8, error: failed waiting for nodepool="np-1" for cluster "pullsecret-hcp-cluster" in resourcegroup="pullsecret-test-f6kzk8" to finish creating: context deadline exceeded
    {
        msg: "failed to create NodePool np-1, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group pullsecret-test-f6kzk8, error: failed waiting for nodepool=\"np-1\" for cluster \"pullsecret-hcp-cluster\" in resourcegroup=\"pullsecret-test-f6kzk8\" to finish creating: context deadline exceeded",
        errs: [
            <*errors.errorString | 0xc000a36550>{
                s: "timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group pullsecret-test-f6kzk8",
            },
            <*fmt.wrapError | 0xc0009dc4e0>{
                msg: "failed waiting for nodepool=\"np-1\" for cluster \"pullsecret-hcp-cluster\" in resourcegroup=\"pullsecret-test-f6kzk8\" to finish creating: context deadline exceeded",
                err: <context.deadlineExceededError>{},
            },
        ],
    }
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_pullsecret.go:112]: Unexpected error:
    <*fmt.wrapErrors | 0xc0009e6630>: 
    failed to create NodePool np-1, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group pullsecret-test-f6kzk8, error: failed waiting for nodepool="np-1" for cluster "pullsecret-hcp-cluster" in resourcegroup="pullsecret-test-f6kzk8" to finish creating: context deadline exceeded
    {
        msg: "failed to create NodePool np-1, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group pullsecret-test-f6kzk8, error: failed waiting for nodepool=\"np-1\" for cluster \"pullsecret-hcp-cluster\" in resourcegroup=\"pullsecret-test-f6kzk8\" to finish creating: context deadline exceeded",
        errs: [
            <*errors.errorString | 0xc000a36550>{
                s: "timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group pullsecret-test-f6kzk8",
            },
            <*fmt.wrapError | 0xc0009dc4e0>{
                msg: "failed waiting for nodepool=\"np-1\" for cluster \"pullsecret-hcp-cluster\" in resourcegroup=\"pullsecret-test-f6kzk8\" to finish creating: context deadline exceeded",
                err: <context.deadlineExceededError>{},
            },
        ],
    }
occurred

CreateOrUpdateExternalAuthAndWait timed out after <minutes> minutes; context deadline exceeded

11:18:47 UTC · failed at e2e · test Customer should be able to lifecycle and confirm external auth on a cluster · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_list_and_verify.go:118]: Unexpected error:
    <*fmt.wrapErrors | 0xc001057380>: 
    failed waiting for external auth "ea-list" in resourcegroup="ea-list-8crjbf" for cluster="ea-listr26d5r" to finish, caused by: timeout '15.000000' minutes exceeded during CreateOrUpdateExternalAuthAndWait for external auth ea-list in cluster ea-listr26d5r in resource group ea-list-8crjbf, error: context deadline exceeded
    {
        msg: "failed waiting for external auth \"ea-list\" in resourcegroup=\"ea-list-8crjbf\" for cluster=\"ea-listr26d5r\" to finish, caused by: timeout '15.000000' minutes exceeded during CreateOrUpdateExternalAuthAndWait for external auth ea-list in cluster ea-listr26d5r in resource group ea-list-8crjbf, error: context deadline exceeded",
        errs: [
            <*errors.errorString | 0xc000717760>{
                s: "timeout '15.000000' minutes exceeded during CreateOrUpdateExternalAuthAndWait for external auth ea-list in cluster ea-listr26d5r in resource group ea-list-8crjbf",
            },
            <context.deadlineExceededError>{},
        ],
    }
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_list_and_verify.go:118]: Unexpected error:
    <*fmt.wrapErrors | 0xc001057380>: 
    failed waiting for external auth "ea-list" in resourcegroup="ea-list-8crjbf" for cluster="ea-listr26d5r" to finish, caused by: timeout '15.000000' minutes exceeded during CreateOrUpdateExternalAuthAndWait for external auth ea-list in cluster ea-listr26d5r in resource group ea-list-8crjbf, error: context deadline exceeded
    {
        msg: "failed waiting for external auth \"ea-list\" in resourcegroup=\"ea-list-8crjbf\" for cluster=\"ea-listr26d5r\" to finish, caused by: timeout '15.000000' minutes exceeded during CreateOrUpdateExternalAuthAndWait for external auth ea-list in cluster ea-listr26d5r in resource group ea-list-8crjbf, error: context deadline exceeded",
        errs: [
            <*errors.errorString | 0xc000717760>{
                s: "timeout '15.000000' minutes exceeded during CreateOrUpdateExternalAuthAndWait for external auth ea-list in cluster ea-listr26d5r in resource group ea-list-8crjbf",
            },
            <context.deadlineExceededError>{},
        ],
    }
occurred

ERROR CODE: Conflict; detail message Cannot request credential while resource is "provisioning"; provider Microsoft.RedHatOpenShift

11:18:47 UTC · failed at e2e · test Update HCPOpenShiftCluster Positive creates a cluster and updates tags with a PATCH request · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:170]: Unexpected error:
    <*fmt.wrapError | 0xc000d461a0>: 
    failed to start credential request: POST https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-tags-l4v8j8/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-tags-cluster/requestAdminCredential
    --------------------------------------------------------------------------------
    RESPONSE 409: 409 Conflict
    ERROR CODE: Conflict
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "Conflict",
        "message": "Cannot request credential while resource is \"provisioning\"",
        "target": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-tags-l4v8j8/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-tags-cluster"
      }
    }
    --------------------------------------------------------------------------------
    
    {
        msg: "failed to start credential request: POST https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-tags-l4v8j8/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-tags-cluster/requestAdminCredential\n--------------------------------------------------------------------------------\nRESPONSE 409: 409 Conflict\nERROR CODE: Conflict\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"Conflict\",\n    \"message\": \"Cannot request credential while resource is \\\"provisioning\\\"\",\n    \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-tags-l4v8j8/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-tags-cluster\"\n  }\n}\n--------------------------------------------------------------------------------\n",
        err: <*exported.ResponseError | 0xc0014c8630>{
            ErrorCode: "Conflict",
            StatusCode: 409,
            RawResponse: {
                Status: "409 Conflict",
                StatusCode: 409,
                Proto: "HTTP/2.0",
                ProtoMajor: 2,
                ProtoMinor: 0,
                Header: {
                    "Pragma": ["no-cache"],
                    "X-Ms-Error-Code": ["Conflict"],
                    "X-Envoy-Upstream-Service-Time": ["24"],
                    "X-Ms-Correlation-Request-Id": [
                        "04a3d0b4-7658-4960-bd57-a56b1404d40f",
                    ],
                    "X-Ms-Routing-Request-Id": [
                        "UKSOUTH:20260304T114425Z:04a3d0b4-7658-4960-bd57-a56b1404d40f",
                    ],
                    "X-Cache": ["CONFIG_NOCACHE"],
                    "Cache-Control": ["no-cache"],
                    "Content-Length": ["316"],
                    "Expires": ["-1"],
                    "X-Ms-Operation-Identifier": [
                        "appId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,tenantId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,objectId=0e96dbb7-17a2-4393-b2d3-d84524e39c32/uksouth/75e212f8-cdc9-4ef9-961a-651c91f9ecd9",
                    ],
                    "X-Ms-Ratelimit-Remaining-Subscription-Writes": ["199"],
                    "Strict-Transport-Security": [
                        "max-age=31536000; includeSubDomains",
                    ],
                    "X-Content-Type-Options": ["nosniff"],
                    "X-Ms-Request-Id": [
                        "51931baa-6371-4efa-a708-4f0f312f1ca0",
                    ],
                    "Date": [
                        "Wed, 04 Mar 2026 11:44:25 GMT",
                    ],
                    "Content-Type": ["application/json"],
                    "X-Ms-Ratelimit-Remaining-Subscription-Global-Writes": ["2999"],
                    "X-Msedge-Ref": [
                        "Ref A: 8417EFDD14874F8BB0E5237BA546D57F Ref B: CHI261140512040 Ref C: 2026-03-04T11:44:25Z",
                    ],
                },
                Body: <*exported.nopClosingBytesReader | 0xc000d46000>{
                    s: [123, 10, 32, 32, 32, 32, 34, 101, 114, 114, 111, 114, 34, 58, 32, 123, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 99, 111, 100, 101, 34, 58, 32, 34, 67, 111, 110, 102, 108, 105, 99, 116, 34, 44, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 109, 101, 115, 115, 97, 103, 101, 34, 58, 32, 34, 67, 97, 110, 110, 111, 116, 32, 114, 101, 113, 117, 101, 115, 116, 32, 99, 114, 101, 100, 101, 110, 116, 105, 97, 108, 32, 119, 104, 105, 108, 101, 32, 114, 101, 115, 111, 117, 114, 99, 101, 32, 105, 115, 32, 92, 34, 112, 114, 111, 118, 105, 115, 105, 111, 110, 105, 110, 103, 92, 34, 34, 44, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 116, 97, 114, 103, 101, 116, 34, 58, 32, 34, 47, 115, 117, 98, 115, 99, 114, 105, 112, 116, 105, 111, 110, 115, 47, 52, 48, 51, 100, 57, 100, 101, 57, 45, 49, 51, 50, 98, 45, 52, 57, 55, 52, 45, 57, 52, 97, 53, 45, 53, 98, 55, 56, 98, 100, 102, 97, 49, 57, 49, 101, 47, 11...

Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.

Learn more here: https://onsi.github.io/gomega/#adjusting-output

occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:170]: Unexpected error:
    <*fmt.wrapError | 0xc000d461a0>: 
    failed to start credential request: POST https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-tags-l4v8j8/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-tags-cluster/requestAdminCredential
    --------------------------------------------------------------------------------
    RESPONSE 409: 409 Conflict
    ERROR CODE: Conflict
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "Conflict",
        "message": "Cannot request credential while resource is \"provisioning\"",
        "target": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-tags-l4v8j8/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-tags-cluster"
      }
    }
    --------------------------------------------------------------------------------
    
    {
        msg: "failed to start credential request: POST https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-tags-l4v8j8/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-tags-cluster/requestAdminCredential\n--------------------------------------------------------------------------------\nRESPONSE 409: 409 Conflict\nERROR CODE: Conflict\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"Conflict\",\n    \"message\": \"Cannot request credential while resource is \\\"provisioning\\\"\",\n    \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-tags-l4v8j8/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-tags-cluster\"\n  }\n}\n--------------------------------------------------------------------------------\n",
        err: <*exported.ResponseError | 0xc0014c8630>{
            ErrorCode: "Conflict",
            StatusCode: 409,
            RawResponse: {
                Status: "409 Conflict",
                StatusCode: 409,
                Proto: "HTTP/2.0",
                ProtoMajor: 2,
                ProtoMinor: 0,
                Header: {
                    "Pragma": ["no-cache"],
                    "X-Ms-Error-Code": ["Conflict"],
                    "X-Envoy-Upstream-Service-Time": ["24"],
                    "X-Ms-Correlation-Request-Id": [
                        "04a3d0b4-7658-4960-bd57-a56b1404d40f",
                    ],
                    "X-Ms-Routing-Request-Id": [
                        "UKSOUTH:20260304T114425Z:04a3d0b4-7658-4960-bd57-a56b1404d40f",
                    ],
                    "X-Cache": ["CONFIG_NOCACHE"],
                    "Cache-Control": ["no-cache"],
                    "Content-Length": ["316"],
                    "Expires": ["-1"],
                    "X-Ms-Operation-Identifier": [
                        "appId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,tenantId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,objectId=0e96dbb7-17a2-4393-b2d3-d84524e39c32/uksouth/75e212f8-cdc9-4ef9-961a-651c91f9ecd9",
                    ],
                    "X-Ms-Ratelimit-Remaining-Subscription-Writes": ["199"],
                    "Strict-Transport-Security": [
                        "max-age=31536000; includeSubDomains",
                    ],
                    "X-Content-Type-Options": ["nosniff"],
                    "X-Ms-Request-Id": [
                        "51931baa-6371-4efa-a708-4f0f312f1ca0",
                    ],
                    "Date": [
                        "Wed, 04 Mar 2026 11:44:25 GMT",
                    ],
                    "Content-Type": ["application/json"],
                    "X-Ms-Ratelimit-Remaining-Subscription-Global-Writes": ["2999"],
                    "X-Msedge-Ref": [
                        "Ref A: 8417EFDD14874F8BB0E5237BA546D57F Ref B: CHI261140512040 Ref C: 2026-03-04T11:44:25Z",
                    ],
                },
                Body: <*exported.nopClosingBytesReader | 0xc000d46000>{
                    s: [123, 10, 32, 32, 32, 32, 34, 101, 114, 114, 111, 114, 34, 58, 32, 123, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 99, 111, 100, 101, 34, 58, 32, 34, 67, 111, 110, 102, 108, 105, 99, 116, 34, 44, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 109, 101, 115, 115, 97, 103, 101, 34, 58, 32, 34, 67, 97, 110, 110, 111, 116, 32, 114, 101, 113, 117, 101, 115, 116, 32, 99, 114, 101, 100, 101, 110, 116, 105, 97, 108, 32, 119, 104, 105, 108, 101, 32, 114, 101, 115, 111, 117, 114, 99, 101, 32, 105, 115, 32, 92, 34, 112, 114, 111, 118, 105, 115, 105, 111, 110, 105, 110, 103, 92, 34, 34, 44, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 116, 97, 114, 103, 101, 116, 34, 58, 32, 34, 47, 115, 117, 98, 115, 99, 114, 105, 112, 116, 105, 111, 110, 115, 47, 52, 48, 51, 100, 57, 100, 101, 57, 45, 49, 51, 50, 98, 45, 52, 57, 55, 52, 45, 57, 52, 97, 53, 45, 53, 98, 55, 56, 98, 100, 102, 97, 49, 57, 49, 101, 47, 11...

Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.

Learn more here: https://onsi.github.io/gomega/#adjusting-output

occurred

ERROR CODE: InternalServerError; detail message insufficient public IP address quota: required <count>, available <count>; provider Microsoft.RedHatOpenShift

11:18:47 UTC · failed at e2e · test Customer should be able to create an HCP cluster with custom autoscaling · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:97]: Unexpected error:
    <*fmt.wrapError | 0xc0007439c0>: 
    failed to create HCP cluster autoscaling-hcp-cluster: failed waiting for cluster="autoscaling-hcp-cluster" in resourcegroup="autoscaling-cluster-rggf4c" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b1c8c857-485d-4a3c-b764-cd54b5d212a2
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b1c8c857-485d-4a3c-b764-cd54b5d212a2",
      "name": "b1c8c857-485d-4a3c-b764-cd54b5d212a2",
      "status": "Failed",
      "startTime": "2026-03-04T11:42:32.91171511Z",
      "endTime": "2026-03-04T11:45:01.618825129Z",
      "error": {
        "code": "InternalServerError",
        "message": "insufficient public IP address quota: required 2, available 1"
      }
    }
    --------------------------------------------------------------------------------
    
    {
        msg: "failed to create HCP cluster autoscaling-hcp-cluster: failed waiting for cluster=\"autoscaling-hcp-cluster\" in resourcegroup=\"autoscaling-cluster-rggf4c\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b1c8c857-485d-4a3c-b764-cd54b5d212a2\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n  \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b1c8c857-485d-4a3c-b764-cd54b5d212a2\",\n  \"name\": \"b1c8c857-485d-4a3c-b764-cd54b5d212a2\",\n  \"status\": \"Failed\",\n  \"startTime\": \"2026-03-04T11:42:32.91171511Z\",\n  \"endTime\": \"2026-03-04T11:45:01.618825129Z\",\n  \"error\": {\n    \"code\": \"InternalServerError\",\n    \"message\": \"insufficient public IP address quota: required 2, available 1\"\n  }\n}\n--------------------------------------------------------------------------------\n",
        err: <*fmt.wrapError | 0xc0007439a0>{
            msg: "failed waiting for cluster=\"autoscaling-hcp-cluster\" in resourcegroup=\"autoscaling-cluster-rggf4c\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b1c8c857-485d-4a3c-b764-cd54b5d212a2\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n  \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b1c8c857-485d-4a3c-b764-cd54b5d212a2\",\n  \"name\": \"b1c8c857-485d-4a3c-b764-cd54b5d212a2\",\n  \"status\": \"Failed\",\n  \"startTime\": \"2026-03-04T11:42:32.91171511Z\",\n  \"endTime\": \"2026-03-04T11:45:01.618825129Z\",\n  \"error\": {\n    \"code\": \"InternalServerError\",\n    \"message\": \"insufficient public IP address quota: required 2, available 1\"\n  }\n}\n--------------------------------------------------------------------------------\n",
            err: <*exported.ResponseError | 0xc000679c80>{
                ErrorCode: "InternalServerError",
                StatusCode: 200,
                RawResponse: {
                    Status: "200 OK",
                    StatusCode: 200,
                    Proto: "HTTP/2.0",
                    ProtoMajor: 2,
                    ProtoMinor: 0,
                    Header: {
                        "Cache-Control": ["no-cache"],
                        "Expires": ["-1"],
                        "X-Ms-Routing-Request-Id": [
                            "CANADACENTRAL:20260304T114509Z:d921fb07-cad8-47f3-beb6-14b898dd9e31",
                        ],
                        "X-Msedge-Ref": [
                            "Ref A: C1EE60EB4B804177AABFC80DC74FA33E Ref B: CHI261140501054 Ref C: 2026-03-04T11:45:08Z",
                        ],
                        "X-Ms-Correlation-Request-Id": [
                            "d921fb07-cad8-47f3-beb6-14b898dd9e31",
                        ],
                        "X-Content-Type-Options": ["nosniff"],
                        "Pragma": ["no-cache"],
                        "Content-Length": ["498"],
                        "X-Envoy-Upstream-Service-Time": ["11"],
                        "X-Ms-Operation-Identifier": [
                            "appId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,tenantId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,objectId=0e96dbb7-17a2-4393-b2d3-d84524e39c32/canadacentral/78cb4115-38f4-400a-a373-de90cb3f6447",
                        ],
                        "X-Ms-Ratelimit-Remaining...

Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.

Learn more here: https://onsi.github.io/gomega/#adjusting-output

occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:97]: Unexpected error:
    <*fmt.wrapError | 0xc0007439c0>: 
    failed to create HCP cluster autoscaling-hcp-cluster: failed waiting for cluster="autoscaling-hcp-cluster" in resourcegroup="autoscaling-cluster-rggf4c" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b1c8c857-485d-4a3c-b764-cd54b5d212a2
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b1c8c857-485d-4a3c-b764-cd54b5d212a2",
      "name": "b1c8c857-485d-4a3c-b764-cd54b5d212a2",
      "status": "Failed",
      "startTime": "2026-03-04T11:42:32.91171511Z",
      "endTime": "2026-03-04T11:45:01.618825129Z",
      "error": {
        "code": "InternalServerError",
        "message": "insufficient public IP address quota: required 2, available 1"
      }
    }
    --------------------------------------------------------------------------------
    
    {
        msg: "failed to create HCP cluster autoscaling-hcp-cluster: failed waiting for cluster=\"autoscaling-hcp-cluster\" in resourcegroup=\"autoscaling-cluster-rggf4c\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b1c8c857-485d-4a3c-b764-cd54b5d212a2\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n  \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b1c8c857-485d-4a3c-b764-cd54b5d212a2\",\n  \"name\": \"b1c8c857-485d-4a3c-b764-cd54b5d212a2\",\n  \"status\": \"Failed\",\n  \"startTime\": \"2026-03-04T11:42:32.91171511Z\",\n  \"endTime\": \"2026-03-04T11:45:01.618825129Z\",\n  \"error\": {\n    \"code\": \"InternalServerError\",\n    \"message\": \"insufficient public IP address quota: required 2, available 1\"\n  }\n}\n--------------------------------------------------------------------------------\n",
        err: <*fmt.wrapError | 0xc0007439a0>{
            msg: "failed waiting for cluster=\"autoscaling-hcp-cluster\" in resourcegroup=\"autoscaling-cluster-rggf4c\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b1c8c857-485d-4a3c-b764-cd54b5d212a2\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n  \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b1c8c857-485d-4a3c-b764-cd54b5d212a2\",\n  \"name\": \"b1c8c857-485d-4a3c-b764-cd54b5d212a2\",\n  \"status\": \"Failed\",\n  \"startTime\": \"2026-03-04T11:42:32.91171511Z\",\n  \"endTime\": \"2026-03-04T11:45:01.618825129Z\",\n  \"error\": {\n    \"code\": \"InternalServerError\",\n    \"message\": \"insufficient public IP address quota: required 2, available 1\"\n  }\n}\n--------------------------------------------------------------------------------\n",
            err: <*exported.ResponseError | 0xc000679c80>{
                ErrorCode: "InternalServerError",
                StatusCode: 200,
                RawResponse: {
                    Status: "200 OK",
                    StatusCode: 200,
                    Proto: "HTTP/2.0",
                    ProtoMajor: 2,
                    ProtoMinor: 0,
                    Header: {
                        "Cache-Control": ["no-cache"],
                        "Expires": ["-1"],
                        "X-Ms-Routing-Request-Id": [
                            "CANADACENTRAL:20260304T114509Z:d921fb07-cad8-47f3-beb6-14b898dd9e31",
                        ],
                        "X-Msedge-Ref": [
                            "Ref A: C1EE60EB4B804177AABFC80DC74FA33E Ref B: CHI261140501054 Ref C: 2026-03-04T11:45:08Z",
                        ],
                        "X-Ms-Correlation-Request-Id": [
                            "d921fb07-cad8-47f3-beb6-14b898dd9e31",
                        ],
                        "X-Content-Type-Options": ["nosniff"],
                        "Pragma": ["no-cache"],
                        "Content-Length": ["498"],
                        "X-Envoy-Upstream-Service-Time": ["11"],
                        "X-Ms-Operation-Identifier": [
                            "appId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,tenantId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,objectId=0e96dbb7-17a2-4393-b2d3-d84524e39c32/canadacentral/78cb4115-38f4-400a-a373-de90cb3f6447",
                        ],
                        "X-Ms-Ratelimit-Remaining...

Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.

Learn more here: https://onsi.github.io/gomega/#adjusting-output

occurred

Cluster provisioning failed

11:18:47 UTC · failed at e2e · test Customer should be able to test admin credentials before cluster ready, then full admin credential lifecycle · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:153]: Cluster provisioning failed
fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:153]: Cluster provisioning failed

ERROR CODE: InternalServerError; detail code PublicIPCountLimitReached; detail message Cannot create more than 300 public IP addresses for this subscription in this region.; provider Microsoft.Network

11:18:47 UTC · failed at e2e · test Customer should respect cluster-wide node limits with nodepool autoscaling · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:166]: Unexpected error:
    <*fmt.wrapError | 0xc000c090a0>: 
    failed to create HCP cluster node-limit-cluster: failed waiting for cluster="node-limit-cluster" in resourcegroup="node-limit-test-v554wg" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b6c96910-d9fe-4ee6-be6a-ddfe7e0aebc3
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b6c96910-d9fe-4ee6-be6a-ddfe7e0aebc3",
      "name": "b6c96910-d9fe-4ee6-be6a-ddfe7e0aebc3",
      "status": "Failed",
      "startTime": "2026-03-04T11:29:07.030229354Z",
      "endTime": "2026-03-04T11:33:56.440551062Z",
      "error": {
        "code": "InternalServerError",
        "message": "PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/node-limit-test-v554wg--managed/providers/Microsoft.Network/publicIPAddresses/2oqsj0l8tte65mugqrulb0nrcn5o984k-0\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"PublicIPCountLimitReached\",\n    \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n    \"details\": []\n  }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/node-limit-test-v554wg--managed/providers/Microsoft.Network/publicIPAddresses/2oqsj0l8tte65mugqrulb0nrcn5o984k-1\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"PublicIPCountLimitReached\",\n    \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n    \"details\": []\n  }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/node-limit-test-v554wg--managed/providers/Microsoft.Network/publicIPAddresses/2oqsj0l8tte65mugqrulb0nrcn5o984k-2\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"PublicIPCountLimitReached\",\n    \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n    \"details\": []\n  }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/node-limit-test-v554wg--managed/providers/Microsoft.Network/publicIPAddresses/2oqsj0l8tte65mugqrulb0nrcn5o984k-3\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"PublicIPCountLimitReached\",\n    \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n    \"details\": []\n  }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/node-limit-test-v554wg--managed/providers/Microsoft.Network/publicIPAddresses/2oqsj0l8tte65mugqrulb0nrcn5o984k-4\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"PublicIPCountLimitReached\",\n    \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n    \"details\": []\n  }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/node-limit-test-v554wg--managed/providers/Microsoft.Network/publicIPAddresses/2oqsj0l8tte65mugqrulb0nrcn5o984k-5\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"PublicIPCountLimitReached\",\n    \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n    \"details\": []\n  }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/node-limit-test-v554wg--managed/providers/Microsoft.Network/publicIPAddresses/2oqsj0l8tte65mugqrulb0nrcn5o984k-6\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"PublicIPCountLimitReached\",\n    \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n    \"details\": []\n  }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/node-limit-test-v554wg--managed/providers/Microsoft.Network/publicIPAddresses/2oqsj0l8tte65mugqrulb0nrcn5o984k-7\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"PublicIPCountLimitReached\",\n    \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n    \"details\": []\n  }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/node-limit-test-v554wg--managed/providers/Microsoft.Network/publicIPAddresses/2oqsj0l8tte65mugqrulb0nrcn5o984k-8\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"PublicIPCountLimitReached\",\n    \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n    \"details\": []\n  }\n}\n--------------------------------------------------------------------------------\n"
      }
    }
    --------------------------------------------------------------------------------
    
    {
        msg: "failed to create HCP cluster node-limit-cluster: failed waiting for cluster=\"node-limit-cluster\" in resourcegroup=\"node-limit-test-v554wg\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b6c96910-d9fe-4ee6-be6a-ddfe7e0aebc3\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n  \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b6c96910-d9fe-4ee6-be6a-ddfe7e0aebc3\",\n  \"name\": \"b6c96910-d9fe-4ee6-be6a-ddfe7e0aebc3\",\n  \"status\": \"Failed\",\n  \"startTime\": \"2026-03-04T11:29:07.030229354Z\",\n  \"endTime\": \"2026-03-04T11:33:56.440551062Z\",\n  \"error\": {\n    \"code\": \"InternalServerError\",\n    \"message\": \"PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/node-limit-test-v554wg--managed/providers/Microsoft.Network/publicIPAddresses/2oqsj0l8tte65mugqrulb0nrcn5o984k-0\\n--------------------------------------------------------------------------------\\nRESPONSE 400: 400 Bad Request\\nERROR CODE: PublicIPCountLimitReached\\n--------------------------------------------------------------------------------\\n{\\n  \\\"error\\\": {\\n    \\\"code\\\": \\\"PublicIPCountLimitReached\\\",\\n    \\\"message\\\": \\\"Cannot create more than 300 public IP addresses for this subscription in this region.\\\",\\n    \\\"details\\\": []\\n  }\\n}\\n--------------------------------------------------------------------------------\\n\\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/node-limit-test-v554wg--managed/providers/Microsoft.Network/publicIPAddresses/2oqsj0l8tte65mugqrulb0nrcn5o984k-1\\n--------------------------------------------------------------------------------\\nRESPONSE 400: 400 Bad Request\\nERROR CODE: PublicIPCountLimitReached\\n--------------------------------------------------------------------------------\\n{\\n  \\\"error\\\": {\\n    \\\"code\\\": \\\"PublicIPCountLimitReached\\\",\\n    \\\"message\\\": \\\"Cannot create more than 300 public IP addresses for this subscription in this region.\\\",\\n    \\\"details\\\": []\\n  }\\n}\\n--------------------------------------------------------------------------------\\n\\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/node-limit-test-v554wg--managed/providers/Microsoft.Network/publicIPAddresses/2oqsj0l8tte65mugqrulb0nrcn5o984k-2\\n--------------------------------------------------------------------------------\\nRESPONSE 400: 400 Bad Request\\nERROR CODE: PublicIPCountLimitReached\\n--------------------------------------------------------------------------------\\n{\\n  \\\"error\\\": {\\n    \\\"code\\\": \\\"PublicIPCountLimitReached\\\",\\n    \\\"message\\\": \\\"Cannot create more than 300 public IP addresses for this subscription in this region.\\\",\\n    \\\"details\\\": []\\n  }\\n}\\n--------------------------------------------------------------------------------\\n\\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/node-limit-test-v554wg--managed/providers/Microsoft.Network/publicIPAddresses/2oqsj0l8tte65mugqrulb0nrcn5o984k-3\\n--------------------------------------------------------------------------------\\nRESPONSE 400: 400 Bad Request\\nERROR CODE: PublicIPCountLimitReached\\n--------------------------------------------------------------------------------\\n{\\n  \\\"error\\\": {\\n    \\\"code\\\": \\\"PublicIPCountLimitReached\\\",\\n    \\\"message\\\": \\\"Cannot create more than 300 public IP addresses for this subscription in this region.\...

Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.

Learn more here: https://onsi.github.io/gomega/#adjusting-output

occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:166]: Unexpected error:
    <*fmt.wrapError | 0xc000c090a0>: 
    failed to create HCP cluster node-limit-cluster: failed waiting for cluster="node-limit-cluster" in resourcegroup="node-limit-test-v554wg" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b6c96910-d9fe-4ee6-be6a-ddfe7e0aebc3
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b6c96910-d9fe-4ee6-be6a-ddfe7e0aebc3",
      "name": "b6c96910-d9fe-4ee6-be6a-ddfe7e0aebc3",
      "status": "Failed",
      "startTime": "2026-03-04T11:29:07.030229354Z",
      "endTime": "2026-03-04T11:33:56.440551062Z",
      "error": {
        "code": "InternalServerError",
        "message": "PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/node-limit-test-v554wg--managed/providers/Microsoft.Network/publicIPAddresses/2oqsj0l8tte65mugqrulb0nrcn5o984k-0\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"PublicIPCountLimitReached\",\n    \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n    \"details\": []\n  }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/node-limit-test-v554wg--managed/providers/Microsoft.Network/publicIPAddresses/2oqsj0l8tte65mugqrulb0nrcn5o984k-1\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"PublicIPCountLimitReached\",\n    \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n    \"details\": []\n  }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/node-limit-test-v554wg--managed/providers/Microsoft.Network/publicIPAddresses/2oqsj0l8tte65mugqrulb0nrcn5o984k-2\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"PublicIPCountLimitReached\",\n    \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n    \"details\": []\n  }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/node-limit-test-v554wg--managed/providers/Microsoft.Network/publicIPAddresses/2oqsj0l8tte65mugqrulb0nrcn5o984k-3\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"PublicIPCountLimitReached\",\n    \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n    \"details\": []\n  }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/node-limit-test-v554wg--managed/providers/Microsoft.Network/publicIPAddresses/2oqsj0l8tte65mugqrulb0nrcn5o984k-4\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"PublicIPCountLimitReached\",\n    \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n    \"details\": []\n  }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/node-limit-test-v554wg--managed/providers/Microsoft.Network/publicIPAddresses/2oqsj0l8tte65mugqrulb0nrcn5o984k-5\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"PublicIPCountLimitReached\",\n    \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n    \"details\": []\n  }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/node-limit-test-v554wg--managed/providers/Microsoft.Network/publicIPAddresses/2oqsj0l8tte65mugqrulb0nrcn5o984k-6\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"PublicIPCountLimitReached\",\n    \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n    \"details\": []\n  }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/node-limit-test-v554wg--managed/providers/Microsoft.Network/publicIPAddresses/2oqsj0l8tte65mugqrulb0nrcn5o984k-7\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"PublicIPCountLimitReached\",\n    \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n    \"details\": []\n  }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/node-limit-test-v554wg--managed/providers/Microsoft.Network/publicIPAddresses/2oqsj0l8tte65mugqrulb0nrcn5o984k-8\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"PublicIPCountLimitReached\",\n    \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n    \"details\": []\n  }\n}\n--------------------------------------------------------------------------------\n"
      }
    }
    --------------------------------------------------------------------------------
    
    {
        msg: "failed to create HCP cluster node-limit-cluster: failed waiting for cluster=\"node-limit-cluster\" in resourcegroup=\"node-limit-test-v554wg\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b6c96910-d9fe-4ee6-be6a-ddfe7e0aebc3\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n  \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b6c96910-d9fe-4ee6-be6a-ddfe7e0aebc3\",\n  \"name\": \"b6c96910-d9fe-4ee6-be6a-ddfe7e0aebc3\",\n  \"status\": \"Failed\",\n  \"startTime\": \"2026-03-04T11:29:07.030229354Z\",\n  \"endTime\": \"2026-03-04T11:33:56.440551062Z\",\n  \"error\": {\n    \"code\": \"InternalServerError\",\n    \"message\": \"PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/node-limit-test-v554wg--managed/providers/Microsoft.Network/publicIPAddresses/2oqsj0l8tte65mugqrulb0nrcn5o984k-0\\n--------------------------------------------------------------------------------\\nRESPONSE 400: 400 Bad Request\\nERROR CODE: PublicIPCountLimitReached\\n--------------------------------------------------------------------------------\\n{\\n  \\\"error\\\": {\\n    \\\"code\\\": \\\"PublicIPCountLimitReached\\\",\\n    \\\"message\\\": \\\"Cannot create more than 300 public IP addresses for this subscription in this region.\\\",\\n    \\\"details\\\": []\\n  }\\n}\\n--------------------------------------------------------------------------------\\n\\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/node-limit-test-v554wg--managed/providers/Microsoft.Network/publicIPAddresses/2oqsj0l8tte65mugqrulb0nrcn5o984k-1\\n--------------------------------------------------------------------------------\\nRESPONSE 400: 400 Bad Request\\nERROR CODE: PublicIPCountLimitReached\\n--------------------------------------------------------------------------------\\n{\\n  \\\"error\\\": {\\n    \\\"code\\\": \\\"PublicIPCountLimitReached\\\",\\n    \\\"message\\\": \\\"Cannot create more than 300 public IP addresses for this subscription in this region.\\\",\\n    \\\"details\\\": []\\n  }\\n}\\n--------------------------------------------------------------------------------\\n\\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/node-limit-test-v554wg--managed/providers/Microsoft.Network/publicIPAddresses/2oqsj0l8tte65mugqrulb0nrcn5o984k-2\\n--------------------------------------------------------------------------------\\nRESPONSE 400: 400 Bad Request\\nERROR CODE: PublicIPCountLimitReached\\n--------------------------------------------------------------------------------\\n{\\n  \\\"error\\\": {\\n    \\\"code\\\": \\\"PublicIPCountLimitReached\\\",\\n    \\\"message\\\": \\\"Cannot create more than 300 public IP addresses for this subscription in this region.\\\",\\n    \\\"details\\\": []\\n  }\\n}\\n--------------------------------------------------------------------------------\\n\\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/node-limit-test-v554wg--managed/providers/Microsoft.Network/publicIPAddresses/2oqsj0l8tte65mugqrulb0nrcn5o984k-3\\n--------------------------------------------------------------------------------\\nRESPONSE 400: 400 Bad Request\\nERROR CODE: PublicIPCountLimitReached\\n--------------------------------------------------------------------------------\\n{\\n  \\\"error\\\": {\\n    \\\"code\\\": \\\"PublicIPCountLimitReached\\\",\\n    \\\"message\\\": \\\"Cannot create more than 300 public IP addresses for this subscription in this region.\...

Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.

Learn more here: https://onsi.github.io/gomega/#adjusting-output

occurred

11:18:47 UTC · failed at e2e · test Customer should be able to create an HCP cluster with Image Registry not present · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/image_registry_cluster_create.go:89]: Unexpected error:
    <*fmt.wrapError | 0xc0014a86a0>: 
    failed to create HCP cluster disabled-image-registry-hcp-cluster: failed waiting for cluster="disabled-image-registry-hcp-cluster" in resourcegroup="disabled-image-registry-cqrzqm" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/ee06715b-b440-41cc-8866-ae10dbcc67f8
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/ee06715b-b440-41cc-8866-ae10dbcc67f8",
      "name": "ee06715b-b440-41cc-8866-ae10dbcc67f8",
      "status": "Failed",
      "startTime": "2026-03-04T11:52:51.02692019Z",
      "endTime": "2026-03-04T11:56:11.562177046Z",
      "error": {
        "code": "InternalServerError",
        "message": "PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/disabled-image-registry-cqrzqm--managed/providers/Microsoft.Network/publicIPAddresses/2oqsu4n7eod01mi46n8r90vsr5m1qqfp-8\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"PublicIPCountLimitReached\",\n    \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n    \"details\": []\n  }\n}\n--------------------------------------------------------------------------------\n"
      }
    }
    --------------------------------------------------------------------------------
    
    {
        msg: "failed to create HCP cluster disabled-image-registry-hcp-cluster: failed waiting for cluster=\"disabled-image-registry-hcp-cluster\" in resourcegroup=\"disabled-image-registry-cqrzqm\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/ee06715b-b440-41cc-8866-ae10dbcc67f8\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n  \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/ee06715b-b440-41cc-8866-ae10dbcc67f8\",\n  \"name\": \"ee06715b-b440-41cc-8866-ae10dbcc67f8\",\n  \"status\": \"Failed\",\n  \"startTime\": \"2026-03-04T11:52:51.02692019Z\",\n  \"endTime\": \"2026-03-04T11:56:11.562177046Z\",\n  \"error\": {\n    \"code\": \"InternalServerError\",\n    \"message\": \"PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/disabled-image-registry-cqrzqm--managed/providers/Microsoft.Network/publicIPAddresses/2oqsu4n7eod01mi46n8r90vsr5m1qqfp-8\\n--------------------------------------------------------------------------------\\nRESPONSE 400: 400 Bad Request\\nERROR CODE: PublicIPCountLimitReached\\n--------------------------------------------------------------------------------\\n{\\n  \\\"error\\\": {\\n    \\\"code\\\": \\\"PublicIPCountLimitReached\\\",\\n    \\\"message\\\": \\\"Cannot create more than 300 public IP addresses for this subscription in this region.\\\",\\n    \\\"details\\\": []\\n  }\\n}\\n--------------------------------------------------------------------------------\\n\"\n  }\n}\n--------------------------------------------------------------------------------\n",
        err: <*fmt.wrapError | 0xc0014a8680>{
            msg: "failed waiting for cluster=\"disabled-image-registry-hcp-cluster\" in resourcegroup=\"disabled-image-registry-cqrzqm\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/ee06715b-b440-41cc-8866-ae10dbcc67f8\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n  \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/ee06715b-b440-41cc-8866-ae10dbcc67f8\",\n  \"name\": \"ee06715b-b440-41cc-8866-ae10dbcc67f8\",\n  \"status\": \"Failed\",\n  \"startTime\": \"2026-03-04T11:52:51.02692019Z\",\n  \"endTime\": \"2026-03-04T11:56:11.562177046Z\",\n  \"error\": {\n    \"code\": \"InternalServerError\",\n    \"message\": \"PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/disabled-image-registry-cqrzqm--managed/providers/Microsoft.Network/publicIPAddresses/2oqsu4n7eod01mi46n8r90vsr5m1qqfp-8\\n--------------------------------------------------------------------------------\\nRESPONSE 400: 400 Bad Request\\nERROR CODE: PublicIPCountLimitReached\\n--------------------------------------------------------------------------------\\n{\\n  \\\"error\\\": {\\n    \\\"code\\\": \\\"PublicIPCountLimitReached\\\",\\n    \\\"message\\\": \\\"Cannot create more than 300 public IP addresses for this subscription in this region.\\\",\\n    \\\"details\\\": []\\n  }\\n}\\n--------------------------------------------------------------------------------\\n\"\n  }\n}\n--------------------------------------------------------------------------------\n",
            err: <*exported.ResponseError | 0xc000df4d20>{
                ErrorCode: "InternalServerError...

Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.

Learn more here: https://onsi.github.io/gomega/#adjusting-output

occurred
fail [github.com/Azure/ARO-HCP/test/e2e/image_registry_cluster_create.go:89]: Unexpected error:
    <*fmt.wrapError | 0xc0014a86a0>: 
    failed to create HCP cluster disabled-image-registry-hcp-cluster: failed waiting for cluster="disabled-image-registry-hcp-cluster" in resourcegroup="disabled-image-registry-cqrzqm" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/ee06715b-b440-41cc-8866-ae10dbcc67f8
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/ee06715b-b440-41cc-8866-ae10dbcc67f8",
      "name": "ee06715b-b440-41cc-8866-ae10dbcc67f8",
      "status": "Failed",
      "startTime": "2026-03-04T11:52:51.02692019Z",
      "endTime": "2026-03-04T11:56:11.562177046Z",
      "error": {
        "code": "InternalServerError",
        "message": "PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/disabled-image-registry-cqrzqm--managed/providers/Microsoft.Network/publicIPAddresses/2oqsu4n7eod01mi46n8r90vsr5m1qqfp-8\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"PublicIPCountLimitReached\",\n    \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n    \"details\": []\n  }\n}\n--------------------------------------------------------------------------------\n"
      }
    }
    --------------------------------------------------------------------------------
    
    {
        msg: "failed to create HCP cluster disabled-image-registry-hcp-cluster: failed waiting for cluster=\"disabled-image-registry-hcp-cluster\" in resourcegroup=\"disabled-image-registry-cqrzqm\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/ee06715b-b440-41cc-8866-ae10dbcc67f8\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n  \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/ee06715b-b440-41cc-8866-ae10dbcc67f8\",\n  \"name\": \"ee06715b-b440-41cc-8866-ae10dbcc67f8\",\n  \"status\": \"Failed\",\n  \"startTime\": \"2026-03-04T11:52:51.02692019Z\",\n  \"endTime\": \"2026-03-04T11:56:11.562177046Z\",\n  \"error\": {\n    \"code\": \"InternalServerError\",\n    \"message\": \"PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/disabled-image-registry-cqrzqm--managed/providers/Microsoft.Network/publicIPAddresses/2oqsu4n7eod01mi46n8r90vsr5m1qqfp-8\\n--------------------------------------------------------------------------------\\nRESPONSE 400: 400 Bad Request\\nERROR CODE: PublicIPCountLimitReached\\n--------------------------------------------------------------------------------\\n{\\n  \\\"error\\\": {\\n    \\\"code\\\": \\\"PublicIPCountLimitReached\\\",\\n    \\\"message\\\": \\\"Cannot create more than 300 public IP addresses for this subscription in this region.\\\",\\n    \\\"details\\\": []\\n  }\\n}\\n--------------------------------------------------------------------------------\\n\"\n  }\n}\n--------------------------------------------------------------------------------\n",
        err: <*fmt.wrapError | 0xc0014a8680>{
            msg: "failed waiting for cluster=\"disabled-image-registry-hcp-cluster\" in resourcegroup=\"disabled-image-registry-cqrzqm\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/ee06715b-b440-41cc-8866-ae10dbcc67f8\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n  \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/ee06715b-b440-41cc-8866-ae10dbcc67f8\",\n  \"name\": \"ee06715b-b440-41cc-8866-ae10dbcc67f8\",\n  \"status\": \"Failed\",\n  \"startTime\": \"2026-03-04T11:52:51.02692019Z\",\n  \"endTime\": \"2026-03-04T11:56:11.562177046Z\",\n  \"error\": {\n    \"code\": \"InternalServerError\",\n    \"message\": \"PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/disabled-image-registry-cqrzqm--managed/providers/Microsoft.Network/publicIPAddresses/2oqsu4n7eod01mi46n8r90vsr5m1qqfp-8\\n--------------------------------------------------------------------------------\\nRESPONSE 400: 400 Bad Request\\nERROR CODE: PublicIPCountLimitReached\\n--------------------------------------------------------------------------------\\n{\\n  \\\"error\\\": {\\n    \\\"code\\\": \\\"PublicIPCountLimitReached\\\",\\n    \\\"message\\\": \\\"Cannot create more than 300 public IP addresses for this subscription in this region.\\\",\\n    \\\"details\\\": []\\n  }\\n}\\n--------------------------------------------------------------------------------\\n\"\n  }\n}\n--------------------------------------------------------------------------------\n",
            err: <*exported.ResponseError | 0xc000df4d20>{
                ErrorCode: "InternalServerError...

Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.

Learn more here: https://onsi.github.io/gomega/#adjusting-output

occurred

CreateNodePoolFromParam timed out after <minutes> minutes; context deadline exceeded

11:18:47 UTC · failed at e2e · test Customer should create an HCP cluster and validate TLS certificates · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:114]: Unexpected error:
    <*fmt.wrapErrors | 0xc0013f4bd0>: 
    failed to create NodePool np-1, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group tls-endpoint-cluster-tjpbzb, error: failed waiting for nodepool="np-1" for cluster "tls-endpoint-hcp-cluster" in resourcegroup="tls-endpoint-cluster-tjpbzb" to finish creating: context deadline exceeded
    {
        msg: "failed to create NodePool np-1, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group tls-endpoint-cluster-tjpbzb, error: failed waiting for nodepool=\"np-1\" for cluster \"tls-endpoint-hcp-cluster\" in resourcegroup=\"tls-endpoint-cluster-tjpbzb\" to finish creating: context deadline exceeded",
        errs: [
            <*errors.errorString | 0xc0009ed300>{
                s: "timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group tls-endpoint-cluster-tjpbzb",
            },
            <*fmt.wrapError | 0xc000f4a800>{
                msg: "failed waiting for nodepool=\"np-1\" for cluster \"tls-endpoint-hcp-cluster\" in resourcegroup=\"tls-endpoint-cluster-tjpbzb\" to finish creating: context deadline exceeded",
                err: <context.deadlineExceededError>{},
            },
        ],
    }
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:114]: Unexpected error:
    <*fmt.wrapErrors | 0xc0013f4bd0>: 
    failed to create NodePool np-1, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group tls-endpoint-cluster-tjpbzb, error: failed waiting for nodepool="np-1" for cluster "tls-endpoint-hcp-cluster" in resourcegroup="tls-endpoint-cluster-tjpbzb" to finish creating: context deadline exceeded
    {
        msg: "failed to create NodePool np-1, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group tls-endpoint-cluster-tjpbzb, error: failed waiting for nodepool=\"np-1\" for cluster \"tls-endpoint-hcp-cluster\" in resourcegroup=\"tls-endpoint-cluster-tjpbzb\" to finish creating: context deadline exceeded",
        errs: [
            <*errors.errorString | 0xc0009ed300>{
                s: "timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group tls-endpoint-cluster-tjpbzb",
            },
            <*fmt.wrapError | 0xc000f4a800>{
                msg: "failed waiting for nodepool=\"np-1\" for cluster \"tls-endpoint-hcp-cluster\" in resourcegroup=\"tls-endpoint-cluster-tjpbzb\" to finish creating: context deadline exceeded",
                err: <context.deadlineExceededError>{},
            },
        ],
    }
occurred

ERROR CODE: InternalServerError; detail message insufficient public IP address quota: required <count>, available <count>; provider Microsoft.RedHatOpenShift

11:18:47 UTC · failed at e2e · test Customer should be able to create a cluster with an external auth config and get the external auth config · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:93]: Unexpected error:
    <*fmt.wrapError | 0xc000a802a0>: 
    failed to create HCP cluster ea-cluster: failed waiting for cluster="ea-cluster" in resourcegroup="external-auth-cluster-66kqnv" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/dca31cdc-d199-43d3-b94c-109078e85b87
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/dca31cdc-d199-43d3-b94c-109078e85b87",
      "name": "dca31cdc-d199-43d3-b94c-109078e85b87",
      "status": "Failed",
      "startTime": "2026-03-04T11:44:31.085504793Z",
      "endTime": "2026-03-04T11:49:48.850191595Z",
      "error": {
        "code": "InternalServerError",
        "message": "insufficient public IP address quota: required 2, available 0"
      }
    }
    --------------------------------------------------------------------------------
    
    {
        msg: "failed to create HCP cluster ea-cluster: failed waiting for cluster=\"ea-cluster\" in resourcegroup=\"external-auth-cluster-66kqnv\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/dca31cdc-d199-43d3-b94c-109078e85b87\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n  \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/dca31cdc-d199-43d3-b94c-109078e85b87\",\n  \"name\": \"dca31cdc-d199-43d3-b94c-109078e85b87\",\n  \"status\": \"Failed\",\n  \"startTime\": \"2026-03-04T11:44:31.085504793Z\",\n  \"endTime\": \"2026-03-04T11:49:48.850191595Z\",\n  \"error\": {\n    \"code\": \"InternalServerError\",\n    \"message\": \"insufficient public IP address quota: required 2, available 0\"\n  }\n}\n--------------------------------------------------------------------------------\n",
        err: <*fmt.wrapError | 0xc000a80280>{
            msg: "failed waiting for cluster=\"ea-cluster\" in resourcegroup=\"external-auth-cluster-66kqnv\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/dca31cdc-d199-43d3-b94c-109078e85b87\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n  \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/dca31cdc-d199-43d3-b94c-109078e85b87\",\n  \"name\": \"dca31cdc-d199-43d3-b94c-109078e85b87\",\n  \"status\": \"Failed\",\n  \"startTime\": \"2026-03-04T11:44:31.085504793Z\",\n  \"endTime\": \"2026-03-04T11:49:48.850191595Z\",\n  \"error\": {\n    \"code\": \"InternalServerError\",\n    \"message\": \"insufficient public IP address quota: required 2, available 0\"\n  }\n}\n--------------------------------------------------------------------------------\n",
            err: <*exported.ResponseError | 0xc000e9a120>{
                ErrorCode: "InternalServerError",
                StatusCode: 200,
                RawResponse: {
                    Status: "200 OK",
                    StatusCode: 200,
                    Proto: "HTTP/2.0",
                    ProtoMajor: 2,
                    ProtoMinor: 0,
                    Header: {
                        "Content-Length": ["499"],
                        "X-Ms-Ratelimit-Remaining-Subscription-Global-Reads": ["3749"],
                        "X-Ms-Correlation-Request-Id": [
                            "f84a4c0c-f1d0-4ea2-9df8-e7ed7e57b882",
                        ],
                        "X-Ms-Routing-Request-Id": [
                            "CANADACENTRAL:20260304T114953Z:f84a4c0c-f1d0-4ea2-9df8-e7ed7e57b882",
                        ],
                        "Date": [
                            "Wed, 04 Mar 2026 11:49:52 GMT",
                        ],
                        "Pragma": ["no-cache"],
                        "Content-Type": ["application/json"],
                        "X-Ms-Operation-Identifier": [
                            "appId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,tenantId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,objectId=0e96dbb7-17a2-4393-b2d3-d84524e39c32/canadacentral/55274095-8948-4add-bada-7f6aff2a7024",
                        ],
                        "X-Content-Type-Options": ["nosniff"],
                        "X-Cache": ["CONFIG_NOCACHE"],
                        "Cache-Control": ["no-cache"],
                        "Expires": ["-1"],
           ...

Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.

Learn more here: https://onsi.github.io/gomega/#adjusting-output

occurred
fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:93]: Unexpected error:
    <*fmt.wrapError | 0xc000a802a0>: 
    failed to create HCP cluster ea-cluster: failed waiting for cluster="ea-cluster" in resourcegroup="external-auth-cluster-66kqnv" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/dca31cdc-d199-43d3-b94c-109078e85b87
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/dca31cdc-d199-43d3-b94c-109078e85b87",
      "name": "dca31cdc-d199-43d3-b94c-109078e85b87",
      "status": "Failed",
      "startTime": "2026-03-04T11:44:31.085504793Z",
      "endTime": "2026-03-04T11:49:48.850191595Z",
      "error": {
        "code": "InternalServerError",
        "message": "insufficient public IP address quota: required 2, available 0"
      }
    }
    --------------------------------------------------------------------------------
    
    {
        msg: "failed to create HCP cluster ea-cluster: failed waiting for cluster=\"ea-cluster\" in resourcegroup=\"external-auth-cluster-66kqnv\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/dca31cdc-d199-43d3-b94c-109078e85b87\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n  \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/dca31cdc-d199-43d3-b94c-109078e85b87\",\n  \"name\": \"dca31cdc-d199-43d3-b94c-109078e85b87\",\n  \"status\": \"Failed\",\n  \"startTime\": \"2026-03-04T11:44:31.085504793Z\",\n  \"endTime\": \"2026-03-04T11:49:48.850191595Z\",\n  \"error\": {\n    \"code\": \"InternalServerError\",\n    \"message\": \"insufficient public IP address quota: required 2, available 0\"\n  }\n}\n--------------------------------------------------------------------------------\n",
        err: <*fmt.wrapError | 0xc000a80280>{
            msg: "failed waiting for cluster=\"ea-cluster\" in resourcegroup=\"external-auth-cluster-66kqnv\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/dca31cdc-d199-43d3-b94c-109078e85b87\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n  \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/dca31cdc-d199-43d3-b94c-109078e85b87\",\n  \"name\": \"dca31cdc-d199-43d3-b94c-109078e85b87\",\n  \"status\": \"Failed\",\n  \"startTime\": \"2026-03-04T11:44:31.085504793Z\",\n  \"endTime\": \"2026-03-04T11:49:48.850191595Z\",\n  \"error\": {\n    \"code\": \"InternalServerError\",\n    \"message\": \"insufficient public IP address quota: required 2, available 0\"\n  }\n}\n--------------------------------------------------------------------------------\n",
            err: <*exported.ResponseError | 0xc000e9a120>{
                ErrorCode: "InternalServerError",
                StatusCode: 200,
                RawResponse: {
                    Status: "200 OK",
                    StatusCode: 200,
                    Proto: "HTTP/2.0",
                    ProtoMajor: 2,
                    ProtoMinor: 0,
                    Header: {
                        "Content-Length": ["499"],
                        "X-Ms-Ratelimit-Remaining-Subscription-Global-Reads": ["3749"],
                        "X-Ms-Correlation-Request-Id": [
                            "f84a4c0c-f1d0-4ea2-9df8-e7ed7e57b882",
                        ],
                        "X-Ms-Routing-Request-Id": [
                            "CANADACENTRAL:20260304T114953Z:f84a4c0c-f1d0-4ea2-9df8-e7ed7e57b882",
                        ],
                        "Date": [
                            "Wed, 04 Mar 2026 11:49:52 GMT",
                        ],
                        "Pragma": ["no-cache"],
                        "Content-Type": ["application/json"],
                        "X-Ms-Operation-Identifier": [
                            "appId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,tenantId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,objectId=0e96dbb7-17a2-4393-b2d3-d84524e39c32/canadacentral/55274095-8948-4add-bada-7f6aff2a7024",
                        ],
                        "X-Content-Type-Options": ["nosniff"],
                        "X-Cache": ["CONFIG_NOCACHE"],
                        "Cache-Control": ["no-cache"],
                        "Expires": ["-1"],
           ...

Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.

Learn more here: https://onsi.github.io/gomega/#adjusting-output

occurred

CreateNodePoolFromParam timed out after <minutes> minutes; context deadline exceeded

11:18:47 UTC · failed at e2e · test Customer should be able to create an HCP cluster and custom node pool osDisk size · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_nodepool_osdisk.go:94]: Unexpected error:
    <*fmt.wrapErrors | 0xc001079320>: 
    failed to create NodePool nodepool-128GiB, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool nodepool-128GiB in resource group clusternp128-59thg6, error: failed waiting for nodepool="nodepool-128GiB" for cluster "hcp-cluster-np-128" in resourcegroup="clusternp128-59thg6" to finish creating: context deadline exceeded
    {
        msg: "failed to create NodePool nodepool-128GiB, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool nodepool-128GiB in resource group clusternp128-59thg6, error: failed waiting for nodepool=\"nodepool-128GiB\" for cluster \"hcp-cluster-np-128\" in resourcegroup=\"clusternp128-59thg6\" to finish creating: context deadline exceeded",
        errs: [
            <*errors.errorString | 0xc0006a1820>{
                s: "timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool nodepool-128GiB in resource group clusternp128-59thg6",
            },
            <*fmt.wrapError | 0xc00079f180>{
                msg: "failed waiting for nodepool=\"nodepool-128GiB\" for cluster \"hcp-cluster-np-128\" in resourcegroup=\"clusternp128-59thg6\" to finish creating: context deadline exceeded",
                err: <context.deadlineExceededError>{},
            },
        ],
    }
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_nodepool_osdisk.go:94]: Unexpected error:
    <*fmt.wrapErrors | 0xc001079320>: 
    failed to create NodePool nodepool-128GiB, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool nodepool-128GiB in resource group clusternp128-59thg6, error: failed waiting for nodepool="nodepool-128GiB" for cluster "hcp-cluster-np-128" in resourcegroup="clusternp128-59thg6" to finish creating: context deadline exceeded
    {
        msg: "failed to create NodePool nodepool-128GiB, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool nodepool-128GiB in resource group clusternp128-59thg6, error: failed waiting for nodepool=\"nodepool-128GiB\" for cluster \"hcp-cluster-np-128\" in resourcegroup=\"clusternp128-59thg6\" to finish creating: context deadline exceeded",
        errs: [
            <*errors.errorString | 0xc0006a1820>{
                s: "timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool nodepool-128GiB in resource group clusternp128-59thg6",
            },
            <*fmt.wrapError | 0xc00079f180>{
                msg: "failed waiting for nodepool=\"nodepool-128GiB\" for cluster \"hcp-cluster-np-128\" in resourcegroup=\"clusternp128-59thg6\" to finish creating: context deadline exceeded",
                err: <context.deadlineExceededError>{},
            },
        ],
    }
occurred

CreateNodePoolFromParam timed out after <minutes> minutes; context deadline exceeded

11:18:47 UTC · failed at e2e · test Customer should be able to create a cluster with default autoscaling and a nodepool with autoscaling enabled · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:102]: Unexpected error:
    <*fmt.wrapErrors | 0xc000b345d0>: 
    failed to create NodePool autoscale-np, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool autoscale-np in resource group np-autoscaling-8clfgs, error: failed waiting for nodepool="autoscale-np" for cluster "np-autoscale-cluster" in resourcegroup="np-autoscaling-8clfgs" to finish creating: context deadline exceeded
    {
        msg: "failed to create NodePool autoscale-np, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool autoscale-np in resource group np-autoscaling-8clfgs, error: failed waiting for nodepool=\"autoscale-np\" for cluster \"np-autoscale-cluster\" in resourcegroup=\"np-autoscaling-8clfgs\" to finish creating: context deadline exceeded",
        errs: [
            <*errors.errorString | 0xc000b2d2e0>{
                s: "timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool autoscale-np in resource group np-autoscaling-8clfgs",
            },
            <*fmt.wrapError | 0xc0008b49c0>{
                msg: "failed waiting for nodepool=\"autoscale-np\" for cluster \"np-autoscale-cluster\" in resourcegroup=\"np-autoscaling-8clfgs\" to finish creating: context deadline exceeded",
                err: <context.deadlineExceededError>{},
            },
        ],
    }
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:102]: Unexpected error:
    <*fmt.wrapErrors | 0xc000b345d0>: 
    failed to create NodePool autoscale-np, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool autoscale-np in resource group np-autoscaling-8clfgs, error: failed waiting for nodepool="autoscale-np" for cluster "np-autoscale-cluster" in resourcegroup="np-autoscaling-8clfgs" to finish creating: context deadline exceeded
    {
        msg: "failed to create NodePool autoscale-np, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool autoscale-np in resource group np-autoscaling-8clfgs, error: failed waiting for nodepool=\"autoscale-np\" for cluster \"np-autoscale-cluster\" in resourcegroup=\"np-autoscaling-8clfgs\" to finish creating: context deadline exceeded",
        errs: [
            <*errors.errorString | 0xc000b2d2e0>{
                s: "timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool autoscale-np in resource group np-autoscaling-8clfgs",
            },
            <*fmt.wrapError | 0xc0008b49c0>{
                msg: "failed waiting for nodepool=\"autoscale-np\" for cluster \"np-autoscale-cluster\" in resourcegroup=\"np-autoscaling-8clfgs\" to finish creating: context deadline exceeded",
                err: <context.deadlineExceededError>{},
            },
        ],
    }
occurred

ERROR CODE: InternalServerError; detail message insufficient public IP address quota: required <count>, available <count>; provider Microsoft.RedHatOpenShift

11:18:47 UTC · failed at e2e · test Customer should be able to create an HCP cluster with back-level version 4.19 · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:147]: Unexpected error:
    <*fmt.wrapError | 0xc0011e4a80>: 
    failed waiting for cluster="cluster-ver-4-19" in resourcegroup="rg-cluster-back-version-8xpmw9" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0407f0ef-72df-4f2b-93de-b19410d40d6d
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0407f0ef-72df-4f2b-93de-b19410d40d6d",
      "name": "0407f0ef-72df-4f2b-93de-b19410d40d6d",
      "status": "Failed",
      "startTime": "2026-03-04T11:42:21.423884671Z",
      "endTime": "2026-03-04T11:45:00.069078308Z",
      "error": {
        "code": "InternalServerError",
        "message": "insufficient public IP address quota: required 2, available 1"
      }
    }
    --------------------------------------------------------------------------------
    
    {
        msg: "failed waiting for cluster=\"cluster-ver-4-19\" in resourcegroup=\"rg-cluster-back-version-8xpmw9\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0407f0ef-72df-4f2b-93de-b19410d40d6d\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n  \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0407f0ef-72df-4f2b-93de-b19410d40d6d\",\n  \"name\": \"0407f0ef-72df-4f2b-93de-b19410d40d6d\",\n  \"status\": \"Failed\",\n  \"startTime\": \"2026-03-04T11:42:21.423884671Z\",\n  \"endTime\": \"2026-03-04T11:45:00.069078308Z\",\n  \"error\": {\n    \"code\": \"InternalServerError\",\n    \"message\": \"insufficient public IP address quota: required 2, available 1\"\n  }\n}\n--------------------------------------------------------------------------------\n",
        err: <*exported.ResponseError | 0xc000ed1320>{
            ErrorCode: "InternalServerError",
            StatusCode: 200,
            RawResponse: {
                Status: "200 OK",
                StatusCode: 200,
                Proto: "HTTP/2.0",
                ProtoMajor: 2,
                ProtoMinor: 0,
                Header: {
                    "Pragma": ["no-cache"],
                    "X-Ms-Request-Id": [
                        "1f116b65-75f8-4195-a4e7-62fd014a5842",
                    ],
                    "X-Ms-Ratelimit-Remaining-Subscription-Global-Reads": ["3749"],
                    "X-Ms-Correlation-Request-Id": [
                        "6fb9a3eb-33d5-49fd-9ee8-9145d49e0a4e",
                    ],
                    "X-Content-Type-Options": ["nosniff"],
                    "X-Ms-Operation-Identifier": [
                        "appId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,tenantId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,objectId=0e96dbb7-17a2-4393-b2d3-d84524e39c32/canadacentral/f0a897d5-d1ee-4de5-9c39-3e46359eafd8",
                    ],
                    "X-Ms-Ratelimit-Remaining-Subscription-Reads": ["249"],
                    "X-Ms-Routing-Request-Id": [
                        "CANADACENTRAL:20260304T114506Z:6fb9a3eb-33d5-49fd-9ee8-9145d49e0a4e",
                    ],
                    "X-Cache": ["CONFIG_NOCACHE"],
                    "Cache-Control": ["no-cache"],
                    "Content-Type": ["application/json"],
                    "Strict-Transport-Security": [
                        "max-age=31536000; includeSubDomains",
                    ],
                    "X-Msedge-Ref": [
                        "Ref A: 38725E90972A49A387B79ED183CDDF6F Ref B: CHI261140501034 Ref C: 2026-03-04T11:45:06Z",
                    ],
                    "Content-Length": ["499"],
                    "Expires": ["-1"],
                    "X-Envoy-Upstream-Service-Time": ["9"],
                    "Date": [
                        "Wed, 04 Mar 2026 11:45:06 GMT",
                    ],
                },
                Body: <*exported.nopClosingBytesReader | 0xc0011e4960>{
                    s: [123, 10, 32, 32, 32, 32, 34, 105, 100, 34, 58, 32, 34, 47, 115, 117, 98, 115, 99, 114, 105, 112, 116, 105, 111, 110, 115, 47, 52, 48, 51, 100, 57, 100, 101, 57, 45, 49, 51, 50, 98, 45, 52, 57, 55, 52, 45, 57, 52, 97, 53, 45, 53, 98, 55, 56, 98, 100, 102, 97, 49, 57, 49, 101, 47, 112, 114, 111, 118, 105, 100, 101, 114, 115, 47, 77, 105, 99, 114, 111, 115, 111, 102, 116, 46, 82, 101, 100, 72, 97, 116, 79, 112, 101, 110, 83, 104, 105, 102, 116, 47, 108, 111, 99, 97, 116, 105, 111, 110, 115, 47, 117, 107, 115, 111, 117, 116, 104, 47, 104, 99, 112, 79, 112, 101, 114, 97, 116, 105, 111, 110, 83, 116, 97, 116, 117, 115, 101, 115, 47, 48, 52, 48, 55, 102, 48, 101, ...

Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.

Learn more here: https://onsi.github.io/gomega/#adjusting-output

occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:147]: Unexpected error:
    <*fmt.wrapError | 0xc0011e4a80>: 
    failed waiting for cluster="cluster-ver-4-19" in resourcegroup="rg-cluster-back-version-8xpmw9" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0407f0ef-72df-4f2b-93de-b19410d40d6d
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0407f0ef-72df-4f2b-93de-b19410d40d6d",
      "name": "0407f0ef-72df-4f2b-93de-b19410d40d6d",
      "status": "Failed",
      "startTime": "2026-03-04T11:42:21.423884671Z",
      "endTime": "2026-03-04T11:45:00.069078308Z",
      "error": {
        "code": "InternalServerError",
        "message": "insufficient public IP address quota: required 2, available 1"
      }
    }
    --------------------------------------------------------------------------------
    
    {
        msg: "failed waiting for cluster=\"cluster-ver-4-19\" in resourcegroup=\"rg-cluster-back-version-8xpmw9\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0407f0ef-72df-4f2b-93de-b19410d40d6d\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n  \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0407f0ef-72df-4f2b-93de-b19410d40d6d\",\n  \"name\": \"0407f0ef-72df-4f2b-93de-b19410d40d6d\",\n  \"status\": \"Failed\",\n  \"startTime\": \"2026-03-04T11:42:21.423884671Z\",\n  \"endTime\": \"2026-03-04T11:45:00.069078308Z\",\n  \"error\": {\n    \"code\": \"InternalServerError\",\n    \"message\": \"insufficient public IP address quota: required 2, available 1\"\n  }\n}\n--------------------------------------------------------------------------------\n",
        err: <*exported.ResponseError | 0xc000ed1320>{
            ErrorCode: "InternalServerError",
            StatusCode: 200,
            RawResponse: {
                Status: "200 OK",
                StatusCode: 200,
                Proto: "HTTP/2.0",
                ProtoMajor: 2,
                ProtoMinor: 0,
                Header: {
                    "Pragma": ["no-cache"],
                    "X-Ms-Request-Id": [
                        "1f116b65-75f8-4195-a4e7-62fd014a5842",
                    ],
                    "X-Ms-Ratelimit-Remaining-Subscription-Global-Reads": ["3749"],
                    "X-Ms-Correlation-Request-Id": [
                        "6fb9a3eb-33d5-49fd-9ee8-9145d49e0a4e",
                    ],
                    "X-Content-Type-Options": ["nosniff"],
                    "X-Ms-Operation-Identifier": [
                        "appId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,tenantId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,objectId=0e96dbb7-17a2-4393-b2d3-d84524e39c32/canadacentral/f0a897d5-d1ee-4de5-9c39-3e46359eafd8",
                    ],
                    "X-Ms-Ratelimit-Remaining-Subscription-Reads": ["249"],
                    "X-Ms-Routing-Request-Id": [
                        "CANADACENTRAL:20260304T114506Z:6fb9a3eb-33d5-49fd-9ee8-9145d49e0a4e",
                    ],
                    "X-Cache": ["CONFIG_NOCACHE"],
                    "Cache-Control": ["no-cache"],
                    "Content-Type": ["application/json"],
                    "Strict-Transport-Security": [
                        "max-age=31536000; includeSubDomains",
                    ],
                    "X-Msedge-Ref": [
                        "Ref A: 38725E90972A49A387B79ED183CDDF6F Ref B: CHI261140501034 Ref C: 2026-03-04T11:45:06Z",
                    ],
                    "Content-Length": ["499"],
                    "Expires": ["-1"],
                    "X-Envoy-Upstream-Service-Time": ["9"],
                    "Date": [
                        "Wed, 04 Mar 2026 11:45:06 GMT",
                    ],
                },
                Body: <*exported.nopClosingBytesReader | 0xc0011e4960>{
                    s: [123, 10, 32, 32, 32, 32, 34, 105, 100, 34, 58, 32, 34, 47, 115, 117, 98, 115, 99, 114, 105, 112, 116, 105, 111, 110, 115, 47, 52, 48, 51, 100, 57, 100, 101, 57, 45, 49, 51, 50, 98, 45, 52, 57, 55, 52, 45, 57, 52, 97, 53, 45, 53, 98, 55, 56, 98, 100, 102, 97, 49, 57, 49, 101, 47, 112, 114, 111, 118, 105, 100, 101, 114, 115, 47, 77, 105, 99, 114, 111, 115, 111, 102, 116, 46, 82, 101, 100, 72, 97, 116, 79, 112, 101, 110, 83, 104, 105, 102, 116, 47, 108, 111, 99, 97, 116, 105, 111, 110, 115, 47, 117, 107, 115, 111, 117, 116, 104, 47, 104, 99, 112, 79, 112, 101, 114, 97, 116, 105, 111, 110, 83, 116, 97, 116, 117, 115, 101, 115, 47, 48, 52, 48, 55, 102, 48, 101, ...

Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.

Learn more here: https://onsi.github.io/gomega/#adjusting-output

occurred

ERROR CODE: InternalServerError; detail message insufficient public IP address quota: required <count>, available <count>; provider Microsoft.RedHatOpenShift

11:18:47 UTC · failed at e2e · test Customer should be able to create node pools with ARM64-based VMs · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/arm64_nodepool.go:90]: Unexpected error:
    <*fmt.wrapError | 0xc0008963e0>: 
    failed to create HCP cluster arm64-vm-hcp-cluster: failed waiting for cluster="arm64-vm-hcp-cluster" in resourcegroup="arm64-vm-cluster-4ljw7f" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/8e2e6926-ad32-404f-9710-045e248d82cd
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/8e2e6926-ad32-404f-9710-045e248d82cd",
      "name": "8e2e6926-ad32-404f-9710-045e248d82cd",
      "status": "Failed",
      "startTime": "2026-03-04T11:42:34.945511327Z",
      "endTime": "2026-03-04T11:49:48.715948579Z",
      "error": {
        "code": "InternalServerError",
        "message": "insufficient public IP address quota: required 2, available 1"
      }
    }
    --------------------------------------------------------------------------------
    
    {
        msg: "failed to create HCP cluster arm64-vm-hcp-cluster: failed waiting for cluster=\"arm64-vm-hcp-cluster\" in resourcegroup=\"arm64-vm-cluster-4ljw7f\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/8e2e6926-ad32-404f-9710-045e248d82cd\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n  \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/8e2e6926-ad32-404f-9710-045e248d82cd\",\n  \"name\": \"8e2e6926-ad32-404f-9710-045e248d82cd\",\n  \"status\": \"Failed\",\n  \"startTime\": \"2026-03-04T11:42:34.945511327Z\",\n  \"endTime\": \"2026-03-04T11:49:48.715948579Z\",\n  \"error\": {\n    \"code\": \"InternalServerError\",\n    \"message\": \"insufficient public IP address quota: required 2, available 1\"\n  }\n}\n--------------------------------------------------------------------------------\n",
        err: <*fmt.wrapError | 0xc0008963c0>{
            msg: "failed waiting for cluster=\"arm64-vm-hcp-cluster\" in resourcegroup=\"arm64-vm-cluster-4ljw7f\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/8e2e6926-ad32-404f-9710-045e248d82cd\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n  \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/8e2e6926-ad32-404f-9710-045e248d82cd\",\n  \"name\": \"8e2e6926-ad32-404f-9710-045e248d82cd\",\n  \"status\": \"Failed\",\n  \"startTime\": \"2026-03-04T11:42:34.945511327Z\",\n  \"endTime\": \"2026-03-04T11:49:48.715948579Z\",\n  \"error\": {\n    \"code\": \"InternalServerError\",\n    \"message\": \"insufficient public IP address quota: required 2, available 1\"\n  }\n}\n--------------------------------------------------------------------------------\n",
            err: <*exported.ResponseError | 0xc00128c5a0>{
                ErrorCode: "InternalServerError",
                StatusCode: 200,
                RawResponse: {
                    Status: "200 OK",
                    StatusCode: 200,
                    Proto: "HTTP/2.0",
                    ProtoMajor: 2,
                    ProtoMinor: 0,
                    Header: {
                        "Content-Length": ["499"],
                        "X-Ms-Correlation-Request-Id": [
                            "8649fcef-a89d-4ebe-a831-d5aa547b7859",
                        ],
                        "X-Ms-Routing-Request-Id": [
                            "CANADACENTRAL:20260304T114951Z:8649fcef-a89d-4ebe-a831-d5aa547b7859",
                        ],
                        "Strict-Transport-Security": [
                            "max-age=31536000; includeSubDomains",
                        ],
                        "Date": [
                            "Wed, 04 Mar 2026 11:49:50 GMT",
                        ],
                        "Content-Type": ["application/json"],
                        "X-Ms-Operation-Identifier": [
                            "appId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,tenantId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,objectId=0e96dbb7-17a2-4393-b2d3-d84524e39c32/canadacentral/43fae7b6-ce0d-421e-b423-059eca2efbf5",
                        ],
                        "X-Ms-Ratelimit-Remaining-Subscription-Reads": ["249"],
                        "X-Cache": ["CONFIG_NOCACHE"],
                        "Cache-Control": ["no-cache"],
    ...

Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.

Learn more here: https://onsi.github.io/gomega/#adjusting-output

occurred
fail [github.com/Azure/ARO-HCP/test/e2e/arm64_nodepool.go:90]: Unexpected error:
    <*fmt.wrapError | 0xc0008963e0>: 
    failed to create HCP cluster arm64-vm-hcp-cluster: failed waiting for cluster="arm64-vm-hcp-cluster" in resourcegroup="arm64-vm-cluster-4ljw7f" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/8e2e6926-ad32-404f-9710-045e248d82cd
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/8e2e6926-ad32-404f-9710-045e248d82cd",
      "name": "8e2e6926-ad32-404f-9710-045e248d82cd",
      "status": "Failed",
      "startTime": "2026-03-04T11:42:34.945511327Z",
      "endTime": "2026-03-04T11:49:48.715948579Z",
      "error": {
        "code": "InternalServerError",
        "message": "insufficient public IP address quota: required 2, available 1"
      }
    }
    --------------------------------------------------------------------------------
    
    {
        msg: "failed to create HCP cluster arm64-vm-hcp-cluster: failed waiting for cluster=\"arm64-vm-hcp-cluster\" in resourcegroup=\"arm64-vm-cluster-4ljw7f\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/8e2e6926-ad32-404f-9710-045e248d82cd\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n  \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/8e2e6926-ad32-404f-9710-045e248d82cd\",\n  \"name\": \"8e2e6926-ad32-404f-9710-045e248d82cd\",\n  \"status\": \"Failed\",\n  \"startTime\": \"2026-03-04T11:42:34.945511327Z\",\n  \"endTime\": \"2026-03-04T11:49:48.715948579Z\",\n  \"error\": {\n    \"code\": \"InternalServerError\",\n    \"message\": \"insufficient public IP address quota: required 2, available 1\"\n  }\n}\n--------------------------------------------------------------------------------\n",
        err: <*fmt.wrapError | 0xc0008963c0>{
            msg: "failed waiting for cluster=\"arm64-vm-hcp-cluster\" in resourcegroup=\"arm64-vm-cluster-4ljw7f\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/8e2e6926-ad32-404f-9710-045e248d82cd\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n  \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/8e2e6926-ad32-404f-9710-045e248d82cd\",\n  \"name\": \"8e2e6926-ad32-404f-9710-045e248d82cd\",\n  \"status\": \"Failed\",\n  \"startTime\": \"2026-03-04T11:42:34.945511327Z\",\n  \"endTime\": \"2026-03-04T11:49:48.715948579Z\",\n  \"error\": {\n    \"code\": \"InternalServerError\",\n    \"message\": \"insufficient public IP address quota: required 2, available 1\"\n  }\n}\n--------------------------------------------------------------------------------\n",
            err: <*exported.ResponseError | 0xc00128c5a0>{
                ErrorCode: "InternalServerError",
                StatusCode: 200,
                RawResponse: {
                    Status: "200 OK",
                    StatusCode: 200,
                    Proto: "HTTP/2.0",
                    ProtoMajor: 2,
                    ProtoMinor: 0,
                    Header: {
                        "Content-Length": ["499"],
                        "X-Ms-Correlation-Request-Id": [
                            "8649fcef-a89d-4ebe-a831-d5aa547b7859",
                        ],
                        "X-Ms-Routing-Request-Id": [
                            "CANADACENTRAL:20260304T114951Z:8649fcef-a89d-4ebe-a831-d5aa547b7859",
                        ],
                        "Strict-Transport-Security": [
                            "max-age=31536000; includeSubDomains",
                        ],
                        "Date": [
                            "Wed, 04 Mar 2026 11:49:50 GMT",
                        ],
                        "Content-Type": ["application/json"],
                        "X-Ms-Operation-Identifier": [
                            "appId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,tenantId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,objectId=0e96dbb7-17a2-4393-b2d3-d84524e39c32/canadacentral/43fae7b6-ce0d-421e-b423-059eca2efbf5",
                        ],
                        "X-Ms-Ratelimit-Remaining-Subscription-Reads": ["249"],
                        "X-Cache": ["CONFIG_NOCACHE"],
                        "Cache-Control": ["no-cache"],
    ...

Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.

Learn more here: https://onsi.github.io/gomega/#adjusting-output

occurred

11:18:47 UTC · failed at e2e · test Authorized CIDRs Connectivity should allow API access only from authorized VM IP · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:133]: Unexpected error:
    <*fmt.wrapError | 0xc001050900>: 
    failed to create HCP cluster cidr-connectivity-test: failed waiting for cluster="cidr-connectivity-test" in resourcegroup="e2e-cidr-connectivity-42hshj" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/6ee8ec73-61b8-4344-ad00-bba98e2702a7
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/6ee8ec73-61b8-4344-ad00-bba98e2702a7",
      "name": "6ee8ec73-61b8-4344-ad00-bba98e2702a7",
      "status": "Failed",
      "startTime": "2026-03-04T11:27:28.76644337Z",
      "endTime": "2026-03-04T11:33:55.189840009Z",
      "error": {
        "code": "InternalServerError",
        "message": "PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-42hshj--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi6tet62ut5t7l4015hb57ec6maei-3\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"PublicIPCountLimitReached\",\n    \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n    \"details\": []\n  }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-42hshj--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi6tet62ut5t7l4015hb57ec6maei-4\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"PublicIPCountLimitReached\",\n    \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n    \"details\": []\n  }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-42hshj--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi6tet62ut5t7l4015hb57ec6maei-5\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"PublicIPCountLimitReached\",\n    \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n    \"details\": []\n  }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-42hshj--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi6tet62ut5t7l4015hb57ec6maei-6\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"PublicIPCountLimitReached\",\n    \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n    \"details\": []\n  }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-42hshj--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi6tet62ut5t7l4015hb57ec6maei-7\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"PublicIPCountLimitReached\",\n    \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n    \"details\": []\n  }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-42hshj--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi6tet62ut5t7l4015hb57ec6maei-8\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"PublicIPCountLimitReached\",\n    \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n    \"details\": []\n  }\n}\n--------------------------------------------------------------------------------\n"
      }
    }
    --------------------------------------------------------------------------------
    
    {
        msg: "failed to create HCP cluster cidr-connectivity-test: failed waiting for cluster=\"cidr-connectivity-test\" in resourcegroup=\"e2e-cidr-connectivity-42hshj\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/6ee8ec73-61b8-4344-ad00-bba98e2702a7\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n  \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/6ee8ec73-61b8-4344-ad00-bba98e2702a7\",\n  \"name\": \"6ee8ec73-61b8-4344-ad00-bba98e2702a7\",\n  \"status\": \"Failed\",\n  \"startTime\": \"2026-03-04T11:27:28.76644337Z\",\n  \"endTime\": \"2026-03-04T11:33:55.189840009Z\",\n  \"error\": {\n    \"code\": \"InternalServerError\",\n    \"message\": \"PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-42hshj--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi6tet62ut5t7l4015hb57ec6maei-3\\n--------------------------------------------------------------------------------\\nRESPONSE 400: 400 Bad Request\\nERROR CODE: PublicIPCountLimitReached\\n--------------------------------------------------------------------------------\\n{\\n  \\\"error\\\": {\\n    \\\"code\\\": \\\"PublicIPCountLimitReached\\\",\\n    \\\"message\\\": \\\"Cannot create more than 300 public IP addresses for this subscription in this region.\\\",\\n    \\\"details\\\": []\\n  }\\n}\\n--------------------------------------------------------------------------------\\n\\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-42hshj--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi6tet62ut5t7l4015hb57ec6maei-4\\n--------------------------------------------------------------------------------\\nRESPONSE 400: 400 Bad Request\\nERROR CODE: PublicIPCountLimitReached\\n--------------------------------------------------------------------------------\\n{\\n  \\\"error\\\": {\\n    \\\"code\\\": \\\"PublicIPCountLimitReached\\\",\\n    \\\"message\\\": \\\"Cannot create more than 300 public IP addresses for this subscription in this region.\\\",\\n    \\\"details\\\": []\\n  }\\n}\\n--------------------------------------------------------------------------------\\n\\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-42hshj--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi6tet62ut5t7l4015hb57ec6maei-5\\n--------------------------------------------------------------------------------\\nRESPONSE 400: 400 Bad Request\\nERROR CODE: PublicIPCountLimitReached\\n--------------------------------------------------------------------------------\\n{\\n  \\\"error\\\": {\\n    \\\"code\\\": \\\"PublicIPCountLimitReached\\\",\\n    \\\"message\\\": \\\"Cannot create more than 300 public IP addresses for this subscription in this region.\\\",\\n    \\\"details\\\": []\\n  }\\n}\\n--------------------------------------------------------------------------------\\n\\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-42hshj--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi6tet62ut5t7l4015hb57ec6maei-6\\n--------------------------------------------------------------------------------\\nRESPONSE 400: 400 Bad Request\\nERROR CODE: PublicIPCountLimitReached\\n--------------------------------------------------------------------------------\\n{\\n  \\\"error\\\": {\\n    \\\"code\\\": \\\"PublicIPCountLimitReached\\\",\\n    \\\"message\\\": \\\"Cannot create more than 300 public IP addresses f...

Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.

Learn more here: https://onsi.github.io/gomega/#adjusting-output

occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:133]: Unexpected error:
    <*fmt.wrapError | 0xc001050900>: 
    failed to create HCP cluster cidr-connectivity-test: failed waiting for cluster="cidr-connectivity-test" in resourcegroup="e2e-cidr-connectivity-42hshj" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/6ee8ec73-61b8-4344-ad00-bba98e2702a7
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/6ee8ec73-61b8-4344-ad00-bba98e2702a7",
      "name": "6ee8ec73-61b8-4344-ad00-bba98e2702a7",
      "status": "Failed",
      "startTime": "2026-03-04T11:27:28.76644337Z",
      "endTime": "2026-03-04T11:33:55.189840009Z",
      "error": {
        "code": "InternalServerError",
        "message": "PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-42hshj--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi6tet62ut5t7l4015hb57ec6maei-3\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"PublicIPCountLimitReached\",\n    \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n    \"details\": []\n  }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-42hshj--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi6tet62ut5t7l4015hb57ec6maei-4\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"PublicIPCountLimitReached\",\n    \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n    \"details\": []\n  }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-42hshj--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi6tet62ut5t7l4015hb57ec6maei-5\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"PublicIPCountLimitReached\",\n    \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n    \"details\": []\n  }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-42hshj--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi6tet62ut5t7l4015hb57ec6maei-6\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"PublicIPCountLimitReached\",\n    \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n    \"details\": []\n  }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-42hshj--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi6tet62ut5t7l4015hb57ec6maei-7\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"PublicIPCountLimitReached\",\n    \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n    \"details\": []\n  }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-42hshj--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi6tet62ut5t7l4015hb57ec6maei-8\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"PublicIPCountLimitReached\",\n    \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n    \"details\": []\n  }\n}\n--------------------------------------------------------------------------------\n"
      }
    }
    --------------------------------------------------------------------------------
    
    {
        msg: "failed to create HCP cluster cidr-connectivity-test: failed waiting for cluster=\"cidr-connectivity-test\" in resourcegroup=\"e2e-cidr-connectivity-42hshj\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/6ee8ec73-61b8-4344-ad00-bba98e2702a7\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n  \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/6ee8ec73-61b8-4344-ad00-bba98e2702a7\",\n  \"name\": \"6ee8ec73-61b8-4344-ad00-bba98e2702a7\",\n  \"status\": \"Failed\",\n  \"startTime\": \"2026-03-04T11:27:28.76644337Z\",\n  \"endTime\": \"2026-03-04T11:33:55.189840009Z\",\n  \"error\": {\n    \"code\": \"InternalServerError\",\n    \"message\": \"PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-42hshj--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi6tet62ut5t7l4015hb57ec6maei-3\\n--------------------------------------------------------------------------------\\nRESPONSE 400: 400 Bad Request\\nERROR CODE: PublicIPCountLimitReached\\n--------------------------------------------------------------------------------\\n{\\n  \\\"error\\\": {\\n    \\\"code\\\": \\\"PublicIPCountLimitReached\\\",\\n    \\\"message\\\": \\\"Cannot create more than 300 public IP addresses for this subscription in this region.\\\",\\n    \\\"details\\\": []\\n  }\\n}\\n--------------------------------------------------------------------------------\\n\\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-42hshj--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi6tet62ut5t7l4015hb57ec6maei-4\\n--------------------------------------------------------------------------------\\nRESPONSE 400: 400 Bad Request\\nERROR CODE: PublicIPCountLimitReached\\n--------------------------------------------------------------------------------\\n{\\n  \\\"error\\\": {\\n    \\\"code\\\": \\\"PublicIPCountLimitReached\\\",\\n    \\\"message\\\": \\\"Cannot create more than 300 public IP addresses for this subscription in this region.\\\",\\n    \\\"details\\\": []\\n  }\\n}\\n--------------------------------------------------------------------------------\\n\\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-42hshj--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi6tet62ut5t7l4015hb57ec6maei-5\\n--------------------------------------------------------------------------------\\nRESPONSE 400: 400 Bad Request\\nERROR CODE: PublicIPCountLimitReached\\n--------------------------------------------------------------------------------\\n{\\n  \\\"error\\\": {\\n    \\\"code\\\": \\\"PublicIPCountLimitReached\\\",\\n    \\\"message\\\": \\\"Cannot create more than 300 public IP addresses for this subscription in this region.\\\",\\n    \\\"details\\\": []\\n  }\\n}\\n--------------------------------------------------------------------------------\\n\\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-42hshj--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi6tet62ut5t7l4015hb57ec6maei-6\\n--------------------------------------------------------------------------------\\nRESPONSE 400: 400 Bad Request\\nERROR CODE: PublicIPCountLimitReached\\n--------------------------------------------------------------------------------\\n{\\n  \\\"error\\\": {\\n    \\\"code\\\": \\\"PublicIPCountLimitReached\\\",\\n    \\\"message\\\": \\\"Cannot create more than 300 public IP addresses f...

Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.

Learn more here: https://onsi.github.io/gomega/#adjusting-output

occurred

11:18:47 UTC · failed at e2e · test Customer should be able to create several HCP clusters in their customer resource group, but not in the same managed resource group · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:129]: Unexpected error:
    <*exported.ResponseError | 0xc001696270>: 
    GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0b35155f-1c87-4a27-b264-b5ffaf2df592
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0b35155f-1c87-4a27-b264-b5ffaf2df592",
      "name": "0b35155f-1c87-4a27-b264-b5ffaf2df592",
      "status": "Failed",
      "startTime": "2026-03-04T11:27:10.141510238Z",
      "endTime": "2026-03-04T11:33:01.507880583Z",
      "error": {
        "code": "InternalServerError",
        "message": "PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/customer-rg-42hl9p--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi3dcu806c95qg7cc9io5migfbgq6-0\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"PublicIPCountLimitReached\",\n    \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n    \"details\": []\n  }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/customer-rg-42hl9p--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi3dcu806c95qg7cc9io5migfbgq6-1\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"PublicIPCountLimitReached\",\n    \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n    \"details\": []\n  }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/customer-rg-42hl9p--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi3dcu806c95qg7cc9io5migfbgq6-2\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"PublicIPCountLimitReached\",\n    \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n    \"details\": []\n  }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/customer-rg-42hl9p--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi3dcu806c95qg7cc9io5migfbgq6-3\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"PublicIPCountLimitReached\",\n    \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n    \"details\": []\n  }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/customer-rg-42hl9p--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi3dcu806c95qg7cc9io5migfbgq6-4\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"PublicIPCountLimitReached\",\n    \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n    \"details\": []\n  }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/customer-rg-42hl9p--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi3dcu806c95qg7cc9io5migfbgq6-5\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"PublicIPCountLimitReached\",\n    \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n    \"details\": []\n  }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/customer-rg-42hl9p--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi3dcu806c95qg7cc9io5migfbgq6-6\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"PublicIPCountLimitReached\",\n    \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n    \"details\": []\n  }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/customer-rg-42hl9p--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi3dcu806c95qg7cc9io5migfbgq6-7\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"PublicIPCountLimitReached\",\n    \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n    \"details\": []\n  }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/customer-rg-42hl9p--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi3dcu806c95qg7cc9io5migfbgq6-8\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"PublicIPCountLimitReached\",\n    \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n    \"details\": []\n  }\n}\n--------------------------------------------------------------------------------\n"
      }
    }
    --------------------------------------------------------------------------------
    
    {
        ErrorCode: "InternalServerError",
        StatusCode: 200,
        RawResponse: {
            Status: "200 OK",
            StatusCode: 200,
            Proto: "HTTP/2.0",
            ProtoMajor: 2,
            ProtoMinor: 0,
            Header: {
                "Cache-Control": ["no-cache"],
                "X-Ms-Ratelimit-Remaining-Subscription-Reads": ["249"],
                "Content-Length": ["7006"],
                "X-Ms-Request-Id": [
                    "8f5597b5-16fa-4df6-bd73-65ab0a1bb005",
                ],
                "X-Envoy-Upstream-Service-Time": ["10"],
                "X-Ms-Ratelimit-Remaining-Subscription-Global-Reads": ["3749"],
                "X-Ms-Correlation-Request-Id": [
                    "bd0a002f-0f30-4cf8-91cd-38e2956ccb93",
                ],
                "Content-Type": ["application/json"],
                "X-Ms-Operation-Identifier": [
                    "appId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,tenantId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,objectId=0e96dbb7-17a2-4393-b2d3-d84524e39c32/canadacentral/eec24277-b57a-425a-a5e4-071fef365907",
                ],
                "X-Ms-Routing-Request-Id": [
                    "CANADACENTRAL:20260304T113304Z:bd0a002f-0f30-4cf8-91cd-38e2956ccb93",
                ],
                "Strict-Transport-Security": [
                    "max-age=31536000; includeSubDomains",
                ],
                "X-Content-Type-Options": ["nosniff"],
                "X-Msedge-Ref": [
                    "Ref A: 7BD958C08EC847A19E41507ED24FBCAF Ref B: CHI261140510034 Ref C: 2026-03-04T11:33:03Z",
                ],
                "Pragma": ["no-cache"],
                "Expires": ["-1"],
                "X-Cache": ["CONFIG_NOCACHE"],
                "Date": [
                    "Wed, 04 Mar 2026 11:33:03 GMT",
                ],
            },
            Body: <*exported.nopClosingBytesReader | 0xc000e12000>{
                s: [123, 10, 32, 32, 32, 32, 34, 105, 100, 34, 58, 32, 34, 47, 115, 117, 98, 115, 99, 114, 105, 112, 116, 105, 111, 110, 115, 47, 52, 48, 51, 100, 57, 100, 101, 57, 45, 49, 51, 50, 98, 45, 52, 57, 55, 52, 45, 57, 52, 97, 53, 45, 53, 98, 55, 56, 98, 100, 102, 97, 49, 57, 49, 101, 47, 112, 114, 111, 118, 105, 100, 101, 114, 115, 47, 77, 105, 99, 114, 111, 115, 111, 102, 116, 46, 82, 101, 100, 72, 97, 116, 79, 112, 101, 110, 83, 104, 105, 102, 116, 47, 108, 111, 99, 97, 116, 105, 111, 110, 115, 47, 117, 107, 115, 111, 117, 116, 104, 47, 104, 99, 112, 79, 112, 101, 114, 97, 116, 105, 111, 110, 83, 116, 97, 116, 117, 115, 101, 115, 47, 48, 98, 51, 53, 49, 53, 53, 102, 45, 49, 99, 56, 55, 45, 52, 97, 50, 55, 45, 98, 50, 54, 52, 45, 98, 53, 102, 102, 97, 102, 50, 100, 102, 53, 57, 50, 34, 44, 10, 32, 32, 32, 32, 34, 110, 97, 109, 101, 34, 58, 32, 34, 48, 98, 51, 53, 49, 53, 53, 102, 45, 49, 99, 56, 55, 45, 52, 97, 50, 55, 45, 98, 50, 54, 52, 45, 98, 53, 102, 102, 97, 102, 50, 100, 102, 53, 57, 50, 34, 44, 10, 32, 32, 32, 32, 34, 115, 116, 97, 116, 117, 115, 34, 58, 32, 34, 70, 97, 105, 108, 101, 100, 34, 44, 10, 32, 32, 32, 32, 34, 115, 116, 97, 114, 116, 84, 105, 109, 101, 34, 58, 32, 34, 50, 48, 50, 54, 45, 48, 51, 45, 48, 52, 84, 49, 49, 58, 50, 55, 58, 49, 48, 46, 49, 52, 49, 53, 49, 48, 50, 51, 56, 90, 34, 44, 10, 32, 32, 32, 32, 34, 101, 110, 100, 84, 105, 109, 101, 34, 58, 32, 34, 50, 48, 50, 54, 45, 48, 51, 45, 48, 52, 84, 49, 49, 58, 51, 51, 58, 48, 49, 46, 53, 48, 55, 56, 56, 48, 53, 56, 51, 90, 34, 44, 10, 32, 32, 32, 32, 34, 101, 114, 114, 111, 114, 34, 58, 32, 123, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 99, 111, 100, 101, 34, 58, 32, 34, 73, 110, 116, 101, 114, 110, 97, 108, 83, 101, 114, 118, 101, 114, 69, 114, 114, 111, 114, 34, 44, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 109, 101, 115, 115, 97, 103, 101, 34, 58, 32, 34, 80, 85, 84, 32, 104, 116, 116, 112, 115, 58, 47, 47, 109, 97, 110, 97, 103, 101, 109, 101, 110, 116, 46, 97, 122, 117, 114, 101, 46, 99, 111, 109, 47, 115, 117, 98, 115, 99, 114, 105, 11...

Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.

Learn more here: https://onsi.github.io/gomega/#adjusting-output

occurred
fail [github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:129]: Unexpected error:
    <*exported.ResponseError | 0xc001696270>: 
    GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0b35155f-1c87-4a27-b264-b5ffaf2df592
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0b35155f-1c87-4a27-b264-b5ffaf2df592",
      "name": "0b35155f-1c87-4a27-b264-b5ffaf2df592",
      "status": "Failed",
      "startTime": "2026-03-04T11:27:10.141510238Z",
      "endTime": "2026-03-04T11:33:01.507880583Z",
      "error": {
        "code": "InternalServerError",
        "message": "PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/customer-rg-42hl9p--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi3dcu806c95qg7cc9io5migfbgq6-0\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"PublicIPCountLimitReached\",\n    \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n    \"details\": []\n  }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/customer-rg-42hl9p--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi3dcu806c95qg7cc9io5migfbgq6-1\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"PublicIPCountLimitReached\",\n    \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n    \"details\": []\n  }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/customer-rg-42hl9p--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi3dcu806c95qg7cc9io5migfbgq6-2\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"PublicIPCountLimitReached\",\n    \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n    \"details\": []\n  }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/customer-rg-42hl9p--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi3dcu806c95qg7cc9io5migfbgq6-3\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"PublicIPCountLimitReached\",\n    \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n    \"details\": []\n  }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/customer-rg-42hl9p--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi3dcu806c95qg7cc9io5migfbgq6-4\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"PublicIPCountLimitReached\",\n    \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n    \"details\": []\n  }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/customer-rg-42hl9p--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi3dcu806c95qg7cc9io5migfbgq6-5\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"PublicIPCountLimitReached\",\n    \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n    \"details\": []\n  }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/customer-rg-42hl9p--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi3dcu806c95qg7cc9io5migfbgq6-6\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"PublicIPCountLimitReached\",\n    \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n    \"details\": []\n  }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/customer-rg-42hl9p--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi3dcu806c95qg7cc9io5migfbgq6-7\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"PublicIPCountLimitReached\",\n    \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n    \"details\": []\n  }\n}\n--------------------------------------------------------------------------------\n\nPUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/customer-rg-42hl9p--managed/providers/Microsoft.Network/publicIPAddresses/2oqsi3dcu806c95qg7cc9io5migfbgq6-8\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: PublicIPCountLimitReached\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"PublicIPCountLimitReached\",\n    \"message\": \"Cannot create more than 300 public IP addresses for this subscription in this region.\",\n    \"details\": []\n  }\n}\n--------------------------------------------------------------------------------\n"
      }
    }
    --------------------------------------------------------------------------------
    
    {
        ErrorCode: "InternalServerError",
        StatusCode: 200,
        RawResponse: {
            Status: "200 OK",
            StatusCode: 200,
            Proto: "HTTP/2.0",
            ProtoMajor: 2,
            ProtoMinor: 0,
            Header: {
                "Cache-Control": ["no-cache"],
                "X-Ms-Ratelimit-Remaining-Subscription-Reads": ["249"],
                "Content-Length": ["7006"],
                "X-Ms-Request-Id": [
                    "8f5597b5-16fa-4df6-bd73-65ab0a1bb005",
                ],
                "X-Envoy-Upstream-Service-Time": ["10"],
                "X-Ms-Ratelimit-Remaining-Subscription-Global-Reads": ["3749"],
                "X-Ms-Correlation-Request-Id": [
                    "bd0a002f-0f30-4cf8-91cd-38e2956ccb93",
                ],
                "Content-Type": ["application/json"],
                "X-Ms-Operation-Identifier": [
                    "appId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,tenantId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,objectId=0e96dbb7-17a2-4393-b2d3-d84524e39c32/canadacentral/eec24277-b57a-425a-a5e4-071fef365907",
                ],
                "X-Ms-Routing-Request-Id": [
                    "CANADACENTRAL:20260304T113304Z:bd0a002f-0f30-4cf8-91cd-38e2956ccb93",
                ],
                "Strict-Transport-Security": [
                    "max-age=31536000; includeSubDomains",
                ],
                "X-Content-Type-Options": ["nosniff"],
                "X-Msedge-Ref": [
                    "Ref A: 7BD958C08EC847A19E41507ED24FBCAF Ref B: CHI261140510034 Ref C: 2026-03-04T11:33:03Z",
                ],
                "Pragma": ["no-cache"],
                "Expires": ["-1"],
                "X-Cache": ["CONFIG_NOCACHE"],
                "Date": [
                    "Wed, 04 Mar 2026 11:33:03 GMT",
                ],
            },
            Body: <*exported.nopClosingBytesReader | 0xc000e12000>{
                s: [123, 10, 32, 32, 32, 32, 34, 105, 100, 34, 58, 32, 34, 47, 115, 117, 98, 115, 99, 114, 105, 112, 116, 105, 111, 110, 115, 47, 52, 48, 51, 100, 57, 100, 101, 57, 45, 49, 51, 50, 98, 45, 52, 57, 55, 52, 45, 57, 52, 97, 53, 45, 53, 98, 55, 56, 98, 100, 102, 97, 49, 57, 49, 101, 47, 112, 114, 111, 118, 105, 100, 101, 114, 115, 47, 77, 105, 99, 114, 111, 115, 111, 102, 116, 46, 82, 101, 100, 72, 97, 116, 79, 112, 101, 110, 83, 104, 105, 102, 116, 47, 108, 111, 99, 97, 116, 105, 111, 110, 115, 47, 117, 107, 115, 111, 117, 116, 104, 47, 104, 99, 112, 79, 112, 101, 114, 97, 116, 105, 111, 110, 83, 116, 97, 116, 117, 115, 101, 115, 47, 48, 98, 51, 53, 49, 53, 53, 102, 45, 49, 99, 56, 55, 45, 52, 97, 50, 55, 45, 98, 50, 54, 52, 45, 98, 53, 102, 102, 97, 102, 50, 100, 102, 53, 57, 50, 34, 44, 10, 32, 32, 32, 32, 34, 110, 97, 109, 101, 34, 58, 32, 34, 48, 98, 51, 53, 49, 53, 53, 102, 45, 49, 99, 56, 55, 45, 52, 97, 50, 55, 45, 98, 50, 54, 52, 45, 98, 53, 102, 102, 97, 102, 50, 100, 102, 53, 57, 50, 34, 44, 10, 32, 32, 32, 32, 34, 115, 116, 97, 116, 117, 115, 34, 58, 32, 34, 70, 97, 105, 108, 101, 100, 34, 44, 10, 32, 32, 32, 32, 34, 115, 116, 97, 114, 116, 84, 105, 109, 101, 34, 58, 32, 34, 50, 48, 50, 54, 45, 48, 51, 45, 48, 52, 84, 49, 49, 58, 50, 55, 58, 49, 48, 46, 49, 52, 49, 53, 49, 48, 50, 51, 56, 90, 34, 44, 10, 32, 32, 32, 32, 34, 101, 110, 100, 84, 105, 109, 101, 34, 58, 32, 34, 50, 48, 50, 54, 45, 48, 51, 45, 48, 52, 84, 49, 49, 58, 51, 51, 58, 48, 49, 46, 53, 48, 55, 56, 56, 48, 53, 56, 51, 90, 34, 44, 10, 32, 32, 32, 32, 34, 101, 114, 114, 111, 114, 34, 58, 32, 123, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 99, 111, 100, 101, 34, 58, 32, 34, 73, 110, 116, 101, 114, 110, 97, 108, 83, 101, 114, 118, 101, 114, 69, 114, 114, 111, 114, 34, 44, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 109, 101, 115, 115, 97, 103, 101, 34, 58, 32, 34, 80, 85, 84, 32, 104, 116, 116, 112, 115, 58, 47, 47, 109, 97, 110, 97, 103, 101, 109, 101, 110, 116, 46, 97, 122, 117, 114, 101, 46, 99, 111, 109, 47, 115, 117, 98, 115, 99, 114, 105, 11...

Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.

Learn more here: https://onsi.github.io/gomega/#adjusting-output

occurred

02:56:34

periodic-ci-Azure-ARO-HCP-main-periodic-prod-e2e-parallel

post-goodmerged PR

No additional run metadata captured

e2e

failed

n/a

e2e (21)

Test Panicked

02:56:34 UTC · failed at e2e · test Customer should be able to test admin credentials before cluster ready, then full admin credential lifecycle · suite prod/parallel

Show raw failure

github.com/Azure/ARO-HCP/test/util/framework.Must[...](...)
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000aca550?, {0x7f5094d33060?, 0xc000e80c90?})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).NewResourceGroup(0xc0008ab0e0, {0x7f5094d33060, 0xc000e80c90}, {0x784e4b5, 0x1f}, {0xc0000140b9, 0x7})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5
github.com/Azure/ARO-HCP/test/e2e.init.func3.1({0x7f5094d33060, 0xc000e80c90})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:66 +0x13f
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000e80c90?})
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b
created by github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 62
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50

fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information
fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked
github.com/Azure/ARO-HCP/test/util/framework.Must[...](...)
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000aca550?, {0x7f5094d33060?, 0xc000e80c90?})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).NewResourceGroup(0xc0008ab0e0, {0x7f5094d33060, 0xc000e80c90}, {0x784e4b5, 0x1f}, {0xc0000140b9, 0x7})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5
github.com/Azure/ARO-HCP/test/e2e.init.func3.1({0x7f5094d33060, 0xc000e80c90})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:66 +0x13f
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000e80c90?})
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b
created by github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 62
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50

fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information
fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked

Interrupted by User

02:56:34 UTC · failed at e2e · test Customer should be able to create a cluster with default autoscaling and a nodepool with autoscaling enabled · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:36]: Interrupted by User
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:36]: Interrupted by User

Test Panicked

02:56:34 UTC · failed at e2e · test Customer should not be able to reuse subnets and NSGs between clusters · suite prod/parallel

Show raw failure

github.com/Azure/ARO-HCP/test/util/framework.Must[...](...)
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000e23cc0?, {0x7f88300fd488?, 0xc000e6bb60?})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).NewResourceGroup(0xc0006fbb00, {0x7f88300fd488, 0xc000e6bb60}, {0x783a7e5, 0x1b}, {0xc0000140b9, 0x7})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5
github.com/Azure/ARO-HCP/test/e2e.init.func11.1({0x7f88300fd488, 0xc000e6bb60})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_nsg_subnet_reuse.go:49 +0xe9
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000e6bb60?})
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b
created by github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 83
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50

fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information
fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked
github.com/Azure/ARO-HCP/test/util/framework.Must[...](...)
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000e23cc0?, {0x7f88300fd488?, 0xc000e6bb60?})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).NewResourceGroup(0xc0006fbb00, {0x7f88300fd488, 0xc000e6bb60}, {0x783a7e5, 0x1b}, {0xc0000140b9, 0x7})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5
github.com/Azure/ARO-HCP/test/e2e.init.func11.1({0x7f88300fd488, 0xc000e6bb60})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_nsg_subnet_reuse.go:49 +0xe9
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000e6bb60?})
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b
created by github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 83
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50

fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information
fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked

Test Panicked

02:56:34 UTC · failed at e2e · test Customer should be able to create an HCP cluster and manage pull secrets · suite prod/parallel

Show raw failure

github.com/Azure/ARO-HCP/test/util/framework.Must[...](...)
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000291f80?, {0x7f122ca56100?, 0xc000a47aa0?})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).NewResourceGroup(0xc000a8efc0, {0x7f122ca56100, 0xc000a47aa0}, {0x77ff021, 0xf}, {0xc0000140b9, 0x7})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5
github.com/Azure/ARO-HCP/test/e2e.init.func12.2({0x7f122ca56100, 0xc000a47aa0})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_pullsecret.go:74 +0x20f
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000a47aa0?})
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b
created by github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 74
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50

fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information
fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked
github.com/Azure/ARO-HCP/test/util/framework.Must[...](...)
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000291f80?, {0x7f122ca56100?, 0xc000a47aa0?})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).NewResourceGroup(0xc000a8efc0, {0x7f122ca56100, 0xc000a47aa0}, {0x77ff021, 0xf}, {0xc0000140b9, 0x7})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5
github.com/Azure/ARO-HCP/test/e2e.init.func12.2({0x7f122ca56100, 0xc000a47aa0})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_pullsecret.go:74 +0x20f
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000a47aa0?})
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b
created by github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 74
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50

fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information
fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked

Test Panicked

02:56:34 UTC · failed at e2e · test Customer should be able to create several HCP clusters in their customer resource group, but not in the same managed resource group · suite prod/parallel

Show raw failure

github.com/Azure/ARO-HCP/test/util/framework.Must[...](...)
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000a33e00?, {0x7f2b24450c60?, 0xc0009a2ea0?})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).NewResourceGroup(0xc0006a2900, {0x7f2b24450c60, 0xc0009a2ea0}, {0x77ee286, 0xb}, {0xc0000140b9, 0x7})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5
github.com/Azure/ARO-HCP/test/e2e.init.func16.2({0x7f2b24450c60, 0xc0009a2ea0})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:65 +0xe9
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc0009a2ea0?})
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b
created by github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 93
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50

fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information
fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked
github.com/Azure/ARO-HCP/test/util/framework.Must[...](...)
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000a33e00?, {0x7f2b24450c60?, 0xc0009a2ea0?})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).NewResourceGroup(0xc0006a2900, {0x7f2b24450c60, 0xc0009a2ea0}, {0x77ee286, 0xb}, {0xc0000140b9, 0x7})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5
github.com/Azure/ARO-HCP/test/e2e.init.func16.2({0x7f2b24450c60, 0xc0009a2ea0})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:65 +0xe9
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc0009a2ea0?})
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b
created by github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 93
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50

fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information
fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked

Interrupted by User

02:56:34 UTC · failed at e2e · test Customer should be able to create an HCP cluster with back-level version 4.19 · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:48]: Interrupted by User
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:48]: Interrupted by User

Interrupted by User

02:56:34 UTC · failed at e2e · test Customer should be able to create an HCP cluster with custom autoscaling · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:36]: Interrupted by User
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:36]: Interrupted by User

Interrupted by User

02:56:34 UTC · failed at e2e · test Customer should be able to create node pools with ARM64-based VMs · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/arm64_nodepool.go:35]: Interrupted by User
fail [github.com/Azure/ARO-HCP/test/e2e/arm64_nodepool.go:35]: Interrupted by User

Test Panicked

02:56:34 UTC · failed at e2e · test Customer should be able to lifecycle and confirm external auth on a cluster · suite prod/parallel

Show raw failure

github.com/Azure/ARO-HCP/test/util/framework.Must[...](...)
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc0008e6680?, {0x7f29796b7660?, 0xc000e00840?})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).NewResourceGroup(0xc0008eefc0, {0x7f29796b7660, 0xc000e00840}, {0x77d9d2e, 0x7}, {0xc0000140b9, 0x7})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5
github.com/Azure/ARO-HCP/test/e2e.init.func19.2({0x7f29796b7660, 0xc000e00840})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/external_auth_list_and_verify.go:58 +0x155
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000e00840?})
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b
created by github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 87
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50

fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information
fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked
github.com/Azure/ARO-HCP/test/util/framework.Must[...](...)
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc0008e6680?, {0x7f29796b7660?, 0xc000e00840?})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).NewResourceGroup(0xc0008eefc0, {0x7f29796b7660, 0xc000e00840}, {0x77d9d2e, 0x7}, {0xc0000140b9, 0x7})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5
github.com/Azure/ARO-HCP/test/e2e.init.func19.2({0x7f29796b7660, 0xc000e00840})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/external_auth_list_and_verify.go:58 +0x155
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000e00840?})
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b
created by github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 87
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50

fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information
fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked

Interrupted by User

02:56:34 UTC · failed at e2e · test Customer should be able to list HCP clusters without node pools at both subscription and resource group levels · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_list_no_infra.go:35]: Interrupted by User
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_list_no_infra.go:35]: Interrupted by User

Interrupted by User

02:56:34 UTC · failed at e2e · test Customer should be able to create an HCP cluster with Image Registry not present · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/image_registry_cluster_create.go:37]: Interrupted by User
fail [github.com/Azure/ARO-HCP/test/e2e/image_registry_cluster_create.go:37]: Interrupted by User

Test Panicked

02:56:34 UTC · failed at e2e · test Customer should be able to create an HCP cluster and custom node pool osDisk size · suite prod/parallel

Show raw failure

github.com/Azure/ARO-HCP/test/util/framework.Must[...](...)
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc0009801c0?, {0x7f138152c460?, 0xc000bf77a0?})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).NewResourceGroup(0xc000a20900, {0x7f138152c460, 0xc000bf77a0}, {0x77f1edf, 0xc}, {0xc0000140b9, 0x7})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5
github.com/Azure/ARO-HCP/test/e2e.init.func9.2({0x7f138152c460, 0xc000bf77a0})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_create_nodepool_osdisk.go:54 +0xe9
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000bf77a0?})
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b
created by github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 99
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50

fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information
fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked
github.com/Azure/ARO-HCP/test/util/framework.Must[...](...)
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc0009801c0?, {0x7f138152c460?, 0xc000bf77a0?})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).NewResourceGroup(0xc000a20900, {0x7f138152c460, 0xc000bf77a0}, {0x77f1edf, 0xc}, {0xc0000140b9, 0x7})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5
github.com/Azure/ARO-HCP/test/e2e.init.func9.2({0x7f138152c460, 0xc000bf77a0})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_create_nodepool_osdisk.go:54 +0xe9
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000bf77a0?})
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b
created by github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 99
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50

fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information
fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked

Interrupted by User

02:56:34 UTC · failed at e2e · test Customer should not be able to create a 4.18 HCP cluster · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_missing_info.go:37]: Interrupted by User
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_missing_info.go:37]: Interrupted by User

Deserializaion Error: no output from command

02:56:34 UTC · failed at e2e · test Customer should respect cluster-wide node limits with nodepool autoscaling · suite prod/parallel

Show raw failure

p=0xc000cf3900 sp=0xc000cf38e0 pc=0x2abd28e
runtime.selectgo(0xc000cf49a0, 0xc000cf3cf4, 0x0?, 0x0, 0x0?, 0x1)
	/usr/local/go/src/runtime/select.go:351 +0x8b7 fp=0xc000cf3a40 sp=0xc000cf3900 pc=0x2a98137
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode(_, {0x42, 0x4, {0x78ef949, 0x41}, 0xc000d855e0, {{0x968cd56, 0x4b}, 0x7a, {0x0, ...}, ...}, ...}, ...)
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:923 +0xf65 fp=0xc000cf4d38 sp=0xc000cf3a40 pc=0x2f00005
github.com/onsi/ginkgo/v2/internal.(*group).attemptSpec(0xc000cf6b98, 0x1, {{0xc000a9ea88?, 0xc000598bc0?, 0x1?}, 0x0?})
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/group.go:199 +0xbd3 fp=0xc000cf5d58 sp=0xc000cf4d38 pc=0x2ee5453
github.com/onsi/ginkgo/v2/internal.(*group).run(0xc000cf6b98, {0xc000159100?, 0x0?, 0x0?})
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/group.go:349 +0x905 fp=0xc000cf6960 sp=0xc000cf5d58 pc=0x2ee79e5
github.com/onsi/ginkgo/v2/internal.(*Suite).runSpecs(0xc00085d508, {0x0, 0x0}, {0xb319ac0, 0x0, 0x0}, {0xc00007a064, 0x2a}, 0x0, {0xc0001590e0, ...})
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:496 +0xa9b fp=0xc000cf6e10 sp=0xc000cf6960 pc=0x2efb4db
github.com/onsi/ginkgo/v2/internal.(*Suite).RunSpec(_, {_, _}, {_, _, _}, {_, _}, {0xc00007a064, 0x2a}, ...)
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite_patch.go:68 +0x3c5 fp=0xc000cf6eb8 sp=0xc000cf6e10 pc=0x2f02845
github.com/openshift-eng/openshift-tests-extension/pkg/ginkgo.BuildExtensionTestSpecsFromOpenShiftGinkgoSuite.func1.1({0x2a4ca65?, 0xc000302300?})
	/go/pkg/mod/github.com/openshift-eng/openshift-tests-extension@v0.0.0-20260127124016-0fed2b824818/pkg/ginkgo/util.go:85 +0x2ba fp=0xc000cf7b88 sp=0xc000cf6eb8 pc=0x521c71a
github.com/openshift-eng/openshift-tests-extension/pkg/extension/extensiontests.runSpec({0x820bbb8, 0xc000b9f0e0}, 0xc000861680, 0x1)
	/go/pkg/mod/github.com/openshift-eng/openshift-tests-extension@v0.0.0-20260127124016-0fed2b824818/pkg/extension/extensiontests/spec.go:596 +0xee fp=0xc000cf7c30 sp=0xc000cf7b88 pc=0x5212a8e
github.com/openshift-eng/openshift-tests-extension/pkg/extension/extensiontests.ExtensionTestSpecs.Run.func2()
	/go/pkg/mod/github.com/openshift-eng/openshift-tests-extension@v0.0.0-20260127124016-0fed2b824818/pkg/extension/extensiontests/spec.go:237 +0x196 fp=0xc000cf7fe0 sp=0xc000cf7c30 pc=0x520e816
runtime.goexit({})
	/usr/local/go/src/runtime/asm_amd64.s:1693 +0x1 fp=0xc000cf7fe8 sp=0xc000cf7fe0 pc=0x2ac5921
created by github.com/openshift-eng/openshift-tests-extension/pkg/extension/extensiontests.ExtensionTestSpecs.Run in goroutine 1
	/go/pkg/mod/github.com/openshift-eng/openshift-tests-extension@v0.0.0-20260127124016-0fed2b824818/pkg/extension/extensiontests/spec.go:230 +0x1e5

goroutine 86 gp=0xc000b32e00 m=nil [select, 2 minutes]:
runtime.gopark(0xc000b3d868?, 0x2?, 0xc8?, 0x2d?, 0xc000b3d834?)
	/usr/local/go/src/runtime/proc.go:460 +0xce fp=0xc000b3d6b8 sp=0xc000b3d698 pc=0x2abd28e
runtime.selectgo(0xc000b3d868, 0xc000b3d830, 0x816e580?, 0x0, 0x0?, 0x1)
	/usr/local/go/src/runtime/select.go:351 +0x8b7 fp=0xc000b3d7f8 sp=0xc000b3d6b8 pc=0x2a98137
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).AssignIdentityContainers(0xc0008ae120, {0x7f0b68c01c60, 0xc000811dd0}, 0x1, 0xdf8475800)
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/identities_helper.go:271 +0x218 fp=0xc000b3d8e0 sp=0xc000b3d7f8 pc=0x4e997f8
github.com/Azure/ARO-HCP/test/e2e.init.func23.3({0x7f0b68c01c60, 0xc000811dd0})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:135 +0x6a fp=0xc000b3df58 sp=0xc000b3d8e0 pc=0x501aaea
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000811dd0?})
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e fp=0xc000b3df88 sp=0xc000b3df58 pc=0x2eed25e
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b fp=0xc000b3dfe0 sp=0xc000b3df88 pc=0x2f0197b
runtime.goexit({})
	/usr/local/go/src/runtime/asm_amd64.s:1693 +0x1 fp=0xc000b3dfe8 sp=0xc000b3dfe0 pc=0x2ac5921
created by github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 83
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50

rax    0xca
rbx    0x0
rcx    0x2ac7743
rdx    0x0
rdi    0xb2ef450
rsi    0x80
rbp    0x7fff3f6424d0
rsp    0x7fff3f642488
r8     0x0
r9     0x0
r10    0x0
r11    0x286
r12    0x2
r13    0x1
r14    0xb2ed360
r15    0x7
rip    0x2ac7741
rflags 0x286
cs     0x33
fs     0x0
gs     0x0
Command Error: exit status 2
Deserializaion Error: no output from command
p=0xc000cf3900 sp=0xc000cf38e0 pc=0x2abd28e
runtime.selectgo(0xc000cf49a0, 0xc000cf3cf4, 0x0?, 0x0, 0x0?, 0x1)
	/usr/local/go/src/runtime/select.go:351 +0x8b7 fp=0xc000cf3a40 sp=0xc000cf3900 pc=0x2a98137
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode(_, {0x42, 0x4, {0x78ef949, 0x41}, 0xc000d855e0, {{0x968cd56, 0x4b}, 0x7a, {0x0, ...}, ...}, ...}, ...)
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:923 +0xf65 fp=0xc000cf4d38 sp=0xc000cf3a40 pc=0x2f00005
github.com/onsi/ginkgo/v2/internal.(*group).attemptSpec(0xc000cf6b98, 0x1, {{0xc000a9ea88?, 0xc000598bc0?, 0x1?}, 0x0?})
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/group.go:199 +0xbd3 fp=0xc000cf5d58 sp=0xc000cf4d38 pc=0x2ee5453
github.com/onsi/ginkgo/v2/internal.(*group).run(0xc000cf6b98, {0xc000159100?, 0x0?, 0x0?})
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/group.go:349 +0x905 fp=0xc000cf6960 sp=0xc000cf5d58 pc=0x2ee79e5
github.com/onsi/ginkgo/v2/internal.(*Suite).runSpecs(0xc00085d508, {0x0, 0x0}, {0xb319ac0, 0x0, 0x0}, {0xc00007a064, 0x2a}, 0x0, {0xc0001590e0, ...})
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:496 +0xa9b fp=0xc000cf6e10 sp=0xc000cf6960 pc=0x2efb4db
github.com/onsi/ginkgo/v2/internal.(*Suite).RunSpec(_, {_, _}, {_, _, _}, {_, _}, {0xc00007a064, 0x2a}, ...)
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite_patch.go:68 +0x3c5 fp=0xc000cf6eb8 sp=0xc000cf6e10 pc=0x2f02845
github.com/openshift-eng/openshift-tests-extension/pkg/ginkgo.BuildExtensionTestSpecsFromOpenShiftGinkgoSuite.func1.1({0x2a4ca65?, 0xc000302300?})
	/go/pkg/mod/github.com/openshift-eng/openshift-tests-extension@v0.0.0-20260127124016-0fed2b824818/pkg/ginkgo/util.go:85 +0x2ba fp=0xc000cf7b88 sp=0xc000cf6eb8 pc=0x521c71a
github.com/openshift-eng/openshift-tests-extension/pkg/extension/extensiontests.runSpec({0x820bbb8, 0xc000b9f0e0}, 0xc000861680, 0x1)
	/go/pkg/mod/github.com/openshift-eng/openshift-tests-extension@v0.0.0-20260127124016-0fed2b824818/pkg/extension/extensiontests/spec.go:596 +0xee fp=0xc000cf7c30 sp=0xc000cf7b88 pc=0x5212a8e
github.com/openshift-eng/openshift-tests-extension/pkg/extension/extensiontests.ExtensionTestSpecs.Run.func2()
	/go/pkg/mod/github.com/openshift-eng/openshift-tests-extension@v0.0.0-20260127124016-0fed2b824818/pkg/extension/extensiontests/spec.go:237 +0x196 fp=0xc000cf7fe0 sp=0xc000cf7c30 pc=0x520e816
runtime.goexit({})
	/usr/local/go/src/runtime/asm_amd64.s:1693 +0x1 fp=0xc000cf7fe8 sp=0xc000cf7fe0 pc=0x2ac5921
created by github.com/openshift-eng/openshift-tests-extension/pkg/extension/extensiontests.ExtensionTestSpecs.Run in goroutine 1
	/go/pkg/mod/github.com/openshift-eng/openshift-tests-extension@v0.0.0-20260127124016-0fed2b824818/pkg/extension/extensiontests/spec.go:230 +0x1e5

goroutine 86 gp=0xc000b32e00 m=nil [select, 2 minutes]:
runtime.gopark(0xc000b3d868?, 0x2?, 0xc8?, 0x2d?, 0xc000b3d834?)
	/usr/local/go/src/runtime/proc.go:460 +0xce fp=0xc000b3d6b8 sp=0xc000b3d698 pc=0x2abd28e
runtime.selectgo(0xc000b3d868, 0xc000b3d830, 0x816e580?, 0x0, 0x0?, 0x1)
	/usr/local/go/src/runtime/select.go:351 +0x8b7 fp=0xc000b3d7f8 sp=0xc000b3d6b8 pc=0x2a98137
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).AssignIdentityContainers(0xc0008ae120, {0x7f0b68c01c60, 0xc000811dd0}, 0x1, 0xdf8475800)
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/identities_helper.go:271 +0x218 fp=0xc000b3d8e0 sp=0xc000b3d7f8 pc=0x4e997f8
github.com/Azure/ARO-HCP/test/e2e.init.func23.3({0x7f0b68c01c60, 0xc000811dd0})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:135 +0x6a fp=0xc000b3df58 sp=0xc000b3d8e0 pc=0x501aaea
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000811dd0?})
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e fp=0xc000b3df88 sp=0xc000b3df58 pc=0x2eed25e
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b fp=0xc000b3dfe0 sp=0xc000b3df88 pc=0x2f0197b
runtime.goexit({})
	/usr/local/go/src/runtime/asm_amd64.s:1693 +0x1 fp=0xc000b3dfe8 sp=0xc000b3dfe0 pc=0x2ac5921
created by github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 83
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50

rax    0xca
rbx    0x0
rcx    0x2ac7743
rdx    0x0
rdi    0xb2ef450
rsi    0x80
rbp    0x7fff3f6424d0
rsp    0x7fff3f642488
r8     0x0
r9     0x0
r10    0x0
r11    0x286
r12    0x2
r13    0x1
r14    0xb2ed360
r15    0x7
rip    0x2ac7741
rflags 0x286
cs     0x33
fs     0x0
gs     0x0
Command Error: exit status 2
Deserializaion Error: no output from command

Test Panicked

02:56:34 UTC · failed at e2e · test Customer should be able to create an HCP cluster · suite prod/parallel

Show raw failure

github.com/Azure/ARO-HCP/test/util/framework.Must[...](...)
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000a6a8c0?, {0x7fb7543ecda0?, 0xc0005e2150?})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).NewResourceGroup(0xc000a2eea0, {0x7fb7543ecda0, 0xc0005e2150}, {0x77f67a9, 0xd}, {0xc0000140b9, 0x7})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5
github.com/Azure/ARO-HCP/test/e2e.init.func17.2({0x7fb7543ecda0, 0xc0005e2150})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create.go:59 +0xe9
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc0005e2150?})
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b
created by github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 85
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50

fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information
fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked
github.com/Azure/ARO-HCP/test/util/framework.Must[...](...)
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000a6a8c0?, {0x7fb7543ecda0?, 0xc0005e2150?})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).NewResourceGroup(0xc000a2eea0, {0x7fb7543ecda0, 0xc0005e2150}, {0x77f67a9, 0xd}, {0xc0000140b9, 0x7})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5
github.com/Azure/ARO-HCP/test/e2e.init.func17.2({0x7fb7543ecda0, 0xc0005e2150})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create.go:59 +0xe9
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc0005e2150?})
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b
created by github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 85
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50

fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information
fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked

Test Panicked

02:56:34 UTC · failed at e2e · test Update HCPOpenShiftCluster Positive creates a cluster and updates tags with a PATCH request · suite prod/parallel

Show raw failure

github.com/Azure/ARO-HCP/test/util/framework.Must[...](...)
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000f22f40?, {0x7f38e8424e60?, 0xc0009e5920?})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).NewResourceGroup(0xc000a52ea0, {0x7f38e8424e60, 0xc0009e5920}, {0x77ea6bf, 0xa}, {0xc0000140b9, 0x7})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5
github.com/Azure/ARO-HCP/test/e2e.init.func14.2.1({0x7f38e8424e60, 0xc0009e5920})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:133 +0xe9
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc0009e5920?})
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b
created by github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 105
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50

fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information
fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked
github.com/Azure/ARO-HCP/test/util/framework.Must[...](...)
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000f22f40?, {0x7f38e8424e60?, 0xc0009e5920?})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).NewResourceGroup(0xc000a52ea0, {0x7f38e8424e60, 0xc0009e5920}, {0x77ea6bf, 0xa}, {0xc0000140b9, 0x7})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5
github.com/Azure/ARO-HCP/test/e2e.init.func14.2.1({0x7f38e8424e60, 0xc0009e5920})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:133 +0xe9
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc0009e5920?})
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b
created by github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 105
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50

fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information
fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked

Test Panicked

02:56:34 UTC · failed at e2e · test Customer should be able to list available HCP OpenShift versions and validate response content · suite prod/parallel

Show raw failure

github.com/Azure/ARO-HCP/test/util/framework.Must[...](...)
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).Get20240610ClientFactoryOrDie(0x78497cf?, {0x7f81f89bf020?, 0xc000ea21e0?})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:611 +0x37
github.com/Azure/ARO-HCP/test/e2e.init.func1.1({0x7f81f89bf020, 0xc000ea21e0})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_versions.go:38 +0x6f
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000ea21e0?})
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b
created by github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 88
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50

fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information
fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked
github.com/Azure/ARO-HCP/test/util/framework.Must[...](...)
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).Get20240610ClientFactoryOrDie(0x78497cf?, {0x7f81f89bf020?, 0xc000ea21e0?})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:611 +0x37
github.com/Azure/ARO-HCP/test/e2e.init.func1.1({0x7f81f89bf020, 0xc000ea21e0})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_versions.go:38 +0x6f
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000ea21e0?})
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b
created by github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 88
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50

fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information
fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked

Test Panicked

02:56:34 UTC · failed at e2e · test Customer should create an HCP cluster and validate TLS certificates · suite prod/parallel

Show raw failure

github.com/Azure/ARO-HCP/test/util/framework.Must[...](...)
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000d21480?, {0x7f4bdc53e060?, 0xc000aefc50?})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).NewResourceGroup(0xc000ed0120, {0x7f4bdc53e060, 0xc000aefc50}, {0x7818357, 0x14}, {0xc0000140b9, 0x7})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5
github.com/Azure/ARO-HCP/test/e2e.init.func13.2({0x7f4bdc53e060, 0xc000aefc50})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:57 +0xe9
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000aefc50?})
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b
created by github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 82
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50

fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information
fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked
github.com/Azure/ARO-HCP/test/util/framework.Must[...](...)
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000d21480?, {0x7f4bdc53e060?, 0xc000aefc50?})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).NewResourceGroup(0xc000ed0120, {0x7f4bdc53e060, 0xc000aefc50}, {0x7818357, 0x14}, {0xc0000140b9, 0x7})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5
github.com/Azure/ARO-HCP/test/e2e.init.func13.2({0x7f4bdc53e060, 0xc000aefc50})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:57 +0xe9
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000aefc50?})
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b
created by github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 82
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50

fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information
fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked

Test Panicked

02:56:34 UTC · failed at e2e · test Customer should be able to create a cluster with an external auth config and get the external auth config · suite prod/parallel

Show raw failure

github.com/Azure/ARO-HCP/test/util/framework.Must[...](...)
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000f32240?, {0x7f3400497460?, 0xc000e9e5d0?})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).NewResourceGroup(0xc000ec6120, {0x7f3400497460, 0xc000e9e5d0}, {0x781d01f, 0x15}, {0xc0000140b9, 0x7})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5
github.com/Azure/ARO-HCP/test/e2e.init.func18.2({0x7f3400497460, 0xc000e9e5d0})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:67 +0xe9
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000e9e5d0?})
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b
created by github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 98
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50

fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information
fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked
github.com/Azure/ARO-HCP/test/util/framework.Must[...](...)
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000f32240?, {0x7f3400497460?, 0xc000e9e5d0?})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).NewResourceGroup(0xc000ec6120, {0x7f3400497460, 0xc000e9e5d0}, {0x781d01f, 0x15}, {0xc0000140b9, 0x7})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5
github.com/Azure/ARO-HCP/test/e2e.init.func18.2({0x7f3400497460, 0xc000e9e5d0})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:67 +0xe9
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000e9e5d0?})
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b
created by github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 98
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50

fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information
fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked

Test Panicked

02:56:34 UTC · failed at e2e · test Authorized CIDRs Connectivity should allow API access only from authorized VM IP · suite prod/parallel

Show raw failure

github.com/Azure/ARO-HCP/test/util/framework.Must[...](...)
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000e90140?, {0x7ff6a549ca60?, 0xc000b0d170?})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).NewResourceGroup(0xc000dc4a20, {0x7ff6a549ca60, 0xc000b0d170}, {0x781cf8c, 0x15}, {0xc0000140b9, 0x7})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5
github.com/Azure/ARO-HCP/test/e2e.init.func5.1.1({0x7ff6a549ca60, 0xc000b0d170})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:68 +0xe9
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000b0d170?})
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b
created by github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 85
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50

fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information
fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked
github.com/Azure/ARO-HCP/test/util/framework.Must[...](...)
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000e90140?, {0x7ff6a549ca60?, 0xc000b0d170?})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).NewResourceGroup(0xc000dc4a20, {0x7ff6a549ca60, 0xc000b0d170}, {0x781cf8c, 0x15}, {0xc0000140b9, 0x7})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5
github.com/Azure/ARO-HCP/test/e2e.init.func5.1.1({0x7ff6a549ca60, 0xc000b0d170})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:68 +0xe9
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000b0d170?})
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b
created by github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 85
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50

fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information
fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked

Test Panicked

02:56:34 UTC · failed at e2e · test Update HCPOpenShiftCluster Negative creates a cluster and fails to update its name with a PATCH request · suite prod/parallel

Show raw failure

github.com/Azure/ARO-HCP/test/util/framework.Must[...](...)
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000ecf480?, {0x7f3e46332348?, 0xc000de9e00?})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).NewResourceGroup(0xc000e918c0, {0x7f3e46332348, 0xc000de9e00}, {0x77ea6b5, 0xa}, {0xc0000140b9, 0x7})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5
github.com/Azure/ARO-HCP/test/e2e.init.func14.1.1({0x7f3e46332348, 0xc000de9e00})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:57 +0xe9
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000de9e00?})
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b
created by github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 111
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50

fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information
fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked
github.com/Azure/ARO-HCP/test/util/framework.Must[...](...)
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000ecf480?, {0x7f3e46332348?, 0xc000de9e00?})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).NewResourceGroup(0xc000e918c0, {0x7f3e46332348, 0xc000de9e00}, {0x77ea6b5, 0xa}, {0xc0000140b9, 0x7})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5
github.com/Azure/ARO-HCP/test/e2e.init.func14.1.1({0x7f3e46332348, 0xc000de9e00})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:57 +0xe9
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000de9e00?})
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b
created by github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 111
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50

fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information
fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked

02:00:12

periodic-ci-Azure-ARO-HCP-main-periodic-prod-e2e-parallel

post-goodmerged PR

No additional run metadata captured

e2e

failed

n/a

e2e (21)

Test Panicked

02:00:12 UTC · failed at e2e · test Customer should be able to create several HCP clusters in their customer resource group, but not in the same managed resource group · suite prod/parallel

Show raw failure

github.com/Azure/ARO-HCP/test/util/framework.Must[...](...)
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000e26280?, {0x7fbe01426860?, 0xc000dbaf30?})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).NewResourceGroup(0xc000dee120, {0x7fbe01426860, 0xc000dbaf30}, {0x77ee286, 0xb}, {0xc0000140b9, 0x7})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5
github.com/Azure/ARO-HCP/test/e2e.init.func16.2({0x7fbe01426860, 0xc000dbaf30})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:65 +0xe9
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000dbaf30?})
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b
created by github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 91
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50

fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information
fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked
github.com/Azure/ARO-HCP/test/util/framework.Must[...](...)
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000e26280?, {0x7fbe01426860?, 0xc000dbaf30?})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).NewResourceGroup(0xc000dee120, {0x7fbe01426860, 0xc000dbaf30}, {0x77ee286, 0xb}, {0xc0000140b9, 0x7})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5
github.com/Azure/ARO-HCP/test/e2e.init.func16.2({0x7fbe01426860, 0xc000dbaf30})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:65 +0xe9
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000dbaf30?})
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b
created by github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 91
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50

fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information
fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked

Test Panicked

02:00:12 UTC · failed at e2e · test Customer should not be able to create a 4.18 HCP cluster · suite prod/parallel

Show raw failure

github.com/Azure/ARO-HCP/test/util/framework.Must[...](...)
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000e10900?, {0x7efc940c7260?, 0xc000abae70?})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).NewResourceGroup(0xc000a5c120, {0x7efc940c7260, 0xc000abae70}, {0x7812f30, 0x13}, {0xc0000140b9, 0x7})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5
github.com/Azure/ARO-HCP/test/e2e.init.func7.2({0x7efc940c7260, 0xc000abae70})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_create_missing_info.go:54 +0x105
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000abae70?})
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b
created by github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 107
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50

fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information
fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked
github.com/Azure/ARO-HCP/test/util/framework.Must[...](...)
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000e10900?, {0x7efc940c7260?, 0xc000abae70?})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).NewResourceGroup(0xc000a5c120, {0x7efc940c7260, 0xc000abae70}, {0x7812f30, 0x13}, {0xc0000140b9, 0x7})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5
github.com/Azure/ARO-HCP/test/e2e.init.func7.2({0x7efc940c7260, 0xc000abae70})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_create_missing_info.go:54 +0x105
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000abae70?})
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b
created by github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 107
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50

fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information
fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked

Test Panicked

02:00:12 UTC · failed at e2e · test Customer should be able to lifecycle and confirm external auth on a cluster · suite prod/parallel

Show raw failure

github.com/Azure/ARO-HCP/test/util/framework.Must[...](...)
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000e5ab80?, {0x7fa170358460?, 0xc000a04090?})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).NewResourceGroup(0xc000de57a0, {0x7fa170358460, 0xc000a04090}, {0x77d9d2e, 0x7}, {0xc0000140b9, 0x7})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5
github.com/Azure/ARO-HCP/test/e2e.init.func19.2({0x7fa170358460, 0xc000a04090})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/external_auth_list_and_verify.go:58 +0x155
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000a04090?})
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b
created by github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 101
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50

fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information
fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked
github.com/Azure/ARO-HCP/test/util/framework.Must[...](...)
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000e5ab80?, {0x7fa170358460?, 0xc000a04090?})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).NewResourceGroup(0xc000de57a0, {0x7fa170358460, 0xc000a04090}, {0x77d9d2e, 0x7}, {0xc0000140b9, 0x7})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5
github.com/Azure/ARO-HCP/test/e2e.init.func19.2({0x7fa170358460, 0xc000a04090})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/external_auth_list_and_verify.go:58 +0x155
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000a04090?})
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b
created by github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 101
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50

fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information
fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked

Test Panicked

02:00:12 UTC · failed at e2e · test Customer should be able to test admin credentials before cluster ready, then full admin credential lifecycle · suite prod/parallel

Show raw failure

github.com/Azure/ARO-HCP/test/util/framework.Must[...](...)
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000d038b0?, {0x7ff65c2bc1a0?, 0xc000da5440?})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).NewResourceGroup(0xc000dd3680, {0x7ff65c2bc1a0, 0xc000da5440}, {0x784e4b5, 0x1f}, {0xc0000140b9, 0x7})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5
github.com/Azure/ARO-HCP/test/e2e.init.func3.1({0x7ff65c2bc1a0, 0xc000da5440})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:66 +0x13f
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000da5440?})
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b
created by github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 109
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50

fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information
fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked
github.com/Azure/ARO-HCP/test/util/framework.Must[...](...)
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000d038b0?, {0x7ff65c2bc1a0?, 0xc000da5440?})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).NewResourceGroup(0xc000dd3680, {0x7ff65c2bc1a0, 0xc000da5440}, {0x784e4b5, 0x1f}, {0xc0000140b9, 0x7})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5
github.com/Azure/ARO-HCP/test/e2e.init.func3.1({0x7ff65c2bc1a0, 0xc000da5440})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:66 +0x13f
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000da5440?})
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b
created by github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 109
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50

fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information
fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked

Test Panicked

02:00:12 UTC · failed at e2e · test Customer should not be able to reuse subnets and NSGs between clusters · suite prod/parallel

Show raw failure

github.com/Azure/ARO-HCP/test/util/framework.Must[...](...)
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000e5e690?, {0x7f6e31d87120?, 0xc000c26210?})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).NewResourceGroup(0xc000c30a20, {0x7f6e31d87120, 0xc000c26210}, {0x783a7e5, 0x1b}, {0xc0000140b9, 0x7})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5
github.com/Azure/ARO-HCP/test/e2e.init.func11.1({0x7f6e31d87120, 0xc000c26210})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_nsg_subnet_reuse.go:49 +0xe9
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000c26210?})
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b
created by github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 82
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50

fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information
fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked
github.com/Azure/ARO-HCP/test/util/framework.Must[...](...)
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000e5e690?, {0x7f6e31d87120?, 0xc000c26210?})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).NewResourceGroup(0xc000c30a20, {0x7f6e31d87120, 0xc000c26210}, {0x783a7e5, 0x1b}, {0xc0000140b9, 0x7})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5
github.com/Azure/ARO-HCP/test/e2e.init.func11.1({0x7f6e31d87120, 0xc000c26210})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_nsg_subnet_reuse.go:49 +0xe9
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000c26210?})
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b
created by github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 82
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50

fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information
fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked

Interrupted by User

02:00:12 UTC · failed at e2e · test Customer should be able to create an HCP cluster and manage pull secrets · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_pullsecret.go:46]: Interrupted by User
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_pullsecret.go:46]: Interrupted by User

Test Panicked

02:00:12 UTC · failed at e2e · test Customer should be able to create a cluster with an external auth config and get the external auth config · suite prod/parallel

Show raw failure

github.com/Azure/ARO-HCP/test/util/framework.Must[...](...)
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000e0be00?, {0x7f4d79470ee0?, 0xc000f620f0?})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).NewResourceGroup(0xc000f07680, {0x7f4d79470ee0, 0xc000f620f0}, {0x781d01f, 0x15}, {0xc0000140b9, 0x7})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5
github.com/Azure/ARO-HCP/test/e2e.init.func18.2({0x7f4d79470ee0, 0xc000f620f0})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:67 +0xe9
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000f620f0?})
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b
created by github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 103
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50

fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information
fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked
github.com/Azure/ARO-HCP/test/util/framework.Must[...](...)
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000e0be00?, {0x7f4d79470ee0?, 0xc000f620f0?})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).NewResourceGroup(0xc000f07680, {0x7f4d79470ee0, 0xc000f620f0}, {0x781d01f, 0x15}, {0xc0000140b9, 0x7})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5
github.com/Azure/ARO-HCP/test/e2e.init.func18.2({0x7f4d79470ee0, 0xc000f620f0})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:67 +0xe9
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000f620f0?})
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b
created by github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 103
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50

fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information
fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked

Interrupted by User

02:00:12 UTC · failed at e2e · test Update HCPOpenShiftCluster Positive creates a cluster and updates tags with a PATCH request · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:120]: Interrupted by User
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:120]: Interrupted by User

Test Panicked

02:00:12 UTC · failed at e2e · test Customer should be able to list HCP clusters without node pools at both subscription and resource group levels · suite prod/parallel

Show raw failure

github.com/Azure/ARO-HCP/test/util/framework.Must[...](...)
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc00097fa00?, {0x7f5fef765660?, 0xc000f0ca80?})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).NewResourceGroup(0xc000f5ea20, {0x7f5fef765660, 0xc000f0ca80}, {0x77ff012, 0xf}, {0xc0000140b9, 0x7})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5
github.com/Azure/ARO-HCP/test/e2e.init.func10.1({0x7f5fef765660, 0xc000f0ca80})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_list_no_infra.go:55 +0x23e
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000f0ca80?})
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b
created by github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 102
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50

fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information
fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked
github.com/Azure/ARO-HCP/test/util/framework.Must[...](...)
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc00097fa00?, {0x7f5fef765660?, 0xc000f0ca80?})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).NewResourceGroup(0xc000f5ea20, {0x7f5fef765660, 0xc000f0ca80}, {0x77ff012, 0xf}, {0xc0000140b9, 0x7})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5
github.com/Azure/ARO-HCP/test/e2e.init.func10.1({0x7f5fef765660, 0xc000f0ca80})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_list_no_infra.go:55 +0x23e
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000f0ca80?})
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b
created by github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 102
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50

fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information
fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked

Test Panicked

02:00:12 UTC · failed at e2e · test Customer should be able to create an HCP cluster with back-level version 4.19 · suite prod/parallel

Show raw failure

github.com/Azure/ARO-HCP/test/util/framework.Must[...](...)
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc00103a050?, {0x7fb94c3f7c60?, 0xc000f0e720?})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).NewResourceGroup(0xc000f306c0, {0x7fb94c3f7c60, 0xc000f0e720}, {0x782740d, 0x17}, {0xc0000140b9, 0x7})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5
github.com/Azure/ARO-HCP/test/e2e.init.func15.1({0x7fb94c3f7c60, 0xc000f0e720})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:69 +0xf1
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000f0e720?})
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b
created by github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 106
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50

fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information
fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked
github.com/Azure/ARO-HCP/test/util/framework.Must[...](...)
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc00103a050?, {0x7fb94c3f7c60?, 0xc000f0e720?})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).NewResourceGroup(0xc000f306c0, {0x7fb94c3f7c60, 0xc000f0e720}, {0x782740d, 0x17}, {0xc0000140b9, 0x7})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5
github.com/Azure/ARO-HCP/test/e2e.init.func15.1({0x7fb94c3f7c60, 0xc000f0e720})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:69 +0xf1
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000f0e720?})
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b
created by github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 106
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50

fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information
fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked

Test Panicked

02:00:12 UTC · failed at e2e · test Customer should create an HCP cluster and validate TLS certificates · suite prod/parallel

Show raw failure

github.com/Azure/ARO-HCP/test/util/framework.Must[...](...)
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000d91780?, {0x7f5edd39dea0?, 0xc000ed19e0?})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).NewResourceGroup(0xc000c82480, {0x7f5edd39dea0, 0xc000ed19e0}, {0x7818357, 0x14}, {0xc0000140b9, 0x7})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5
github.com/Azure/ARO-HCP/test/e2e.init.func13.2({0x7f5edd39dea0, 0xc000ed19e0})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:57 +0xe9
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000ed19e0?})
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b
created by github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 96
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50

fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information
fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked
github.com/Azure/ARO-HCP/test/util/framework.Must[...](...)
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000d91780?, {0x7f5edd39dea0?, 0xc000ed19e0?})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).NewResourceGroup(0xc000c82480, {0x7f5edd39dea0, 0xc000ed19e0}, {0x7818357, 0x14}, {0xc0000140b9, 0x7})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5
github.com/Azure/ARO-HCP/test/e2e.init.func13.2({0x7f5edd39dea0, 0xc000ed19e0})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:57 +0xe9
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000ed19e0?})
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b
created by github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 96
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50

fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information
fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked

Interrupted by User

02:00:12 UTC · failed at e2e · test Customer should be able to create an HCP cluster and custom node pool osDisk size · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_nodepool_osdisk.go:35]: Interrupted by User
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_nodepool_osdisk.go:35]: Interrupted by User

Test Panicked

02:00:12 UTC · failed at e2e · test Customer should be able to create an HCP cluster · suite prod/parallel

Show raw failure

github.com/Azure/ARO-HCP/test/util/framework.Must[...](...)
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000d17100?, {0x7f0a10091060?, 0xc000be5500?})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).NewResourceGroup(0xc0009c18c0, {0x7f0a10091060, 0xc000be5500}, {0x77f67a9, 0xd}, {0xc0000140b9, 0x7})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5
github.com/Azure/ARO-HCP/test/e2e.init.func17.2({0x7f0a10091060, 0xc000be5500})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create.go:59 +0xe9
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000be5500?})
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b
created by github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 104
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50

fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information
fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked
github.com/Azure/ARO-HCP/test/util/framework.Must[...](...)
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000d17100?, {0x7f0a10091060?, 0xc000be5500?})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).NewResourceGroup(0xc0009c18c0, {0x7f0a10091060, 0xc000be5500}, {0x77f67a9, 0xd}, {0xc0000140b9, 0x7})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5
github.com/Azure/ARO-HCP/test/e2e.init.func17.2({0x7f0a10091060, 0xc000be5500})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create.go:59 +0xe9
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000be5500?})
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b
created by github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 104
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50

fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information
fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked

Interrupted by User

02:00:12 UTC · failed at e2e · test Customer should be able to create an HCP cluster with Image Registry not present · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/image_registry_cluster_create.go:37]: Interrupted by User
fail [github.com/Azure/ARO-HCP/test/e2e/image_registry_cluster_create.go:37]: Interrupted by User

Interrupted by User

02:00:12 UTC · failed at e2e · test Customer should be able to create a cluster with default autoscaling and a nodepool with autoscaling enabled · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:36]: Interrupted by User
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:36]: Interrupted by User

Test Panicked

02:00:12 UTC · failed at e2e · test Customer should be able to create node pools with ARM64-based VMs · suite prod/parallel

Show raw failure

github.com/Azure/ARO-HCP/test/util/framework.Must[...](...)
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000df4300?, {0x7fa2da729c60?, 0xc000f1c150?})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).NewResourceGroup(0xc000f26240, {0x7fa2da729c60, 0xc000f1c150}, {0x7803b39, 0x10}, {0xc0000140b9, 0x7})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5
github.com/Azure/ARO-HCP/test/e2e.init.func4.2({0x7fa2da729c60, 0xc000f1c150})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/arm64_nodepool.go:59 +0x105
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000f1c150?})
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b
created by github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 94
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50

fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information
fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked
github.com/Azure/ARO-HCP/test/util/framework.Must[...](...)
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000df4300?, {0x7fa2da729c60?, 0xc000f1c150?})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).NewResourceGroup(0xc000f26240, {0x7fa2da729c60, 0xc000f1c150}, {0x7803b39, 0x10}, {0xc0000140b9, 0x7})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5
github.com/Azure/ARO-HCP/test/e2e.init.func4.2({0x7fa2da729c60, 0xc000f1c150})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/arm64_nodepool.go:59 +0x105
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000f1c150?})
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b
created by github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 94
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50

fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information
fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked

Test Panicked

02:00:12 UTC · failed at e2e · test Customer should be able to list available HCP OpenShift versions and validate response content · suite prod/parallel

Show raw failure

github.com/Azure/ARO-HCP/test/util/framework.Must[...](...)
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).Get20240610ClientFactoryOrDie(0x78497cf?, {0x7f3fd40c6060?, 0xc000ebf860?})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:611 +0x37
github.com/Azure/ARO-HCP/test/e2e.init.func1.1({0x7f3fd40c6060, 0xc000ebf860})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_versions.go:38 +0x6f
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000ebf860?})
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b
created by github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 90
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50

fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information
fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked
github.com/Azure/ARO-HCP/test/util/framework.Must[...](...)
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).Get20240610ClientFactoryOrDie(0x78497cf?, {0x7f3fd40c6060?, 0xc000ebf860?})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:611 +0x37
github.com/Azure/ARO-HCP/test/e2e.init.func1.1({0x7f3fd40c6060, 0xc000ebf860})
	/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_versions.go:38 +0x6f
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000ebf860?})
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b
created by github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 90
	/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50

fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information
fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked

Interrupted by User

02:00:12 UTC · failed at e2e · test Authorized CIDRs Connectivity should allow API access only from authorized VM IP · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:45]: Interrupted by User
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:45]: Interrupted by User

Interrupted by User

02:00:12 UTC · failed at e2e · test Customer should respect cluster-wide node limits with nodepool autoscaling · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:122]: Interrupted by User
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:122]: Interrupted by User

Interrupted by User

02:00:12 UTC · failed at e2e · test Update HCPOpenShiftCluster Negative creates a cluster and fails to update its name with a PATCH request · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:44]: Interrupted by User
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:44]: Interrupted by User

Interrupted by User

02:00:12 UTC · failed at e2e · test Customer should be able to create an HCP cluster with custom autoscaling · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:36]: Interrupted by User
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:36]: Interrupted by User