No failure patterns match your search.
Environment: STG
Failure patterns: 5 · Failures matched: 46 · Total runs: 6 · Runs affected: 5
| Failure patternThe canonical description of a recurring CI failure, extracted and normalized from raw logs. | Failed atThe stage of the job run where this failure occurred: 'provision' (environment setup, DEV only), 'e2e' (test suite execution), or 'other' (CI infrastructure issues that did not produce a failure pattern). | Number of distinct job runs where this failure pattern was detected. | Percentage of all job runs in this environment affected by this failure pattern during the selected window. | Classification of this failure pattern: Regression (likely caused by a specific PR), Flake (intermittent failure spread across days), Noise (low-quality or generic pattern), or Indeterminate. | TrendShows daily activity for this failure pattern in a trailing window anchored to the selected end date. The sparkline covers at least 7 days and at most 14 days, depending on the current window size. | Also inOther environments where the same failure pattern was also detected during the selected window. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
timeout during GetAdminRESTConfigForHCPCluster while waiting for hcpcluster credsfull failure pattern: timeout during GetAdminRESTConfigForHCPCluster while waiting for hcpcluster creds Signal: IndeterminateSignal: Indeterminate — seen in 1 prior week(s); active 1 day(s) | e2e | 3 | 50.00%3 of 6 job runs affected | IndeterminateSignal: Indeterminate — seen in 1 prior week(s); active 1 day(s) | Show trend detailsMar 27: 0 · Mar 28: 0 · Mar 29: 0 · Mar 30: 0 · Mar 31: 0 · Apr 1: 0 · Apr 2: 25 | none | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (3)fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_labels_taints.go:101]: Unexpected error:
<*fmt.wrapErrors | 0xc0009decc0>:
failed waiting for hcpCluster="cluster-np-labels-taints" in resourcegroup="rg-np-labels-taints-782d8g" to finish getting creds, caused by: timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster cluster-np-labels-taints in resource group rg-np-labels-taints-782d8g, error: context deadline exceeded
{
msg: "failed waiting for hcpCluster=\"cluster-np-labels-taints\" in resourcegroup=\"rg-np-labels-taints-782d8g\" to finish getting creds, caused by: timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster cluster-np-labels-taints in resource group rg-np-labels-taints-782d8g, error: context deadline exceeded",
errs: [
<*errors.errorString | 0xc000c04df0>{
s: "timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster cluster-np-labels-taints in resource group rg-np-labels-taints-782d8g",
},
<context.deadlineExceededError>{},
],
}
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_labels_taints.go:101]: Unexpected error:
<*fmt.wrapErrors | 0xc0009decc0>:
failed waiting for hcpCluster="cluster-np-labels-taints" in resourcegroup="rg-np-labels-taints-782d8g" to finish getting creds, caused by: timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster cluster-np-labels-taints in resource group rg-np-labels-taints-782d8g, error: context deadline exceeded
{
msg: "failed waiting for hcpCluster=\"cluster-np-labels-taints\" in resourcegroup=\"rg-np-labels-taints-782d8g\" to finish getting creds, caused by: timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster cluster-np-labels-taints in resource group rg-np-labels-taints-782d8g, error: context deadline exceeded",
errs: [
<*errors.errorString | 0xc000c04df0>{
s: "timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster cluster-np-labels-taints in resource group rg-np-labels-taints-782d8g",
},
<context.deadlineExceededError>{},
],
}
occurredfail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:103]: Unexpected error:
<*fmt.wrapErrors | 0xc00156a3c0>:
failed waiting for hcpCluster="ea-cluster" in resourcegroup="external-auth-cluster-x5nbs9" to finish getting creds, caused by: timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster ea-cluster in resource group external-auth-cluster-x5nbs9, error: context deadline exceeded
{
msg: "failed waiting for hcpCluster=\"ea-cluster\" in resourcegroup=\"external-auth-cluster-x5nbs9\" to finish getting creds, caused by: timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster ea-cluster in resource group external-auth-cluster-x5nbs9, error: context deadline exceeded",
errs: [
<*errors.errorString | 0xc000dbddd0>{
s: "timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster ea-cluster in resource group external-auth-cluster-x5nbs9",
},
<context.deadlineExceededError>{},
],
}
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:103]: Unexpected error:
<*fmt.wrapErrors | 0xc00156a3c0>:
failed waiting for hcpCluster="ea-cluster" in resourcegroup="external-auth-cluster-x5nbs9" to finish getting creds, caused by: timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster ea-cluster in resource group external-auth-cluster-x5nbs9, error: context deadline exceeded
{
msg: "failed waiting for hcpCluster=\"ea-cluster\" in resourcegroup=\"external-auth-cluster-x5nbs9\" to finish getting creds, caused by: timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster ea-cluster in resource group external-auth-cluster-x5nbs9, error: context deadline exceeded",
errs: [
<*errors.errorString | 0xc000dbddd0>{
s: "timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster ea-cluster in resource group external-auth-cluster-x5nbs9",
},
<context.deadlineExceededError>{},
],
}
occurredfail [github.com/Azure/ARO-HCP/test/e2e/image_registry_cluster_create.go:105]: Unexpected error:
<*fmt.wrapErrors | 0xc00071ba70>:
failed waiting for hcpCluster="disabled-image-registry-hcp-cluster" in resourcegroup="disabled-image-registry-dmzpzs" to finish getting creds, caused by: timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster disabled-image-registry-hcp-cluster in resource group disabled-image-registry-dmzpzs, error: context deadline exceeded
{
msg: "failed waiting for hcpCluster=\"disabled-image-registry-hcp-cluster\" in resourcegroup=\"disabled-image-registry-dmzpzs\" to finish getting creds, caused by: timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster disabled-image-registry-hcp-cluster in resource group disabled-image-registry-dmzpzs, error: context deadline exceeded",
errs: [
<*errors.errorString | 0xc0009c5c60>{
s: "timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster disabled-image-registry-hcp-cluster in resource group disabled-image-registry-dmzpzs",
},
<context.deadlineExceededError>{},
],
}
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/image_registry_cluster_create.go:105]: Unexpected error:
<*fmt.wrapErrors | 0xc00071ba70>:
failed waiting for hcpCluster="disabled-image-registry-hcp-cluster" in resourcegroup="disabled-image-registry-dmzpzs" to finish getting creds, caused by: timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster disabled-image-registry-hcp-cluster in resource group disabled-image-registry-dmzpzs, error: context deadline exceeded
{
msg: "failed waiting for hcpCluster=\"disabled-image-registry-hcp-cluster\" in resourcegroup=\"disabled-image-registry-dmzpzs\" to finish getting creds, caused by: timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster disabled-image-registry-hcp-cluster in resource group disabled-image-registry-dmzpzs, error: context deadline exceeded",
errs: [
<*errors.errorString | 0xc0009c5c60>{
s: "timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster disabled-image-registry-hcp-cluster in resource group disabled-image-registry-dmzpzs",
},
<context.deadlineExceededError>{},
],
}
occurredContributing tests (16)
Affected runs (3)
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
ERROR CODE: InternalServerError; detail message insufficient public IP address quota: required <count>, available <count>; provider Microsoft.RedHatOpenShiftfull failure pattern: ERROR CODE: InternalServerError; detail message insufficient public IP address quota: required <count>, available <count>; provider Microsoft.RedHatOpenShift Signal: IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | e2e | 2 | 33.33%2 of 6 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | Show trend detailsMar 27: 0 · Mar 28: 0 · Mar 29: 0 · Mar 30: 0 · Mar 31: 0 · Apr 1: 0 · Apr 2: 10 | none | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (3)fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:141]: Unexpected error:
<*fmt.wrapError | 0xc000732640>:
failed to create HCP cluster cluster-zstream-4-20-7mzwzp: failed waiting for cluster="cluster-zstream-4-20-7mzwzp" in resourcegroup="rg-zstream-upgrade-4-20-tq69zx" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/e3ee4618-26eb-473e-867f-a23c3644e2bb
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/e3ee4618-26eb-473e-867f-a23c3644e2bb",
"name": "e3ee4618-26eb-473e-867f-a23c3644e2bb",
"status": "Failed",
"startTime": "2026-04-02T17:31:57.472030714Z",
"endTime": "2026-04-02T17:32:55.049606818Z",
"error": {
"code": "InternalServerError",
"message": "insufficient public IP address quota: required 2, available 0"
}
}
--------------------------------------------------------------------------------
{
msg: "failed to create HCP cluster cluster-zstream-4-20-7mzwzp: failed waiting for cluster=\"cluster-zstream-4-20-7mzwzp\" in resourcegroup=\"rg-zstream-upgrade-4-20-tq69zx\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/e3ee4618-26eb-473e-867f-a23c3644e2bb\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/e3ee4618-26eb-473e-867f-a23c3644e2bb\",\n \"name\": \"e3ee4618-26eb-473e-867f-a23c3644e2bb\",\n \"status\": \"Failed\",\n \"startTime\": \"2026-04-02T17:31:57.472030714Z\",\n \"endTime\": \"2026-04-02T17:32:55.049606818Z\",\n \"error\": {\n \"code\": \"InternalServerError\",\n \"message\": \"insufficient public IP address quota: required 2, available 0\"\n }\n}\n--------------------------------------------------------------------------------\n",
err: <*fmt.wrapError | 0xc000732600>{
msg: "failed waiting for cluster=\"cluster-zstream-4-20-7mzwzp\" in resourcegroup=\"rg-zstream-upgrade-4-20-tq69zx\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/e3ee4618-26eb-473e-867f-a23c3644e2bb\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/e3ee4618-26eb-473e-867f-a23c3644e2bb\",\n \"name\": \"e3ee4618-26eb-473e-867f-a23c3644e2bb\",\n \"status\": \"Failed\",\n \"startTime\": \"2026-04-02T17:31:57.472030714Z\",\n \"endTime\": \"2026-04-02T17:32:55.049606818Z\",\n \"error\": {\n \"code\": \"InternalServerError\",\n \"message\": \"insufficient public IP address quota: required 2, available 0\"\n }\n}\n--------------------------------------------------------------------------------\n",
err: <*exported.ResponseError | 0xc0014af200>{
ErrorCode: "InternalServerError",
StatusCode: 200,
RawResponse: {
Status: "200 OK",
StatusCode: 200,
Proto: "HTTP/2.0",
ProtoMajor: 2,
ProtoMinor: 0,
Header: {
"X-Ms-Operation-Identifier": [
"appId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,tenantId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,objectId=0e96dbb7-17a2-4393-b2d3-d84524e39c32/canadaeast/81439e1d-c310-4abd-91e5-f37963acb072",
],
"Pragma": ["no-cache"],
"Expires": ["-1"],
"X-Ms-Ratelimit-Remaining-Subscription-Reads": ["249"],
"X-Ms-Ratelimit-Remaining-Subscription-Global-Reads": ["3749"],
"X-Cache": ["CONFIG_NOCACHE"],
"Cache-Control": ["no-cache"],
"Content-Length": ["499"],
"Content-Type": ["application/json"],
"Strict-Transport-Security": [
"max-age=31536000; includeSubDomains",
],
"X-Content-Type-Options": ["nosniff"],
"X-Msedge-Ref": [
"Ref A: 41853D22B093487EB33229F443FF6553 Ref B: CHI261140513025 Ref C: 2026-04-02T17:33:00Z",
...
Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.
Learn more here: https://onsi.github.io/gomega/#adjusting-output
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:141]: Unexpected error:
<*fmt.wrapError | 0xc000732640>:
failed to create HCP cluster cluster-zstream-4-20-7mzwzp: failed waiting for cluster="cluster-zstream-4-20-7mzwzp" in resourcegroup="rg-zstream-upgrade-4-20-tq69zx" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/e3ee4618-26eb-473e-867f-a23c3644e2bb
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/e3ee4618-26eb-473e-867f-a23c3644e2bb",
"name": "e3ee4618-26eb-473e-867f-a23c3644e2bb",
"status": "Failed",
"startTime": "2026-04-02T17:31:57.472030714Z",
"endTime": "2026-04-02T17:32:55.049606818Z",
"error": {
"code": "InternalServerError",
"message": "insufficient public IP address quota: required 2, available 0"
}
}
--------------------------------------------------------------------------------
{
msg: "failed to create HCP cluster cluster-zstream-4-20-7mzwzp: failed waiting for cluster=\"cluster-zstream-4-20-7mzwzp\" in resourcegroup=\"rg-zstream-upgrade-4-20-tq69zx\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/e3ee4618-26eb-473e-867f-a23c3644e2bb\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/e3ee4618-26eb-473e-867f-a23c3644e2bb\",\n \"name\": \"e3ee4618-26eb-473e-867f-a23c3644e2bb\",\n \"status\": \"Failed\",\n \"startTime\": \"2026-04-02T17:31:57.472030714Z\",\n \"endTime\": \"2026-04-02T17:32:55.049606818Z\",\n \"error\": {\n \"code\": \"InternalServerError\",\n \"message\": \"insufficient public IP address quota: required 2, available 0\"\n }\n}\n--------------------------------------------------------------------------------\n",
err: <*fmt.wrapError | 0xc000732600>{
msg: "failed waiting for cluster=\"cluster-zstream-4-20-7mzwzp\" in resourcegroup=\"rg-zstream-upgrade-4-20-tq69zx\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/e3ee4618-26eb-473e-867f-a23c3644e2bb\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/e3ee4618-26eb-473e-867f-a23c3644e2bb\",\n \"name\": \"e3ee4618-26eb-473e-867f-a23c3644e2bb\",\n \"status\": \"Failed\",\n \"startTime\": \"2026-04-02T17:31:57.472030714Z\",\n \"endTime\": \"2026-04-02T17:32:55.049606818Z\",\n \"error\": {\n \"code\": \"InternalServerError\",\n \"message\": \"insufficient public IP address quota: required 2, available 0\"\n }\n}\n--------------------------------------------------------------------------------\n",
err: <*exported.ResponseError | 0xc0014af200>{
ErrorCode: "InternalServerError",
StatusCode: 200,
RawResponse: {
Status: "200 OK",
StatusCode: 200,
Proto: "HTTP/2.0",
ProtoMajor: 2,
ProtoMinor: 0,
Header: {
"X-Ms-Operation-Identifier": [
"appId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,tenantId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,objectId=0e96dbb7-17a2-4393-b2d3-d84524e39c32/canadaeast/81439e1d-c310-4abd-91e5-f37963acb072",
],
"Pragma": ["no-cache"],
"Expires": ["-1"],
"X-Ms-Ratelimit-Remaining-Subscription-Reads": ["249"],
"X-Ms-Ratelimit-Remaining-Subscription-Global-Reads": ["3749"],
"X-Cache": ["CONFIG_NOCACHE"],
"Cache-Control": ["no-cache"],
"Content-Length": ["499"],
"Content-Type": ["application/json"],
"Strict-Transport-Security": [
"max-age=31536000; includeSubDomains",
],
"X-Content-Type-Options": ["nosniff"],
"X-Msedge-Ref": [
"Ref A: 41853D22B093487EB33229F443FF6553 Ref B: CHI261140513025 Ref C: 2026-04-02T17:33:00Z",
...
Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.
Learn more here: https://onsi.github.io/gomega/#adjusting-output
occurredfail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:141]: Unexpected error:
<*fmt.wrapError | 0xc0008d7580>:
failed to create HCP cluster cluster-zstream-4-19-456qg5: failed waiting for cluster="cluster-zstream-4-19-456qg5" in resourcegroup="rg-zstream-upgrade-4-19-p762zb" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/78ee839b-27e8-45ab-9bef-ab361680e06a
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/78ee839b-27e8-45ab-9bef-ab361680e06a",
"name": "78ee839b-27e8-45ab-9bef-ab361680e06a",
"status": "Failed",
"startTime": "2026-04-02T17:27:46.835065402Z",
"endTime": "2026-04-02T17:27:57.932415489Z",
"error": {
"code": "InternalServerError",
"message": "insufficient public IP address quota: required 2, available 0"
}
}
--------------------------------------------------------------------------------
{
msg: "failed to create HCP cluster cluster-zstream-4-19-456qg5: failed waiting for cluster=\"cluster-zstream-4-19-456qg5\" in resourcegroup=\"rg-zstream-upgrade-4-19-p762zb\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/78ee839b-27e8-45ab-9bef-ab361680e06a\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/78ee839b-27e8-45ab-9bef-ab361680e06a\",\n \"name\": \"78ee839b-27e8-45ab-9bef-ab361680e06a\",\n \"status\": \"Failed\",\n \"startTime\": \"2026-04-02T17:27:46.835065402Z\",\n \"endTime\": \"2026-04-02T17:27:57.932415489Z\",\n \"error\": {\n \"code\": \"InternalServerError\",\n \"message\": \"insufficient public IP address quota: required 2, available 0\"\n }\n}\n--------------------------------------------------------------------------------\n",
err: <*fmt.wrapError | 0xc0008d7560>{
msg: "failed waiting for cluster=\"cluster-zstream-4-19-456qg5\" in resourcegroup=\"rg-zstream-upgrade-4-19-p762zb\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/78ee839b-27e8-45ab-9bef-ab361680e06a\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/78ee839b-27e8-45ab-9bef-ab361680e06a\",\n \"name\": \"78ee839b-27e8-45ab-9bef-ab361680e06a\",\n \"status\": \"Failed\",\n \"startTime\": \"2026-04-02T17:27:46.835065402Z\",\n \"endTime\": \"2026-04-02T17:27:57.932415489Z\",\n \"error\": {\n \"code\": \"InternalServerError\",\n \"message\": \"insufficient public IP address quota: required 2, available 0\"\n }\n}\n--------------------------------------------------------------------------------\n",
err: <*exported.ResponseError | 0xc001317380>{
ErrorCode: "InternalServerError",
StatusCode: 200,
RawResponse: {
Status: "200 OK",
StatusCode: 200,
Proto: "HTTP/2.0",
ProtoMajor: 2,
ProtoMinor: 0,
Header: {
"Pragma": ["no-cache"],
"Content-Length": ["499"],
"X-Ms-Request-Id": [
"f6c45648-41ff-430b-8947-420c54510b28",
],
"X-Ms-Routing-Request-Id": [
"CANADACENTRAL:20260402T172807Z:e6630a99-063f-431a-9926-65553b1b3548",
],
"X-Cache": ["CONFIG_NOCACHE"],
"X-Ms-Operation-Identifier": [
"appId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,tenantId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,objectId=0e96dbb7-17a2-4393-b2d3-d84524e39c32/canadacentral/26acb1f6-ebb4-4c9f-9e33-8c891414cd26",
],
"X-Ms-Ratelimit-Remaining-Subscription-Global-Reads": ["3749"],
"X-Msedge-Ref": [
"Ref A: C88F82DD8BF54F1BA57C9FED21B63FDF Ref B: CHI261140514034 Ref C: 2026-04-02T17:28:07Z",
],
"Date": [
"Thu, 02 Apr 2026 17:28:07 GMT",
],
...
Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.
Learn more here: https://onsi.github.io/gomega/#adjusting-output
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:141]: Unexpected error:
<*fmt.wrapError | 0xc0008d7580>:
failed to create HCP cluster cluster-zstream-4-19-456qg5: failed waiting for cluster="cluster-zstream-4-19-456qg5" in resourcegroup="rg-zstream-upgrade-4-19-p762zb" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/78ee839b-27e8-45ab-9bef-ab361680e06a
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/78ee839b-27e8-45ab-9bef-ab361680e06a",
"name": "78ee839b-27e8-45ab-9bef-ab361680e06a",
"status": "Failed",
"startTime": "2026-04-02T17:27:46.835065402Z",
"endTime": "2026-04-02T17:27:57.932415489Z",
"error": {
"code": "InternalServerError",
"message": "insufficient public IP address quota: required 2, available 0"
}
}
--------------------------------------------------------------------------------
{
msg: "failed to create HCP cluster cluster-zstream-4-19-456qg5: failed waiting for cluster=\"cluster-zstream-4-19-456qg5\" in resourcegroup=\"rg-zstream-upgrade-4-19-p762zb\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/78ee839b-27e8-45ab-9bef-ab361680e06a\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/78ee839b-27e8-45ab-9bef-ab361680e06a\",\n \"name\": \"78ee839b-27e8-45ab-9bef-ab361680e06a\",\n \"status\": \"Failed\",\n \"startTime\": \"2026-04-02T17:27:46.835065402Z\",\n \"endTime\": \"2026-04-02T17:27:57.932415489Z\",\n \"error\": {\n \"code\": \"InternalServerError\",\n \"message\": \"insufficient public IP address quota: required 2, available 0\"\n }\n}\n--------------------------------------------------------------------------------\n",
err: <*fmt.wrapError | 0xc0008d7560>{
msg: "failed waiting for cluster=\"cluster-zstream-4-19-456qg5\" in resourcegroup=\"rg-zstream-upgrade-4-19-p762zb\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/78ee839b-27e8-45ab-9bef-ab361680e06a\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/78ee839b-27e8-45ab-9bef-ab361680e06a\",\n \"name\": \"78ee839b-27e8-45ab-9bef-ab361680e06a\",\n \"status\": \"Failed\",\n \"startTime\": \"2026-04-02T17:27:46.835065402Z\",\n \"endTime\": \"2026-04-02T17:27:57.932415489Z\",\n \"error\": {\n \"code\": \"InternalServerError\",\n \"message\": \"insufficient public IP address quota: required 2, available 0\"\n }\n}\n--------------------------------------------------------------------------------\n",
err: <*exported.ResponseError | 0xc001317380>{
ErrorCode: "InternalServerError",
StatusCode: 200,
RawResponse: {
Status: "200 OK",
StatusCode: 200,
Proto: "HTTP/2.0",
ProtoMajor: 2,
ProtoMinor: 0,
Header: {
"Pragma": ["no-cache"],
"Content-Length": ["499"],
"X-Ms-Request-Id": [
"f6c45648-41ff-430b-8947-420c54510b28",
],
"X-Ms-Routing-Request-Id": [
"CANADACENTRAL:20260402T172807Z:e6630a99-063f-431a-9926-65553b1b3548",
],
"X-Cache": ["CONFIG_NOCACHE"],
"X-Ms-Operation-Identifier": [
"appId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,tenantId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,objectId=0e96dbb7-17a2-4393-b2d3-d84524e39c32/canadacentral/26acb1f6-ebb4-4c9f-9e33-8c891414cd26",
],
"X-Ms-Ratelimit-Remaining-Subscription-Global-Reads": ["3749"],
"X-Msedge-Ref": [
"Ref A: C88F82DD8BF54F1BA57C9FED21B63FDF Ref B: CHI261140514034 Ref C: 2026-04-02T17:28:07Z",
],
"Date": [
"Thu, 02 Apr 2026 17:28:07 GMT",
],
...
Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.
Learn more here: https://onsi.github.io/gomega/#adjusting-output
occurredfail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:132]: Unexpected error:
<*fmt.wrapError | 0xc0005873e0>:
failed to create HCP cluster np-version-upgrade-cluster-qhj6bm: failed waiting for cluster="np-version-upgrade-cluster-qhj6bm" in resourcegroup="rg-np-version-upgrade-qhj6bm-hp2rjl" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/774a88cb-0063-4a09-b407-ee6a7f59a9e6
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/774a88cb-0063-4a09-b407-ee6a7f59a9e6",
"name": "774a88cb-0063-4a09-b407-ee6a7f59a9e6",
"status": "Failed",
"startTime": "2026-04-02T17:22:52.081012471Z",
"endTime": "2026-04-02T17:23:13.48665448Z",
"error": {
"code": "InternalServerError",
"message": "insufficient public IP address quota: required 2, available 0"
}
}
--------------------------------------------------------------------------------
{
msg: "failed to create HCP cluster np-version-upgrade-cluster-qhj6bm: failed waiting for cluster=\"np-version-upgrade-cluster-qhj6bm\" in resourcegroup=\"rg-np-version-upgrade-qhj6bm-hp2rjl\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/774a88cb-0063-4a09-b407-ee6a7f59a9e6\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/774a88cb-0063-4a09-b407-ee6a7f59a9e6\",\n \"name\": \"774a88cb-0063-4a09-b407-ee6a7f59a9e6\",\n \"status\": \"Failed\",\n \"startTime\": \"2026-04-02T17:22:52.081012471Z\",\n \"endTime\": \"2026-04-02T17:23:13.48665448Z\",\n \"error\": {\n \"code\": \"InternalServerError\",\n \"message\": \"insufficient public IP address quota: required 2, available 0\"\n }\n}\n--------------------------------------------------------------------------------\n",
err: <*fmt.wrapError | 0xc0005873c0>{
msg: "failed waiting for cluster=\"np-version-upgrade-cluster-qhj6bm\" in resourcegroup=\"rg-np-version-upgrade-qhj6bm-hp2rjl\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/774a88cb-0063-4a09-b407-ee6a7f59a9e6\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/774a88cb-0063-4a09-b407-ee6a7f59a9e6\",\n \"name\": \"774a88cb-0063-4a09-b407-ee6a7f59a9e6\",\n \"status\": \"Failed\",\n \"startTime\": \"2026-04-02T17:22:52.081012471Z\",\n \"endTime\": \"2026-04-02T17:23:13.48665448Z\",\n \"error\": {\n \"code\": \"InternalServerError\",\n \"message\": \"insufficient public IP address quota: required 2, available 0\"\n }\n}\n--------------------------------------------------------------------------------\n",
err: <*exported.ResponseError | 0xc0010284e0>{
ErrorCode: "InternalServerError",
StatusCode: 200,
RawResponse: {
Status: "200 OK",
StatusCode: 200,
Proto: "HTTP/2.0",
ProtoMajor: 2,
ProtoMinor: 0,
Header: {
"Cache-Control": ["no-cache"],
"Content-Length": ["498"],
"X-Ms-Correlation-Request-Id": [
"8a74d4be-8a1d-4808-936e-22bae5b4ccd3",
],
"X-Msedge-Ref": [
"Ref A: 37CA783E55BC4DE4BA0F59B834B5F490 Ref B: CHI261140504062 Ref C: 2026-04-02T17:23:22Z",
],
"Pragma": ["no-cache"],
"Expires": ["-1"],
"X-Envoy-Upstream-Service-Time": ["20"],
"X-Ms-Routing-Request-Id": [
"CANADACENTRAL:20260402T172322Z:8a74d4be-8a1d-4808-936e-22bae5b4ccd3",
],
"Strict-Transport-Security": [
"max-age=31536000; includeSubDomains",
],
"X-Cache": ["CONFIG_NOCACHE"],
"Content-Type": ["application/json"],
"X-Ms-Request-Id": [
"c674f25c-3bae-4b96-9f1e...
Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.
Learn more here: https://onsi.github.io/gomega/#adjusting-output
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:132]: Unexpected error:
<*fmt.wrapError | 0xc0005873e0>:
failed to create HCP cluster np-version-upgrade-cluster-qhj6bm: failed waiting for cluster="np-version-upgrade-cluster-qhj6bm" in resourcegroup="rg-np-version-upgrade-qhj6bm-hp2rjl" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/774a88cb-0063-4a09-b407-ee6a7f59a9e6
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/774a88cb-0063-4a09-b407-ee6a7f59a9e6",
"name": "774a88cb-0063-4a09-b407-ee6a7f59a9e6",
"status": "Failed",
"startTime": "2026-04-02T17:22:52.081012471Z",
"endTime": "2026-04-02T17:23:13.48665448Z",
"error": {
"code": "InternalServerError",
"message": "insufficient public IP address quota: required 2, available 0"
}
}
--------------------------------------------------------------------------------
{
msg: "failed to create HCP cluster np-version-upgrade-cluster-qhj6bm: failed waiting for cluster=\"np-version-upgrade-cluster-qhj6bm\" in resourcegroup=\"rg-np-version-upgrade-qhj6bm-hp2rjl\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/774a88cb-0063-4a09-b407-ee6a7f59a9e6\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/774a88cb-0063-4a09-b407-ee6a7f59a9e6\",\n \"name\": \"774a88cb-0063-4a09-b407-ee6a7f59a9e6\",\n \"status\": \"Failed\",\n \"startTime\": \"2026-04-02T17:22:52.081012471Z\",\n \"endTime\": \"2026-04-02T17:23:13.48665448Z\",\n \"error\": {\n \"code\": \"InternalServerError\",\n \"message\": \"insufficient public IP address quota: required 2, available 0\"\n }\n}\n--------------------------------------------------------------------------------\n",
err: <*fmt.wrapError | 0xc0005873c0>{
msg: "failed waiting for cluster=\"np-version-upgrade-cluster-qhj6bm\" in resourcegroup=\"rg-np-version-upgrade-qhj6bm-hp2rjl\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/774a88cb-0063-4a09-b407-ee6a7f59a9e6\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/774a88cb-0063-4a09-b407-ee6a7f59a9e6\",\n \"name\": \"774a88cb-0063-4a09-b407-ee6a7f59a9e6\",\n \"status\": \"Failed\",\n \"startTime\": \"2026-04-02T17:22:52.081012471Z\",\n \"endTime\": \"2026-04-02T17:23:13.48665448Z\",\n \"error\": {\n \"code\": \"InternalServerError\",\n \"message\": \"insufficient public IP address quota: required 2, available 0\"\n }\n}\n--------------------------------------------------------------------------------\n",
err: <*exported.ResponseError | 0xc0010284e0>{
ErrorCode: "InternalServerError",
StatusCode: 200,
RawResponse: {
Status: "200 OK",
StatusCode: 200,
Proto: "HTTP/2.0",
ProtoMajor: 2,
ProtoMinor: 0,
Header: {
"Cache-Control": ["no-cache"],
"Content-Length": ["498"],
"X-Ms-Correlation-Request-Id": [
"8a74d4be-8a1d-4808-936e-22bae5b4ccd3",
],
"X-Msedge-Ref": [
"Ref A: 37CA783E55BC4DE4BA0F59B834B5F490 Ref B: CHI261140504062 Ref C: 2026-04-02T17:23:22Z",
],
"Pragma": ["no-cache"],
"Expires": ["-1"],
"X-Envoy-Upstream-Service-Time": ["20"],
"X-Ms-Routing-Request-Id": [
"CANADACENTRAL:20260402T172322Z:8a74d4be-8a1d-4808-936e-22bae5b4ccd3",
],
"Strict-Transport-Security": [
"max-age=31536000; includeSubDomains",
],
"X-Cache": ["CONFIG_NOCACHE"],
"Content-Type": ["application/json"],
"X-Ms-Request-Id": [
"c674f25c-3bae-4b96-9f1e...
Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.
Learn more here: https://onsi.github.io/gomega/#adjusting-output
occurredContributing tests (8)
Affected runs (2)
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
ERROR CODE: InternalServerError; context ; provider Microsoft.RedHatOpenShiftfull failure pattern: ERROR CODE: InternalServerError; context ; provider Microsoft.RedHatOpenShift Signal: IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | e2e | 1 | 16.67%1 of 6 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | Show trend detailsMar 27: 0 · Mar 28: 0 · Mar 29: 0 · Mar 30: 0 · Mar 31: 0 · Apr 1: 0 · Apr 2: 6 | none | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (3)fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_ephemeral_osdisk.go:181]: Unexpected error:
<*fmt.wrapError | 0xc0012b0780>:
failed to start credential request: POST https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/ephemeral-osdisk-8kqbht/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/ephemeral-disk/requestAdminCredential
--------------------------------------------------------------------------------
RESPONSE 500: 500 Internal Server Error
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"error": {
"code": "InternalServerError",
"message": "Internal server error."
}
}
--------------------------------------------------------------------------------
{
msg: "failed to start credential request: POST https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/ephemeral-osdisk-8kqbht/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/ephemeral-disk/requestAdminCredential\n--------------------------------------------------------------------------------\nRESPONSE 500: 500 Internal Server Error\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"InternalServerError\",\n \"message\": \"Internal server error.\"\n }\n}\n--------------------------------------------------------------------------------\n",
err: <*exported.ResponseError | 0xc0012f34a0>{
ErrorCode: "InternalServerError",
StatusCode: 500,
RawResponse: {
Status: "500 Internal Server Error",
StatusCode: 500,
Proto: "HTTP/2.0",
ProtoMajor: 2,
ProtoMinor: 0,
Header: {
"Pragma": ["no-cache"],
"Expires": ["-1"],
"X-Ms-Ratelimit-Remaining-Subscription-Global-Writes": ["2999"],
"Content-Length": ["107"],
"Content-Type": ["application/json"],
"X-Ms-Error-Code": [
"InternalServerError",
],
"X-Ms-Failure-Cause": ["service"],
"X-Ms-Operation-Identifier": [
"appId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,tenantId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,objectId=0e96dbb7-17a2-4393-b2d3-d84524e39c32/uksouth/1f894e75-bf9b-4f38-b36f-086c2d1ebd50",
],
"X-Msedge-Ref": [
"Ref A: 49D6E7E0AC224A1688162351432A0445 Ref B: CH1AA2020610019 Ref C: 2026-04-02T02:19:06Z",
],
"Date": [
"Thu, 02 Apr 2026 02:19:06 GMT",
],
"X-Ms-Request-Id": [
"4c158d49-d82a-452b-867b-1015459e1703",
],
"X-Ms-Routing-Request-Id": [
"UKSOUTH:20260402T021907Z:0ff2a06f-4992-4cf9-ae72-86b939159372",
],
"Strict-Transport-Security": [
"max-age=31536000; includeSubDomains",
],
"X-Content-Type-Options": ["nosniff"],
"X-Cache": ["CONFIG_NOCACHE"],
"Cache-Control": ["no-cache"],
"X-Envoy-Upstream-Service-Time": ["399"],
"X-Ms-Ratelimit-Remaining-Subscription-Writes": ["199"],
"X-Ms-Correlation-Request-Id": [
"0ff2a06f-4992-4cf9-ae72-86b939159372",
],
},
Body: <*exported.nopClosingBytesReader | 0xc0012b06c0>{
s: [123, 10, 32, 32, 32, 32, 34, 101, 114, 114, 111, 114, 34, 58, 32, 123, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 99, 111, 100, 101, 34, 58, 32, 34, 73, 110, 116, 101, 114, 110, 97, 108, 83, 101, 114, 118, 101, 114, 69, 114, 114, 111, 114, 34, 44, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 109, 101, 115, 115, 97, 103, 101, 34, 58, 32, 34, 73, 110, 116, 101, 114, 110, 97, 108, 32, 115, 101, 114, 118, 101, 114, 32, 101, 114, 114, 111, 114, 46, 34, 10, 32, 32, 32, 32, 125, 10, 125],
i: 0,
},
ContentLength: 107,
TransferEncoding: nil,
Close: false,
Uncompressed: false,
Trailer: nil,
Request: {
Method: "POST",
URL: {
Scheme: "https",
Opaque: "",
User: nil,
Host: "manageme...
Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.
Learn more here: https://onsi.github.io/gomega/#adjusting-output
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_ephemeral_osdisk.go:181]: Unexpected error:
<*fmt.wrapError | 0xc0012b0780>:
failed to start credential request: POST https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/ephemeral-osdisk-8kqbht/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/ephemeral-disk/requestAdminCredential
--------------------------------------------------------------------------------
RESPONSE 500: 500 Internal Server Error
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"error": {
"code": "InternalServerError",
"message": "Internal server error."
}
}
--------------------------------------------------------------------------------
{
msg: "failed to start credential request: POST https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/ephemeral-osdisk-8kqbht/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/ephemeral-disk/requestAdminCredential\n--------------------------------------------------------------------------------\nRESPONSE 500: 500 Internal Server Error\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"InternalServerError\",\n \"message\": \"Internal server error.\"\n }\n}\n--------------------------------------------------------------------------------\n",
err: <*exported.ResponseError | 0xc0012f34a0>{
ErrorCode: "InternalServerError",
StatusCode: 500,
RawResponse: {
Status: "500 Internal Server Error",
StatusCode: 500,
Proto: "HTTP/2.0",
ProtoMajor: 2,
ProtoMinor: 0,
Header: {
"Pragma": ["no-cache"],
"Expires": ["-1"],
"X-Ms-Ratelimit-Remaining-Subscription-Global-Writes": ["2999"],
"Content-Length": ["107"],
"Content-Type": ["application/json"],
"X-Ms-Error-Code": [
"InternalServerError",
],
"X-Ms-Failure-Cause": ["service"],
"X-Ms-Operation-Identifier": [
"appId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,tenantId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,objectId=0e96dbb7-17a2-4393-b2d3-d84524e39c32/uksouth/1f894e75-bf9b-4f38-b36f-086c2d1ebd50",
],
"X-Msedge-Ref": [
"Ref A: 49D6E7E0AC224A1688162351432A0445 Ref B: CH1AA2020610019 Ref C: 2026-04-02T02:19:06Z",
],
"Date": [
"Thu, 02 Apr 2026 02:19:06 GMT",
],
"X-Ms-Request-Id": [
"4c158d49-d82a-452b-867b-1015459e1703",
],
"X-Ms-Routing-Request-Id": [
"UKSOUTH:20260402T021907Z:0ff2a06f-4992-4cf9-ae72-86b939159372",
],
"Strict-Transport-Security": [
"max-age=31536000; includeSubDomains",
],
"X-Content-Type-Options": ["nosniff"],
"X-Cache": ["CONFIG_NOCACHE"],
"Cache-Control": ["no-cache"],
"X-Envoy-Upstream-Service-Time": ["399"],
"X-Ms-Ratelimit-Remaining-Subscription-Writes": ["199"],
"X-Ms-Correlation-Request-Id": [
"0ff2a06f-4992-4cf9-ae72-86b939159372",
],
},
Body: <*exported.nopClosingBytesReader | 0xc0012b06c0>{
s: [123, 10, 32, 32, 32, 32, 34, 101, 114, 114, 111, 114, 34, 58, 32, 123, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 99, 111, 100, 101, 34, 58, 32, 34, 73, 110, 116, 101, 114, 110, 97, 108, 83, 101, 114, 118, 101, 114, 69, 114, 114, 111, 114, 34, 44, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 109, 101, 115, 115, 97, 103, 101, 34, 58, 32, 34, 73, 110, 116, 101, 114, 110, 97, 108, 32, 115, 101, 114, 118, 101, 114, 32, 101, 114, 114, 111, 114, 46, 34, 10, 32, 32, 32, 32, 125, 10, 125],
i: 0,
},
ContentLength: 107,
TransferEncoding: nil,
Close: false,
Uncompressed: false,
Trailer: nil,
Request: {
Method: "POST",
URL: {
Scheme: "https",
Opaque: "",
User: nil,
Host: "manageme...
Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.
Learn more here: https://onsi.github.io/gomega/#adjusting-output
occurredfail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:147]: Unexpected error:
<*fmt.wrapError | 0xc0008c23a0>:
failed to start credential request: POST https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-19-c4ph7r/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-zstream-4-19-pw4hx4/requestAdminCredential
--------------------------------------------------------------------------------
RESPONSE 500: 500 Internal Server Error
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"error": {
"code": "InternalServerError",
"message": "Internal server error."
}
}
--------------------------------------------------------------------------------
{
msg: "failed to start credential request: POST https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-19-c4ph7r/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-zstream-4-19-pw4hx4/requestAdminCredential\n--------------------------------------------------------------------------------\nRESPONSE 500: 500 Internal Server Error\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"InternalServerError\",\n \"message\": \"Internal server error.\"\n }\n}\n--------------------------------------------------------------------------------\n",
err: <*exported.ResponseError | 0xc00031b110>{
ErrorCode: "InternalServerError",
StatusCode: 500,
RawResponse: {
Status: "500 Internal Server Error",
StatusCode: 500,
Proto: "HTTP/2.0",
ProtoMajor: 2,
ProtoMinor: 0,
Header: {
"X-Ms-Routing-Request-Id": [
"UKSOUTH:20260402T022005Z:bae2f98c-c749-481d-9054-9921e081945e",
],
"X-Content-Type-Options": ["nosniff"],
"Pragma": ["no-cache"],
"X-Ms-Error-Code": [
"InternalServerError",
],
"X-Ms-Operation-Identifier": [
"appId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,tenantId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,objectId=0e96dbb7-17a2-4393-b2d3-d84524e39c32/uksouth/62d919b4-4225-4967-9df6-08b0b5351023",
],
"X-Ms-Ratelimit-Remaining-Subscription-Writes": ["199"],
"X-Ms-Ratelimit-Remaining-Subscription-Global-Writes": ["2999"],
"Strict-Transport-Security": [
"max-age=31536000; includeSubDomains",
],
"X-Cache": ["CONFIG_NOCACHE"],
"Date": [
"Thu, 02 Apr 2026 02:20:05 GMT",
],
"Cache-Control": ["no-cache"],
"Content-Type": ["application/json"],
"X-Ms-Request-Id": [
"4c5f9568-bb9b-40c3-81ad-0ca630f99d24",
],
"X-Msedge-Ref": [
"Ref A: 2552FC52616C4D5BB46364A7B7C0692E Ref B: CH1AA2020617017 Ref C: 2026-04-02T02:20:02Z",
],
"X-Envoy-Upstream-Service-Time": ["2628"],
"X-Ms-Failure-Cause": ["service"],
"Content-Length": ["107"],
"Expires": ["-1"],
"X-Ms-Correlation-Request-Id": [
"bae2f98c-c749-481d-9054-9921e081945e",
],
},
Body: <*exported.nopClosingBytesReader | 0xc0008c21e0>{
s: [123, 10, 32, 32, 32, 32, 34, 101, 114, 114, 111, 114, 34, 58, 32, 123, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 99, 111, 100, 101, 34, 58, 32, 34, 73, 110, 116, 101, 114, 110, 97, 108, 83, 101, 114, 118, 101, 114, 69, 114, 114, 111, 114, 34, 44, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 109, 101, 115, 115, 97, 103, 101, 34, 58, 32, 34, 73, 110, 116, 101, 114, 110, 97, 108, 32, 115, 101, 114, 118, 101, 114, 32, 101, 114, 114, 111, 114, 46, 34, 10, 32, 32, 32, 32, 125, 10, 125],
i: 0,
},
ContentLength: 107,
TransferEncoding: nil,
Close: false,
Uncompressed: false,
Trailer: nil,
Request: {
Method: "POST",
URL: {
Scheme: "https",
Opaque: "",
User: nil,
...
Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.
Learn more here: https://onsi.github.io/gomega/#adjusting-output
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:147]: Unexpected error:
<*fmt.wrapError | 0xc0008c23a0>:
failed to start credential request: POST https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-19-c4ph7r/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-zstream-4-19-pw4hx4/requestAdminCredential
--------------------------------------------------------------------------------
RESPONSE 500: 500 Internal Server Error
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"error": {
"code": "InternalServerError",
"message": "Internal server error."
}
}
--------------------------------------------------------------------------------
{
msg: "failed to start credential request: POST https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-19-c4ph7r/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-zstream-4-19-pw4hx4/requestAdminCredential\n--------------------------------------------------------------------------------\nRESPONSE 500: 500 Internal Server Error\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"InternalServerError\",\n \"message\": \"Internal server error.\"\n }\n}\n--------------------------------------------------------------------------------\n",
err: <*exported.ResponseError | 0xc00031b110>{
ErrorCode: "InternalServerError",
StatusCode: 500,
RawResponse: {
Status: "500 Internal Server Error",
StatusCode: 500,
Proto: "HTTP/2.0",
ProtoMajor: 2,
ProtoMinor: 0,
Header: {
"X-Ms-Routing-Request-Id": [
"UKSOUTH:20260402T022005Z:bae2f98c-c749-481d-9054-9921e081945e",
],
"X-Content-Type-Options": ["nosniff"],
"Pragma": ["no-cache"],
"X-Ms-Error-Code": [
"InternalServerError",
],
"X-Ms-Operation-Identifier": [
"appId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,tenantId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,objectId=0e96dbb7-17a2-4393-b2d3-d84524e39c32/uksouth/62d919b4-4225-4967-9df6-08b0b5351023",
],
"X-Ms-Ratelimit-Remaining-Subscription-Writes": ["199"],
"X-Ms-Ratelimit-Remaining-Subscription-Global-Writes": ["2999"],
"Strict-Transport-Security": [
"max-age=31536000; includeSubDomains",
],
"X-Cache": ["CONFIG_NOCACHE"],
"Date": [
"Thu, 02 Apr 2026 02:20:05 GMT",
],
"Cache-Control": ["no-cache"],
"Content-Type": ["application/json"],
"X-Ms-Request-Id": [
"4c5f9568-bb9b-40c3-81ad-0ca630f99d24",
],
"X-Msedge-Ref": [
"Ref A: 2552FC52616C4D5BB46364A7B7C0692E Ref B: CH1AA2020617017 Ref C: 2026-04-02T02:20:02Z",
],
"X-Envoy-Upstream-Service-Time": ["2628"],
"X-Ms-Failure-Cause": ["service"],
"Content-Length": ["107"],
"Expires": ["-1"],
"X-Ms-Correlation-Request-Id": [
"bae2f98c-c749-481d-9054-9921e081945e",
],
},
Body: <*exported.nopClosingBytesReader | 0xc0008c21e0>{
s: [123, 10, 32, 32, 32, 32, 34, 101, 114, 114, 111, 114, 34, 58, 32, 123, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 99, 111, 100, 101, 34, 58, 32, 34, 73, 110, 116, 101, 114, 110, 97, 108, 83, 101, 114, 118, 101, 114, 69, 114, 114, 111, 114, 34, 44, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 109, 101, 115, 115, 97, 103, 101, 34, 58, 32, 34, 73, 110, 116, 101, 114, 110, 97, 108, 32, 115, 101, 114, 118, 101, 114, 32, 101, 114, 114, 111, 114, 46, 34, 10, 32, 32, 32, 32, 125, 10, 125],
i: 0,
},
ContentLength: 107,
TransferEncoding: nil,
Close: false,
Uncompressed: false,
Trailer: nil,
Request: {
Method: "POST",
URL: {
Scheme: "https",
Opaque: "",
User: nil,
...
Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.
Learn more here: https://onsi.github.io/gomega/#adjusting-output
occurredfail [github.com/Azure/ARO-HCP/test/e2e/external_auth_list_and_verify.go:118]: Unexpected error:
<*fmt.wrapError | 0xc001418640>:
failed creating external auth "ea-list" in resourcegroup="ea-list-sqv9wn" for cluster="ea-list4lvwkr": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/ea-list-sqv9wn/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/ea-list4lvwkr/externalAuths/ea-list
--------------------------------------------------------------------------------
RESPONSE 500: 500 Internal Server Error
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"error": {
"code": "InternalServerError",
"message": "Internal server error."
}
}
--------------------------------------------------------------------------------
{
msg: "failed creating external auth \"ea-list\" in resourcegroup=\"ea-list-sqv9wn\" for cluster=\"ea-list4lvwkr\": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/ea-list-sqv9wn/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/ea-list4lvwkr/externalAuths/ea-list\n--------------------------------------------------------------------------------\nRESPONSE 500: 500 Internal Server Error\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"InternalServerError\",\n \"message\": \"Internal server error.\"\n }\n}\n--------------------------------------------------------------------------------\n",
err: <*exported.ResponseError | 0xc001366990>{
ErrorCode: "InternalServerError",
StatusCode: 500,
RawResponse: {
Status: "500 Internal Server Error",
StatusCode: 500,
Proto: "HTTP/2.0",
ProtoMajor: 2,
ProtoMinor: 0,
Header: {
"Expires": ["-1"],
"X-Ms-Request-Id": [
"f4e61ff2-b02c-4f78-930a-516f5367173c",
],
"X-Ms-Ratelimit-Remaining-Subscription-Global-Writes": ["2999"],
"Date": [
"Thu, 02 Apr 2026 02:10:05 GMT",
],
"X-Ms-Error-Code": [
"InternalServerError",
],
"X-Content-Type-Options": ["nosniff"],
"X-Msedge-Ref": [
"Ref A: 04AE8E5AF86B478685A5B528872AB82B Ref B: CHI261140505062 Ref C: 2026-04-02T02:09:59Z",
],
"Content-Length": ["107"],
"X-Ms-Failure-Cause": ["service"],
"X-Ms-Operation-Identifier": [
"appId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,tenantId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,objectId=0e96dbb7-17a2-4393-b2d3-d84524e39c32/uksouth/786cdfbd-1a09-42c1-be7d-896a3083330d",
],
"X-Ms-Correlation-Request-Id": [
"4be39819-497a-401e-a7b1-442d6a3bb89a",
],
"X-Envoy-Upstream-Service-Time": ["7051"],
"X-Ms-Ratelimit-Remaining-Subscription-Writes": ["199"],
"X-Ms-Routing-Request-Id": [
"UKSOUTH:20260402T021006Z:4be39819-497a-401e-a7b1-442d6a3bb89a",
],
"Strict-Transport-Security": [
"max-age=31536000; includeSubDomains",
],
"X-Cache": ["CONFIG_NOCACHE"],
"Cache-Control": ["no-cache"],
"Pragma": ["no-cache"],
"Content-Type": ["application/json"],
},
Body: <*exported.nopClosingBytesReader | 0xc001418500>{
s: [123, 10, 32, 32, 32, 32, 34, 101, 114, 114, 111, 114, 34, 58, 32, 123, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 99, 111, 100, 101, 34, 58, 32, 34, 73, 110, 116, 101, 114, 110, 97, 108, 83, 101, 114, 118, 101, 114, 69, 114, 114, 111, 114, 34, 44, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 109, 101, 115, 115, 97, 103, 101, 34, 58, 32, 34, 73, 110, 116, 101, 114, 110, 97, 108, 32, 115, 101, 114, 118, 101, 114, 32, 101, 114, 114, 111, 114, 46, 34, 10, 32, 32, 32, 32, 125, 10, 125],
i: 0,
},
ContentLength: 107,
TransferEncoding: nil,
Close: false,
Uncompressed: false,
Trailer: nil,
Request: {
Method: "PUT",
URL: {
Scheme: "https",
Opaque: "",
...
Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.
Learn more here: https://onsi.github.io/gomega/#adjusting-output
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_list_and_verify.go:118]: Unexpected error:
<*fmt.wrapError | 0xc001418640>:
failed creating external auth "ea-list" in resourcegroup="ea-list-sqv9wn" for cluster="ea-list4lvwkr": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/ea-list-sqv9wn/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/ea-list4lvwkr/externalAuths/ea-list
--------------------------------------------------------------------------------
RESPONSE 500: 500 Internal Server Error
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"error": {
"code": "InternalServerError",
"message": "Internal server error."
}
}
--------------------------------------------------------------------------------
{
msg: "failed creating external auth \"ea-list\" in resourcegroup=\"ea-list-sqv9wn\" for cluster=\"ea-list4lvwkr\": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/ea-list-sqv9wn/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/ea-list4lvwkr/externalAuths/ea-list\n--------------------------------------------------------------------------------\nRESPONSE 500: 500 Internal Server Error\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"InternalServerError\",\n \"message\": \"Internal server error.\"\n }\n}\n--------------------------------------------------------------------------------\n",
err: <*exported.ResponseError | 0xc001366990>{
ErrorCode: "InternalServerError",
StatusCode: 500,
RawResponse: {
Status: "500 Internal Server Error",
StatusCode: 500,
Proto: "HTTP/2.0",
ProtoMajor: 2,
ProtoMinor: 0,
Header: {
"Expires": ["-1"],
"X-Ms-Request-Id": [
"f4e61ff2-b02c-4f78-930a-516f5367173c",
],
"X-Ms-Ratelimit-Remaining-Subscription-Global-Writes": ["2999"],
"Date": [
"Thu, 02 Apr 2026 02:10:05 GMT",
],
"X-Ms-Error-Code": [
"InternalServerError",
],
"X-Content-Type-Options": ["nosniff"],
"X-Msedge-Ref": [
"Ref A: 04AE8E5AF86B478685A5B528872AB82B Ref B: CHI261140505062 Ref C: 2026-04-02T02:09:59Z",
],
"Content-Length": ["107"],
"X-Ms-Failure-Cause": ["service"],
"X-Ms-Operation-Identifier": [
"appId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,tenantId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,objectId=0e96dbb7-17a2-4393-b2d3-d84524e39c32/uksouth/786cdfbd-1a09-42c1-be7d-896a3083330d",
],
"X-Ms-Correlation-Request-Id": [
"4be39819-497a-401e-a7b1-442d6a3bb89a",
],
"X-Envoy-Upstream-Service-Time": ["7051"],
"X-Ms-Ratelimit-Remaining-Subscription-Writes": ["199"],
"X-Ms-Routing-Request-Id": [
"UKSOUTH:20260402T021006Z:4be39819-497a-401e-a7b1-442d6a3bb89a",
],
"Strict-Transport-Security": [
"max-age=31536000; includeSubDomains",
],
"X-Cache": ["CONFIG_NOCACHE"],
"Cache-Control": ["no-cache"],
"Pragma": ["no-cache"],
"Content-Type": ["application/json"],
},
Body: <*exported.nopClosingBytesReader | 0xc001418500>{
s: [123, 10, 32, 32, 32, 32, 34, 101, 114, 114, 111, 114, 34, 58, 32, 123, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 99, 111, 100, 101, 34, 58, 32, 34, 73, 110, 116, 101, 114, 110, 97, 108, 83, 101, 114, 118, 101, 114, 69, 114, 114, 111, 114, 34, 44, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 109, 101, 115, 115, 97, 103, 101, 34, 58, 32, 34, 73, 110, 116, 101, 114, 110, 97, 108, 32, 115, 101, 114, 118, 101, 114, 32, 101, 114, 114, 111, 114, 46, 34, 10, 32, 32, 32, 32, 125, 10, 125],
i: 0,
},
ContentLength: 107,
TransferEncoding: nil,
Close: false,
Uncompressed: false,
Trailer: nil,
Request: {
Method: "PUT",
URL: {
Scheme: "https",
Opaque: "",
...
Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.
Learn more here: https://onsi.github.io/gomega/#adjusting-output
occurredContributing tests (6)
Affected runs (1)
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
ERROR CODE: ServiceUnavailable; detail message Cluster installation has failed due to unavailable capacity. While we are bringing on additional capacity, try your cluster installa...full failure pattern: ERROR CODE: ServiceUnavailable; detail message Cluster installation has failed due to unavailable capacity. While we are bringing on additional capacity, try your cluster installation in another region or try again later; provider Microsoft.RedHatOpenShift Signal: IndeterminateSignal: Indeterminate — seen in 2 prior week(s); active 1 day(s) | e2e | 1 | 16.67%1 of 6 job runs affected | IndeterminateSignal: Indeterminate — seen in 2 prior week(s); active 1 day(s) | Show trend detailsMar 27: 0 · Mar 28: 0 · Mar 29: 0 · Mar 30: 0 · Mar 31: 0 · Apr 1: 0 · Apr 2: 3 | none | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (3)fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:84]: Unexpected error:
<*fmt.wrapError | 0xc00075c1e0>:
failed to create HCP cluster patch-name-cluster: failed starting cluster creation "patch-name-cluster" in resourcegroup="patch-name-wvg7xj": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-name-wvg7xj/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-name-cluster
--------------------------------------------------------------------------------
RESPONSE 503: 503 Service Unavailable
ERROR CODE: ServiceUnavailable
--------------------------------------------------------------------------------
{
"error": {
"code": "ServiceUnavailable",
"message": "Cluster installation has failed due to unavailable capacity. While we are bringing on additional capacity, try your cluster installation in another region or try again later"
}
}
--------------------------------------------------------------------------------
{
msg: "failed to create HCP cluster patch-name-cluster: failed starting cluster creation \"patch-name-cluster\" in resourcegroup=\"patch-name-wvg7xj\": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-name-wvg7xj/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-name-cluster\n--------------------------------------------------------------------------------\nRESPONSE 503: 503 Service Unavailable\nERROR CODE: ServiceUnavailable\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"ServiceUnavailable\",\n \"message\": \"Cluster installation has failed due to unavailable capacity. While we are bringing on additional capacity, try your cluster installation in another region or try again later\"\n }\n}\n--------------------------------------------------------------------------------\n",
err: <*fmt.wrapError | 0xc00075c1c0>{
msg: "failed starting cluster creation \"patch-name-cluster\" in resourcegroup=\"patch-name-wvg7xj\": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-name-wvg7xj/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-name-cluster\n--------------------------------------------------------------------------------\nRESPONSE 503: 503 Service Unavailable\nERROR CODE: ServiceUnavailable\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"ServiceUnavailable\",\n \"message\": \"Cluster installation has failed due to unavailable capacity. While we are bringing on additional capacity, try your cluster installation in another region or try again later\"\n }\n}\n--------------------------------------------------------------------------------\n",
err: <*exported.ResponseError | 0xc00004d2f0>{
ErrorCode: "ServiceUnavailable",
StatusCode: 503,
RawResponse: {
Status: "503 Service Unavailable",
StatusCode: 503,
Proto: "HTTP/2.0",
ProtoMajor: 2,
ProtoMinor: 0,
Header: {
"Cache-Control": ["no-cache"],
"Pragma": ["no-cache"],
"X-Envoy-Upstream-Service-Time": ["2878"],
"X-Ms-Ratelimit-Remaining-Subscription-Writes": ["199"],
"X-Ms-Correlation-Request-Id": [
"0b9741c0-9777-4781-b4fb-bf1afce41505",
],
"Retry-After": ["59"],
"X-Ms-Ratelimit-Remaining-Subscription-Global-Writes": ["2999"],
"Date": [
"Thu, 02 Apr 2026 08:57:47 GMT",
],
"Content-Length": ["257"],
"Content-Type": ["application/json"],
"X-Ms-Error-Code": ["ServiceUnavailable"],
"X-Ms-Request-Id": [
"87531355-f28f-4c12-bf8b-c7f8a31db6ce",
],
"X-Ms-Operation-Identifier": [
"appId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,tenantId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,objectId=0e96dbb7-17a2-4393-b2d3-d84524e39c32/uksouth/7ad18b62-8874-4d58-81d6-01921cd42fc7",
],
"X-Ms-Routing-Request-Id": [
"UKSOUTH:20260402T085747Z:0b9741c0-9777-4781-b4fb-bf1afce41505",
],
"Strict-Transport-Security": [
"max-age=31536000; includeSubDomains",
],
"X-Cache": ["CONFIG_NOCACHE"],
"Expires": ["-1"],
"X-Ms-...
Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.
Learn more here: https://onsi.github.io/gomega/#adjusting-output
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:84]: Unexpected error:
<*fmt.wrapError | 0xc00075c1e0>:
failed to create HCP cluster patch-name-cluster: failed starting cluster creation "patch-name-cluster" in resourcegroup="patch-name-wvg7xj": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-name-wvg7xj/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-name-cluster
--------------------------------------------------------------------------------
RESPONSE 503: 503 Service Unavailable
ERROR CODE: ServiceUnavailable
--------------------------------------------------------------------------------
{
"error": {
"code": "ServiceUnavailable",
"message": "Cluster installation has failed due to unavailable capacity. While we are bringing on additional capacity, try your cluster installation in another region or try again later"
}
}
--------------------------------------------------------------------------------
{
msg: "failed to create HCP cluster patch-name-cluster: failed starting cluster creation \"patch-name-cluster\" in resourcegroup=\"patch-name-wvg7xj\": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-name-wvg7xj/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-name-cluster\n--------------------------------------------------------------------------------\nRESPONSE 503: 503 Service Unavailable\nERROR CODE: ServiceUnavailable\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"ServiceUnavailable\",\n \"message\": \"Cluster installation has failed due to unavailable capacity. While we are bringing on additional capacity, try your cluster installation in another region or try again later\"\n }\n}\n--------------------------------------------------------------------------------\n",
err: <*fmt.wrapError | 0xc00075c1c0>{
msg: "failed starting cluster creation \"patch-name-cluster\" in resourcegroup=\"patch-name-wvg7xj\": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-name-wvg7xj/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-name-cluster\n--------------------------------------------------------------------------------\nRESPONSE 503: 503 Service Unavailable\nERROR CODE: ServiceUnavailable\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"ServiceUnavailable\",\n \"message\": \"Cluster installation has failed due to unavailable capacity. While we are bringing on additional capacity, try your cluster installation in another region or try again later\"\n }\n}\n--------------------------------------------------------------------------------\n",
err: <*exported.ResponseError | 0xc00004d2f0>{
ErrorCode: "ServiceUnavailable",
StatusCode: 503,
RawResponse: {
Status: "503 Service Unavailable",
StatusCode: 503,
Proto: "HTTP/2.0",
ProtoMajor: 2,
ProtoMinor: 0,
Header: {
"Cache-Control": ["no-cache"],
"Pragma": ["no-cache"],
"X-Envoy-Upstream-Service-Time": ["2878"],
"X-Ms-Ratelimit-Remaining-Subscription-Writes": ["199"],
"X-Ms-Correlation-Request-Id": [
"0b9741c0-9777-4781-b4fb-bf1afce41505",
],
"Retry-After": ["59"],
"X-Ms-Ratelimit-Remaining-Subscription-Global-Writes": ["2999"],
"Date": [
"Thu, 02 Apr 2026 08:57:47 GMT",
],
"Content-Length": ["257"],
"Content-Type": ["application/json"],
"X-Ms-Error-Code": ["ServiceUnavailable"],
"X-Ms-Request-Id": [
"87531355-f28f-4c12-bf8b-c7f8a31db6ce",
],
"X-Ms-Operation-Identifier": [
"appId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,tenantId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,objectId=0e96dbb7-17a2-4393-b2d3-d84524e39c32/uksouth/7ad18b62-8874-4d58-81d6-01921cd42fc7",
],
"X-Ms-Routing-Request-Id": [
"UKSOUTH:20260402T085747Z:0b9741c0-9777-4781-b4fb-bf1afce41505",
],
"Strict-Transport-Security": [
"max-age=31536000; includeSubDomains",
],
"X-Cache": ["CONFIG_NOCACHE"],
"Expires": ["-1"],
"X-Ms-...
Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.
Learn more here: https://onsi.github.io/gomega/#adjusting-output
occurredfail [github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:123]: Unexpected error:
<*fmt.wrapError | 0xc0004e6700>:
failed starting cluster creation "basic-hcp-cluster2" in resourcegroup="customer-rg-p6dr85": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/customer-rg-p6dr85/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/basic-hcp-cluster2
--------------------------------------------------------------------------------
RESPONSE 503: 503 Service Unavailable
ERROR CODE: ServiceUnavailable
--------------------------------------------------------------------------------
{
"error": {
"code": "ServiceUnavailable",
"message": "Cluster installation has failed due to unavailable capacity. While we are bringing on additional capacity, try your cluster installation in another region or try again later"
}
}
--------------------------------------------------------------------------------
{
msg: "failed starting cluster creation \"basic-hcp-cluster2\" in resourcegroup=\"customer-rg-p6dr85\": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/customer-rg-p6dr85/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/basic-hcp-cluster2\n--------------------------------------------------------------------------------\nRESPONSE 503: 503 Service Unavailable\nERROR CODE: ServiceUnavailable\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"ServiceUnavailable\",\n \"message\": \"Cluster installation has failed due to unavailable capacity. While we are bringing on additional capacity, try your cluster installation in another region or try again later\"\n }\n}\n--------------------------------------------------------------------------------\n",
err: <*exported.ResponseError | 0xc001402630>{
ErrorCode: "ServiceUnavailable",
StatusCode: 503,
RawResponse: {
Status: "503 Service Unavailable",
StatusCode: 503,
Proto: "HTTP/2.0",
ProtoMajor: 2,
ProtoMinor: 0,
Header: {
"Cache-Control": ["no-cache"],
"Content-Type": ["application/json"],
"X-Ms-Request-Id": [
"981e3b54-276e-45f0-a48f-b05f015c79dc",
],
"X-Envoy-Upstream-Service-Time": ["8461"],
"X-Ms-Failure-Cause": ["service"],
"X-Ms-Operation-Identifier": [
"appId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,tenantId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,objectId=0e96dbb7-17a2-4393-b2d3-d84524e39c32/uksouth/d750dcff-9cd4-44ec-8a65-6f1b69e57a9d",
],
"X-Ms-Correlation-Request-Id": [
"45bdda71-4b5b-4b9b-9fd7-4df364bf7808",
],
"X-Ms-Routing-Request-Id": [
"UKSOUTH:20260402T085811Z:45bdda71-4b5b-4b9b-9fd7-4df364bf7808",
],
"Pragma": ["no-cache"],
"Content-Length": ["257"],
"Expires": ["-1"],
"X-Ms-Error-Code": ["ServiceUnavailable"],
"X-Ms-Ratelimit-Remaining-Subscription-Writes": ["199"],
"X-Cache": ["CONFIG_NOCACHE"],
"X-Msedge-Ref": [
"Ref A: C36632182EBE4B8EB52F9106BF8C5BB2 Ref B: CHI261140502029 Ref C: 2026-04-02T08:58:02Z",
],
"Retry-After": ["59"],
"X-Ms-Ratelimit-Remaining-Subscription-Global-Writes": ["2999"],
"Strict-Transport-Security": [
"max-age=31536000; includeSubDomains",
],
"X-Content-Type-Options": ["nosniff"],
"Date": [
"Thu, 02 Apr 2026 08:58:11 GMT",
],
},
Body: <*exported.nopClosingBytesReader | 0xc0004e6480>{
s: [123, 10, 32, 32, 32, 32, 34, 101, 114, 114, 111, 114, 34, 58, 32, 123, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 99, 111, 100, 101, 34, 58, 32, 34, 83, 101, 114, 118, 105, 99, 101, 85, 110, 97, 118, 97, 105, 108, 97, 98, 108, 101, 34, 44, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 109, 101, 115, 115, 97, 103, 101, 34, 58, 32, 34, 67, 108, 117, 115, 116, 101, 114, 32, 105, 110, 115, 116, 97, 108, 108, 97, 116, 105, 111, 110, 32, 104, 97, 115, 32, 102, 97, 105, 108, 101, 100, 32, 100, 117, 101, 32, 116, 111, 32, 117, 110, 97, 118, 97, 105, 108, 97, 98, 108, 101, 32, 99, 97, 112, 97, 99, 105, 116, 121, 46, 32, 87, 104, 105, 108, 101, 32, 119, 101, 32, 97, 114, 101, 32, 98, 114, 105, 110, 103, 105, 110, 103, 32, 111, 110, 32, 97, 100, 100, 105, 116, 105, 111, ...
Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.
Learn more here: https://onsi.github.io/gomega/#adjusting-output
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:123]: Unexpected error:
<*fmt.wrapError | 0xc0004e6700>:
failed starting cluster creation "basic-hcp-cluster2" in resourcegroup="customer-rg-p6dr85": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/customer-rg-p6dr85/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/basic-hcp-cluster2
--------------------------------------------------------------------------------
RESPONSE 503: 503 Service Unavailable
ERROR CODE: ServiceUnavailable
--------------------------------------------------------------------------------
{
"error": {
"code": "ServiceUnavailable",
"message": "Cluster installation has failed due to unavailable capacity. While we are bringing on additional capacity, try your cluster installation in another region or try again later"
}
}
--------------------------------------------------------------------------------
{
msg: "failed starting cluster creation \"basic-hcp-cluster2\" in resourcegroup=\"customer-rg-p6dr85\": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/customer-rg-p6dr85/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/basic-hcp-cluster2\n--------------------------------------------------------------------------------\nRESPONSE 503: 503 Service Unavailable\nERROR CODE: ServiceUnavailable\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"ServiceUnavailable\",\n \"message\": \"Cluster installation has failed due to unavailable capacity. While we are bringing on additional capacity, try your cluster installation in another region or try again later\"\n }\n}\n--------------------------------------------------------------------------------\n",
err: <*exported.ResponseError | 0xc001402630>{
ErrorCode: "ServiceUnavailable",
StatusCode: 503,
RawResponse: {
Status: "503 Service Unavailable",
StatusCode: 503,
Proto: "HTTP/2.0",
ProtoMajor: 2,
ProtoMinor: 0,
Header: {
"Cache-Control": ["no-cache"],
"Content-Type": ["application/json"],
"X-Ms-Request-Id": [
"981e3b54-276e-45f0-a48f-b05f015c79dc",
],
"X-Envoy-Upstream-Service-Time": ["8461"],
"X-Ms-Failure-Cause": ["service"],
"X-Ms-Operation-Identifier": [
"appId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,tenantId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,objectId=0e96dbb7-17a2-4393-b2d3-d84524e39c32/uksouth/d750dcff-9cd4-44ec-8a65-6f1b69e57a9d",
],
"X-Ms-Correlation-Request-Id": [
"45bdda71-4b5b-4b9b-9fd7-4df364bf7808",
],
"X-Ms-Routing-Request-Id": [
"UKSOUTH:20260402T085811Z:45bdda71-4b5b-4b9b-9fd7-4df364bf7808",
],
"Pragma": ["no-cache"],
"Content-Length": ["257"],
"Expires": ["-1"],
"X-Ms-Error-Code": ["ServiceUnavailable"],
"X-Ms-Ratelimit-Remaining-Subscription-Writes": ["199"],
"X-Cache": ["CONFIG_NOCACHE"],
"X-Msedge-Ref": [
"Ref A: C36632182EBE4B8EB52F9106BF8C5BB2 Ref B: CHI261140502029 Ref C: 2026-04-02T08:58:02Z",
],
"Retry-After": ["59"],
"X-Ms-Ratelimit-Remaining-Subscription-Global-Writes": ["2999"],
"Strict-Transport-Security": [
"max-age=31536000; includeSubDomains",
],
"X-Content-Type-Options": ["nosniff"],
"Date": [
"Thu, 02 Apr 2026 08:58:11 GMT",
],
},
Body: <*exported.nopClosingBytesReader | 0xc0004e6480>{
s: [123, 10, 32, 32, 32, 32, 34, 101, 114, 114, 111, 114, 34, 58, 32, 123, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 99, 111, 100, 101, 34, 58, 32, 34, 83, 101, 114, 118, 105, 99, 101, 85, 110, 97, 118, 97, 105, 108, 97, 98, 108, 101, 34, 44, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 109, 101, 115, 115, 97, 103, 101, 34, 58, 32, 34, 67, 108, 117, 115, 116, 101, 114, 32, 105, 110, 115, 116, 97, 108, 108, 97, 116, 105, 111, 110, 32, 104, 97, 115, 32, 102, 97, 105, 108, 101, 100, 32, 100, 117, 101, 32, 116, 111, 32, 117, 110, 97, 118, 97, 105, 108, 97, 98, 108, 101, 32, 99, 97, 112, 97, 99, 105, 116, 121, 46, 32, 87, 104, 105, 108, 101, 32, 119, 101, 32, 97, 114, 101, 32, 98, 114, 105, 110, 103, 105, 110, 103, 32, 111, 110, 32, 97, 100, 100, 105, 116, 105, 111, ...
Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.
Learn more here: https://onsi.github.io/gomega/#adjusting-output
occurredfail [github.com/Azure/ARO-HCP/test/e2e/cluster_nsg_subnet_reuse.go:106]: Unexpected error:
<*fmt.wrapError | 0xc000880260>:
failed to create HCP cluster basic-cluster: failed starting cluster creation "basic-cluster" in resourcegroup="rg-cluster-nsg-subnet-reuse-6tff8z": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cluster-nsg-subnet-reuse-6tff8z/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/basic-cluster
--------------------------------------------------------------------------------
RESPONSE 503: 503 Service Unavailable
ERROR CODE: ServiceUnavailable
--------------------------------------------------------------------------------
{
"error": {
"code": "ServiceUnavailable",
"message": "Cluster installation has failed due to unavailable capacity. While we are bringing on additional capacity, try your cluster installation in another region or try again later"
}
}
--------------------------------------------------------------------------------
{
msg: "failed to create HCP cluster basic-cluster: failed starting cluster creation \"basic-cluster\" in resourcegroup=\"rg-cluster-nsg-subnet-reuse-6tff8z\": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cluster-nsg-subnet-reuse-6tff8z/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/basic-cluster\n--------------------------------------------------------------------------------\nRESPONSE 503: 503 Service Unavailable\nERROR CODE: ServiceUnavailable\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"ServiceUnavailable\",\n \"message\": \"Cluster installation has failed due to unavailable capacity. While we are bringing on additional capacity, try your cluster installation in another region or try again later\"\n }\n}\n--------------------------------------------------------------------------------\n",
err: <*fmt.wrapError | 0xc000880200>{
msg: "failed starting cluster creation \"basic-cluster\" in resourcegroup=\"rg-cluster-nsg-subnet-reuse-6tff8z\": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cluster-nsg-subnet-reuse-6tff8z/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/basic-cluster\n--------------------------------------------------------------------------------\nRESPONSE 503: 503 Service Unavailable\nERROR CODE: ServiceUnavailable\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"ServiceUnavailable\",\n \"message\": \"Cluster installation has failed due to unavailable capacity. While we are bringing on additional capacity, try your cluster installation in another region or try again later\"\n }\n}\n--------------------------------------------------------------------------------\n",
err: <*exported.ResponseError | 0xc0012ca1b0>{
ErrorCode: "ServiceUnavailable",
StatusCode: 503,
RawResponse: {
Status: "503 Service Unavailable",
StatusCode: 503,
Proto: "HTTP/2.0",
ProtoMajor: 2,
ProtoMinor: 0,
Header: {
"X-Ms-Request-Id": [
"2a87bdad-8340-4cda-a740-2d7cc278c733",
],
"X-Ms-Correlation-Request-Id": [
"294bd1e5-7abf-4f05-a3c2-931fa8a72c4e",
],
"X-Ms-Routing-Request-Id": [
"UKSOUTH:20260402T085750Z:294bd1e5-7abf-4f05-a3c2-931fa8a72c4e",
],
"Strict-Transport-Security": [
"max-age=31536000; includeSubDomains",
],
"Date": [
"Thu, 02 Apr 2026 08:57:50 GMT",
],
"Content-Type": ["application/json"],
"X-Envoy-Upstream-Service-Time": ["2649"],
"X-Ms-Operation-Identifier": [
"appId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,tenantId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,objectId=0e96dbb7-17a2-4393-b2d3-d84524e39c32/uksouth/cc9e8570-fcc2-4ba9-9d49-08bb67825537",
],
"X-Ms-Ratelimit-Remaining-Subscription-Writes": ["199"],
"X-Content-Type-Options": ["nosniff"],
"Content-Length": ["257"],
"Retry-After": ["59"],
"X-Ms-Ratelimit-Remaining-Subscription-Global-Writes": ["2999"],
"X-Cache": ["CONFIG_NOCACHE"],
"Cache-Control": ["no-cache"],
"Pragma": ["no-cache"],
"X-Ms-Fail...
Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.
Learn more here: https://onsi.github.io/gomega/#adjusting-output
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_nsg_subnet_reuse.go:106]: Unexpected error:
<*fmt.wrapError | 0xc000880260>:
failed to create HCP cluster basic-cluster: failed starting cluster creation "basic-cluster" in resourcegroup="rg-cluster-nsg-subnet-reuse-6tff8z": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cluster-nsg-subnet-reuse-6tff8z/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/basic-cluster
--------------------------------------------------------------------------------
RESPONSE 503: 503 Service Unavailable
ERROR CODE: ServiceUnavailable
--------------------------------------------------------------------------------
{
"error": {
"code": "ServiceUnavailable",
"message": "Cluster installation has failed due to unavailable capacity. While we are bringing on additional capacity, try your cluster installation in another region or try again later"
}
}
--------------------------------------------------------------------------------
{
msg: "failed to create HCP cluster basic-cluster: failed starting cluster creation \"basic-cluster\" in resourcegroup=\"rg-cluster-nsg-subnet-reuse-6tff8z\": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cluster-nsg-subnet-reuse-6tff8z/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/basic-cluster\n--------------------------------------------------------------------------------\nRESPONSE 503: 503 Service Unavailable\nERROR CODE: ServiceUnavailable\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"ServiceUnavailable\",\n \"message\": \"Cluster installation has failed due to unavailable capacity. While we are bringing on additional capacity, try your cluster installation in another region or try again later\"\n }\n}\n--------------------------------------------------------------------------------\n",
err: <*fmt.wrapError | 0xc000880200>{
msg: "failed starting cluster creation \"basic-cluster\" in resourcegroup=\"rg-cluster-nsg-subnet-reuse-6tff8z\": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cluster-nsg-subnet-reuse-6tff8z/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/basic-cluster\n--------------------------------------------------------------------------------\nRESPONSE 503: 503 Service Unavailable\nERROR CODE: ServiceUnavailable\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"ServiceUnavailable\",\n \"message\": \"Cluster installation has failed due to unavailable capacity. While we are bringing on additional capacity, try your cluster installation in another region or try again later\"\n }\n}\n--------------------------------------------------------------------------------\n",
err: <*exported.ResponseError | 0xc0012ca1b0>{
ErrorCode: "ServiceUnavailable",
StatusCode: 503,
RawResponse: {
Status: "503 Service Unavailable",
StatusCode: 503,
Proto: "HTTP/2.0",
ProtoMajor: 2,
ProtoMinor: 0,
Header: {
"X-Ms-Request-Id": [
"2a87bdad-8340-4cda-a740-2d7cc278c733",
],
"X-Ms-Correlation-Request-Id": [
"294bd1e5-7abf-4f05-a3c2-931fa8a72c4e",
],
"X-Ms-Routing-Request-Id": [
"UKSOUTH:20260402T085750Z:294bd1e5-7abf-4f05-a3c2-931fa8a72c4e",
],
"Strict-Transport-Security": [
"max-age=31536000; includeSubDomains",
],
"Date": [
"Thu, 02 Apr 2026 08:57:50 GMT",
],
"Content-Type": ["application/json"],
"X-Envoy-Upstream-Service-Time": ["2649"],
"X-Ms-Operation-Identifier": [
"appId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,tenantId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,objectId=0e96dbb7-17a2-4393-b2d3-d84524e39c32/uksouth/cc9e8570-fcc2-4ba9-9d49-08bb67825537",
],
"X-Ms-Ratelimit-Remaining-Subscription-Writes": ["199"],
"X-Content-Type-Options": ["nosniff"],
"Content-Length": ["257"],
"Retry-After": ["59"],
"X-Ms-Ratelimit-Remaining-Subscription-Global-Writes": ["2999"],
"X-Cache": ["CONFIG_NOCACHE"],
"Cache-Control": ["no-cache"],
"Pragma": ["no-cache"],
"X-Ms-Fail...
Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.
Learn more here: https://onsi.github.io/gomega/#adjusting-output
occurredContributing tests (3)
Affected runs (1)
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
ERROR CODE: LockContention; detail message Failed to acquire lock: timed out; provider Microsoft.RedHatOpenShiftfull failure pattern: ERROR CODE: LockContention; detail message Failed to acquire lock: timed out; provider Microsoft.RedHatOpenShift Signal: IndeterminateSignal: Indeterminate — seen in 2 prior week(s); active 2 day(s) | e2e | 1 | 16.67%1 of 6 job runs affected | IndeterminateSignal: Indeterminate — seen in 2 prior week(s); active 2 day(s) | Show trend detailsMar 27: 0 · Mar 28: 0 · Mar 29: 0 · Mar 30: 5 · Mar 31: 0 · Apr 1: 0 · Apr 2: 2 | none | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (2)fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:170]: Unexpected error:
<*fmt.wrapError | 0xc0009f2840>:
failed to start credential request: POST https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-tags-k2qhj7/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-tags-cluster/requestAdminCredential
--------------------------------------------------------------------------------
RESPONSE 503: 503 Service Unavailable
ERROR CODE: LockContention
--------------------------------------------------------------------------------
{
"error": {
"code": "LockContention",
"message": "Failed to acquire lock: timed out",
"target": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
}
}
--------------------------------------------------------------------------------
{
msg: "failed to start credential request: POST https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-tags-k2qhj7/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-tags-cluster/requestAdminCredential\n--------------------------------------------------------------------------------\nRESPONSE 503: 503 Service Unavailable\nERROR CODE: LockContention\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"LockContention\",\n \"message\": \"Failed to acquire lock: timed out\",\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX\"\n }\n}\n--------------------------------------------------------------------------------\n",
err: <*exported.ResponseError | 0xc00159e990>{
ErrorCode: "LockContention",
StatusCode: 503,
RawResponse: {
Status: "503 Service Unavailable",
StatusCode: 503,
Proto: "HTTP/2.0",
ProtoMajor: 2,
ProtoMinor: 0,
Header: {
"X-Ms-Operation-Identifier": [
"appId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,tenantId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,objectId=0e96dbb7-17a2-4393-b2d3-d84524e39c32/uksouth/1c5d69eb-b218-43c3-89e1-797546221c53",
],
"X-Ms-Ratelimit-Remaining-Subscription-Global-Writes": ["2999"],
"X-Cache": ["CONFIG_NOCACHE"],
"Date": [
"Thu, 02 Apr 2026 02:10:08 GMT",
],
"Content-Type": ["application/json"],
"X-Ms-Error-Code": ["LockContention"],
"X-Ms-Ratelimit-Remaining-Subscription-Writes": ["199"],
"X-Ms-Routing-Request-Id": [
"UKSOUTH:20260402T021008Z:afdf4447-88c0-4010-85eb-2250eadd30fe",
],
"X-Msedge-Ref": [
"Ref A: 1EC287F8926644E099B85CCB3B5E9590 Ref B: CHI261140504042 Ref C: 2026-04-02T02:09:58Z",
],
"Content-Length": ["186"],
"Expires": ["-1"],
"X-Ms-Request-Id": [
"2b953333-0bff-468f-ad05-d66baee5ebf5",
],
"X-Envoy-Upstream-Service-Time": ["10006"],
"X-Ms-Correlation-Request-Id": [
"afdf4447-88c0-4010-85eb-2250eadd30fe",
],
"Strict-Transport-Security": [
"max-age=31536000; includeSubDomains",
],
"X-Content-Type-Options": ["nosniff"],
"Cache-Control": ["no-cache"],
"Pragma": ["no-cache"],
"Retry-After": ["10"],
"X-Ms-Failure-Cause": ["service"],
},
Body: <*exported.nopClosingBytesReader | 0xc0009f2700>{
s: [123, 10, 32, 32, 32, 32, 34, 101, 114, 114, 111, 114, 34, 58, 32, 123, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 99, 111, 100, 101, 34, 58, 32, 34, 76, 111, 99, 107, 67, 111, 110, 116, 101, 110, 116, 105, 111, 110, 34, 44, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 109, 101, 115, 115, 97, 103, 101, 34, 58, 32, 34, 70, 97, 105, 108, 101, 100, 32, 116, 111, 32, 97, 99, 113, 117, 105, 114, 101, 32, 108, 111, 99, 107, 58, 32, 116, 105, 109, 101, 100, 32, 111, 117, 116, 34, 44, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 116, 97, 114, 103, 101, 116, 34, 58, 32, 34, 47, 115, 117, 98, 115, 99, 114, 105, 112, 116, 105, 111, 110, 115, 47, 57, 57, 51, 57, 57, 50, 56, 49, 45, 48, 48, 97, 50, 45, 52, 98, 51, 57, 45, 98, 98, 51, 100, 45, 98, 50, 54, 52, 53, 98, 98, 98, 100, 98, 57, 51, 34, 10, 32, 32, 32, 32, 125, 10, 125],
i: 0,
},
Conte...
Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.
Learn more here: https://onsi.github.io/gomega/#adjusting-output
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:170]: Unexpected error:
<*fmt.wrapError | 0xc0009f2840>:
failed to start credential request: POST https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-tags-k2qhj7/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-tags-cluster/requestAdminCredential
--------------------------------------------------------------------------------
RESPONSE 503: 503 Service Unavailable
ERROR CODE: LockContention
--------------------------------------------------------------------------------
{
"error": {
"code": "LockContention",
"message": "Failed to acquire lock: timed out",
"target": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
}
}
--------------------------------------------------------------------------------
{
msg: "failed to start credential request: POST https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-tags-k2qhj7/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-tags-cluster/requestAdminCredential\n--------------------------------------------------------------------------------\nRESPONSE 503: 503 Service Unavailable\nERROR CODE: LockContention\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"LockContention\",\n \"message\": \"Failed to acquire lock: timed out\",\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX\"\n }\n}\n--------------------------------------------------------------------------------\n",
err: <*exported.ResponseError | 0xc00159e990>{
ErrorCode: "LockContention",
StatusCode: 503,
RawResponse: {
Status: "503 Service Unavailable",
StatusCode: 503,
Proto: "HTTP/2.0",
ProtoMajor: 2,
ProtoMinor: 0,
Header: {
"X-Ms-Operation-Identifier": [
"appId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,tenantId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,objectId=0e96dbb7-17a2-4393-b2d3-d84524e39c32/uksouth/1c5d69eb-b218-43c3-89e1-797546221c53",
],
"X-Ms-Ratelimit-Remaining-Subscription-Global-Writes": ["2999"],
"X-Cache": ["CONFIG_NOCACHE"],
"Date": [
"Thu, 02 Apr 2026 02:10:08 GMT",
],
"Content-Type": ["application/json"],
"X-Ms-Error-Code": ["LockContention"],
"X-Ms-Ratelimit-Remaining-Subscription-Writes": ["199"],
"X-Ms-Routing-Request-Id": [
"UKSOUTH:20260402T021008Z:afdf4447-88c0-4010-85eb-2250eadd30fe",
],
"X-Msedge-Ref": [
"Ref A: 1EC287F8926644E099B85CCB3B5E9590 Ref B: CHI261140504042 Ref C: 2026-04-02T02:09:58Z",
],
"Content-Length": ["186"],
"Expires": ["-1"],
"X-Ms-Request-Id": [
"2b953333-0bff-468f-ad05-d66baee5ebf5",
],
"X-Envoy-Upstream-Service-Time": ["10006"],
"X-Ms-Correlation-Request-Id": [
"afdf4447-88c0-4010-85eb-2250eadd30fe",
],
"Strict-Transport-Security": [
"max-age=31536000; includeSubDomains",
],
"X-Content-Type-Options": ["nosniff"],
"Cache-Control": ["no-cache"],
"Pragma": ["no-cache"],
"Retry-After": ["10"],
"X-Ms-Failure-Cause": ["service"],
},
Body: <*exported.nopClosingBytesReader | 0xc0009f2700>{
s: [123, 10, 32, 32, 32, 32, 34, 101, 114, 114, 111, 114, 34, 58, 32, 123, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 99, 111, 100, 101, 34, 58, 32, 34, 76, 111, 99, 107, 67, 111, 110, 116, 101, 110, 116, 105, 111, 110, 34, 44, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 109, 101, 115, 115, 97, 103, 101, 34, 58, 32, 34, 70, 97, 105, 108, 101, 100, 32, 116, 111, 32, 97, 99, 113, 117, 105, 114, 101, 32, 108, 111, 99, 107, 58, 32, 116, 105, 109, 101, 100, 32, 111, 117, 116, 34, 44, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 116, 97, 114, 103, 101, 116, 34, 58, 32, 34, 47, 115, 117, 98, 115, 99, 114, 105, 112, 116, 105, 111, 110, 115, 47, 57, 57, 51, 57, 57, 50, 56, 49, 45, 48, 48, 97, 50, 45, 52, 98, 51, 57, 45, 98, 98, 51, 100, 45, 98, 50, 54, 52, 53, 98, 98, 98, 100, 98, 57, 51, 34, 10, 32, 32, 32, 32, 125, 10, 125],
i: 0,
},
Conte...
Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.
Learn more here: https://onsi.github.io/gomega/#adjusting-output
occurredfail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:103]: Unexpected error:
<*fmt.wrapError | 0xc00142ee20>:
failed to start credential request: POST https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/external-auth-cluster-4qqv6m/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/ea-cluster/requestAdminCredential
--------------------------------------------------------------------------------
RESPONSE 503: 503 Service Unavailable
ERROR CODE: LockContention
--------------------------------------------------------------------------------
{
"error": {
"code": "LockContention",
"message": "Failed to acquire lock: timed out",
"target": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
}
}
--------------------------------------------------------------------------------
{
msg: "failed to start credential request: POST https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/external-auth-cluster-4qqv6m/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/ea-cluster/requestAdminCredential\n--------------------------------------------------------------------------------\nRESPONSE 503: 503 Service Unavailable\nERROR CODE: LockContention\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"LockContention\",\n \"message\": \"Failed to acquire lock: timed out\",\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX\"\n }\n}\n--------------------------------------------------------------------------------\n",
err: <*exported.ResponseError | 0xc00104d530>{
ErrorCode: "LockContention",
StatusCode: 503,
RawResponse: {
Status: "503 Service Unavailable",
StatusCode: 503,
Proto: "HTTP/2.0",
ProtoMajor: 2,
ProtoMinor: 0,
Header: {
"X-Ms-Ratelimit-Remaining-Subscription-Writes": ["199"],
"X-Content-Type-Options": ["nosniff"],
"Content-Type": ["application/json"],
"X-Ms-Failure-Cause": ["service"],
"X-Ms-Operation-Identifier": [
"appId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,tenantId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,objectId=0e96dbb7-17a2-4393-b2d3-d84524e39c32/uksouth/0949565f-2116-4c2d-ac42-f874e16826f1",
],
"X-Ms-Ratelimit-Remaining-Subscription-Global-Writes": ["2999"],
"X-Ms-Routing-Request-Id": [
"UKSOUTH:20260402T021009Z:c60ff00c-0498-471e-83f9-643c3970b3df",
],
"Strict-Transport-Security": [
"max-age=31536000; includeSubDomains",
],
"X-Cache": ["CONFIG_NOCACHE"],
"X-Msedge-Ref": [
"Ref A: 84A2B7061A1D427CAC14B9844929E76E Ref B: CHI261140502054 Ref C: 2026-04-02T02:09:58Z",
],
"Pragma": ["no-cache"],
"Content-Length": ["186"],
"Retry-After": ["10"],
"X-Ms-Error-Code": ["LockContention"],
"X-Ms-Request-Id": [
"342b364f-60a1-493c-a980-5f75e6f1c77f",
],
"X-Envoy-Upstream-Service-Time": ["10010"],
"Date": [
"Thu, 02 Apr 2026 02:10:08 GMT",
],
"Cache-Control": ["no-cache"],
"Expires": ["-1"],
"X-Ms-Correlation-Request-Id": [
"c60ff00c-0498-471e-83f9-643c3970b3df",
],
},
Body: <*exported.nopClosingBytesReader | 0xc00142ed60>{
s: [123, 10, 32, 32, 32, 32, 34, 101, 114, 114, 111, 114, 34, 58, 32, 123, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 99, 111, 100, 101, 34, 58, 32, 34, 76, 111, 99, 107, 67, 111, 110, 116, 101, 110, 116, 105, 111, 110, 34, 44, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 109, 101, 115, 115, 97, 103, 101, 34, 58, 32, 34, 70, 97, 105, 108, 101, 100, 32, 116, 111, 32, 97, 99, 113, 117, 105, 114, 101, 32, 108, 111, 99, 107, 58, 32, 116, 105, 109, 101, 100, 32, 111, 117, 116, 34, 44, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 116, 97, 114, 103, 101, 116, 34, 58, 32, 34, 47, 115, 117, 98, 115, 99, 114, 105, 112, 116, 105, 111, 110, 115, 47, 57, 57, 51, 57, 57, 50, 56, 49, 45, 48, 48, 97, 50, 45, 52, 98, 51, 57, 45, 98, 98, 51, 100, 45, 98, 50, 54, 52, 53, 98, 98, 98, 100, 98, 57, 51, 34, 10, 32, 32, 32, 32, 125, 10, 125],
i: 0,
},
Co...
Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.
Learn more here: https://onsi.github.io/gomega/#adjusting-output
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:103]: Unexpected error:
<*fmt.wrapError | 0xc00142ee20>:
failed to start credential request: POST https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/external-auth-cluster-4qqv6m/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/ea-cluster/requestAdminCredential
--------------------------------------------------------------------------------
RESPONSE 503: 503 Service Unavailable
ERROR CODE: LockContention
--------------------------------------------------------------------------------
{
"error": {
"code": "LockContention",
"message": "Failed to acquire lock: timed out",
"target": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
}
}
--------------------------------------------------------------------------------
{
msg: "failed to start credential request: POST https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/external-auth-cluster-4qqv6m/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/ea-cluster/requestAdminCredential\n--------------------------------------------------------------------------------\nRESPONSE 503: 503 Service Unavailable\nERROR CODE: LockContention\n--------------------------------------------------------------------------------\n{\n \"error\": {\n \"code\": \"LockContention\",\n \"message\": \"Failed to acquire lock: timed out\",\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX\"\n }\n}\n--------------------------------------------------------------------------------\n",
err: <*exported.ResponseError | 0xc00104d530>{
ErrorCode: "LockContention",
StatusCode: 503,
RawResponse: {
Status: "503 Service Unavailable",
StatusCode: 503,
Proto: "HTTP/2.0",
ProtoMajor: 2,
ProtoMinor: 0,
Header: {
"X-Ms-Ratelimit-Remaining-Subscription-Writes": ["199"],
"X-Content-Type-Options": ["nosniff"],
"Content-Type": ["application/json"],
"X-Ms-Failure-Cause": ["service"],
"X-Ms-Operation-Identifier": [
"appId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,tenantId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,objectId=0e96dbb7-17a2-4393-b2d3-d84524e39c32/uksouth/0949565f-2116-4c2d-ac42-f874e16826f1",
],
"X-Ms-Ratelimit-Remaining-Subscription-Global-Writes": ["2999"],
"X-Ms-Routing-Request-Id": [
"UKSOUTH:20260402T021009Z:c60ff00c-0498-471e-83f9-643c3970b3df",
],
"Strict-Transport-Security": [
"max-age=31536000; includeSubDomains",
],
"X-Cache": ["CONFIG_NOCACHE"],
"X-Msedge-Ref": [
"Ref A: 84A2B7061A1D427CAC14B9844929E76E Ref B: CHI261140502054 Ref C: 2026-04-02T02:09:58Z",
],
"Pragma": ["no-cache"],
"Content-Length": ["186"],
"Retry-After": ["10"],
"X-Ms-Error-Code": ["LockContention"],
"X-Ms-Request-Id": [
"342b364f-60a1-493c-a980-5f75e6f1c77f",
],
"X-Envoy-Upstream-Service-Time": ["10010"],
"Date": [
"Thu, 02 Apr 2026 02:10:08 GMT",
],
"Cache-Control": ["no-cache"],
"Expires": ["-1"],
"X-Ms-Correlation-Request-Id": [
"c60ff00c-0498-471e-83f9-643c3970b3df",
],
},
Body: <*exported.nopClosingBytesReader | 0xc00142ed60>{
s: [123, 10, 32, 32, 32, 32, 34, 101, 114, 114, 111, 114, 34, 58, 32, 123, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 99, 111, 100, 101, 34, 58, 32, 34, 76, 111, 99, 107, 67, 111, 110, 116, 101, 110, 116, 105, 111, 110, 34, 44, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 109, 101, 115, 115, 97, 103, 101, 34, 58, 32, 34, 70, 97, 105, 108, 101, 100, 32, 116, 111, 32, 97, 99, 113, 117, 105, 114, 101, 32, 108, 111, 99, 107, 58, 32, 116, 105, 109, 101, 100, 32, 111, 117, 116, 34, 44, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 116, 97, 114, 103, 101, 116, 34, 58, 32, 34, 47, 115, 117, 98, 115, 99, 114, 105, 112, 116, 105, 111, 110, 115, 47, 57, 57, 51, 57, 57, 50, 56, 49, 45, 48, 48, 97, 50, 45, 52, 98, 51, 57, 45, 98, 98, 51, 100, 45, 98, 50, 54, 52, 53, 98, 98, 98, 100, 98, 57, 51, 34, 10, 32, 32, 32, 32, 125, 10, 125],
i: 0,
},
Co...
Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.
Learn more here: https://onsi.github.io/gomega/#adjusting-output
occurredContributing tests (2)
Affected runs (1)
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||