No failure patterns match your search.
Environment: PROD
Failure patterns: 18 · Failures matched: 163 · Total runs: 17 · Runs affected: 16
| Failure patternThe canonical description of a recurring CI failure, extracted and normalized from raw logs. | Failed atThe stage of the job run where this failure occurred: 'provision' (environment setup, DEV only), 'e2e' (test suite execution), or 'other' (CI infrastructure issues that did not produce a failure pattern). | Number of distinct job runs where this failure pattern was detected. | Percentage of all job runs in this environment affected by this failure pattern during the selected window. | Classification of this failure pattern: Regression (likely caused by a specific PR), Flake (intermittent failure spread across days), Noise (low-quality or generic pattern), or Indeterminate. | TrendShows daily activity for this failure pattern in a trailing window anchored to the selected end date. The sparkline covers at least 7 days and at most 14 days, depending on the current window size. | Also inOther environments where the same failure pattern was also detected during the selected window. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
ERROR CODE: InternalServerError; detail message node pool creation did not complete before the deadline; provider Microsoft.RedHatOpenShiftfull failure pattern: ERROR CODE: InternalServerError; detail message node pool creation did not complete before the deadline; provider Microsoft.RedHatOpenShift Signal: IndeterminateSignal: Indeterminate — seen in 1 prior week(s); active 8 day(s) | e2e | 11 | 64.71%11 of 17 job runs affected | IndeterminateSignal: Indeterminate — seen in 1 prior week(s); active 8 day(s) | Show trend detailsSep 20: 3 · Sep 21: 1 · Sep 22: 1 · Sep 23: 2 · Sep 24: 3 · Sep 25: 2 · Sep 26: 3 · Sep 27: 1 | none | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (3)fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_ingress.go:123]: failed to create node pool "np-1" for private ingress cluster "private-ingress"
Unexpected error:
<*fmt.wrapError | 0xc000b13b60>:
failed to create NodePool np-1: failed waiting for nodepool="np-1" for cluster "private-ingress" in resourcegroup="private-ingress-6k9kth8pvmf8" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/eb2dcee5-7fa0-489a-bdf1-3568cea3218a
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/eb2dcee5-7fa0-489a-bdf1-3568cea3218a",
"name": "eb2dcee5-7fa0-489a-bdf1-3568cea3218a",
"status": "Failed",
"startTime": "2026-09-27T02:13:34.441508798Z",
"endTime": "2026-09-27T02:32:39.228262555Z",
"error": {
"code": "InternalServerError",
"message": "node pool creation did not complete before the deadline"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_ingress.go:123]: failed to create node pool "np-1" for private ingress cluster "private-ingress"
Unexpected error:
<*fmt.wrapError | 0xc000b13b60>:
failed to create NodePool np-1: failed waiting for nodepool="np-1" for cluster "private-ingress" in resourcegroup="private-ingress-6k9kth8pvmf8" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/eb2dcee5-7fa0-489a-bdf1-3568cea3218a
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/eb2dcee5-7fa0-489a-bdf1-3568cea3218a",
"name": "eb2dcee5-7fa0-489a-bdf1-3568cea3218a",
"status": "Failed",
"startTime": "2026-09-27T02:13:34.441508798Z",
"endTime": "2026-09-27T02:32:39.228262555Z",
"error": {
"code": "InternalServerError",
"message": "node pool creation did not complete before the deadline"
}
}
--------------------------------------------------------------------------------
...
occurredfail [github.com/Azure/ARO-HCP/test/e2e/cluster_delete_cx_rg.go:139]: failed to create node pools in parallel for cluster "cx-rg-hcp-cluster"
Unexpected error:
<*fmt.wrapError | 0xc00138eac0>:
failed to create NodePool np-1: failed waiting for nodepool="np-1" for cluster "cx-rg-hcp-cluster" in resourcegroup="cx-rg-cluster-vn4grw565jrj" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/7f58a11a-50d1-416b-9f82-356e8273b038
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/7f58a11a-50d1-416b-9f82-356e8273b038",
"name": "7f58a11a-50d1-416b-9f82-356e8273b038",
"status": "Failed",
"startTime": "2026-09-26T23:14:10.630574624Z",
"endTime": "2026-09-26T23:33:16.736724619Z",
"error": {
"code": "InternalServerError",
"message": "node pool creation did not complete before the deadline"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delete_cx_rg.go:139]: failed to create node pools in parallel for cluster "cx-rg-hcp-cluster"
Unexpected error:
<*fmt.wrapError | 0xc00138eac0>:
failed to create NodePool np-1: failed waiting for nodepool="np-1" for cluster "cx-rg-hcp-cluster" in resourcegroup="cx-rg-cluster-vn4grw565jrj" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/7f58a11a-50d1-416b-9f82-356e8273b038
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/7f58a11a-50d1-416b-9f82-356e8273b038",
"name": "7f58a11a-50d1-416b-9f82-356e8273b038",
"status": "Failed",
"startTime": "2026-09-26T23:14:10.630574624Z",
"endTime": "2026-09-26T23:33:16.736724619Z",
"error": {
"code": "InternalServerError",
"message": "node pool creation did not complete before the deadline"
}
}
--------------------------------------------------------------------------------
...
occurredfail [github.com/Azure/ARO-HCP/test/e2e/arm64_nodepool.go:109]: failed to create ARM64 nodepool "arm64-vm-np-1" with VM size "Standard_D2plds_v6"
Unexpected error:
<*fmt.wrapError | 0xc0010653a0>:
failed to create NodePool arm64-vm-np-1: failed waiting for nodepool="arm64-vm-np-1" for cluster "arm64-vm-hcp-cluster" in resourcegroup="arm64-vm-cluster-5jg8xwpwj9d7" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/a82865e3-4228-4f7d-bc65-f5af89262225
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/a82865e3-4228-4f7d-bc65-f5af89262225",
"name": "a82865e3-4228-4f7d-bc65-f5af89262225",
"status": "Failed",
"startTime": "2026-09-26T23:13:30.607288907Z",
"endTime": "2026-09-26T23:32:36.744317191Z",
"error": {
"code": "InternalServerError",
"message": "node pool creation did not complete before the deadline"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/arm64_nodepool.go:109]: failed to create ARM64 nodepool "arm64-vm-np-1" with VM size "Standard_D2plds_v6"
Unexpected error:
<*fmt.wrapError | 0xc0010653a0>:
failed to create NodePool arm64-vm-np-1: failed waiting for nodepool="arm64-vm-np-1" for cluster "arm64-vm-hcp-cluster" in resourcegroup="arm64-vm-cluster-5jg8xwpwj9d7" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/a82865e3-4228-4f7d-bc65-f5af89262225
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/a82865e3-4228-4f7d-bc65-f5af89262225",
"name": "a82865e3-4228-4f7d-bc65-f5af89262225",
"status": "Failed",
"startTime": "2026-09-26T23:13:30.607288907Z",
"endTime": "2026-09-26T23:32:36.744317191Z",
"error": {
"code": "InternalServerError",
"message": "node pool creation did not complete before the deadline"
}
}
--------------------------------------------------------------------------------
...
occurredContributing tests (12)
Affected runs (11)
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
storage account log verification failedfull failure pattern: storage account log verification failed Signal: IndeterminateSignal: Indeterminate — no prior history; active 4 day(s) — New failure pattern — no prior history | e2e | 10 | 58.82%10 of 17 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 4 day(s) — New failure pattern — no prior history | Show trend detailsSep 20: 0 · Sep 21: 0 · Sep 22: 0 · Sep 23: 0 · Sep 24: 4 · Sep 25: 2 · Sep 26: 2 · Sep 27: 2 | none | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (3)fail [github.com/Azure/ARO-HCP/test/e2e/shoebox_logs.go:353]: failed to verify shoebox logs reached storage account shoeboxwdj8sw and Event Hub shoebox-eh for cluster /subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/shoebox-cluster-fct27gcqstbt/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/shoebox-hcp-cluster
Unexpected error:
<*errors.errorString | 0xc0007a08f0>:
storage account log verification failed
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/shoebox_logs.go:353]: failed to verify shoebox logs reached storage account shoeboxwdj8sw and Event Hub shoebox-eh for cluster /subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/shoebox-cluster-fct27gcqstbt/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/shoebox-hcp-cluster
Unexpected error:
<*errors.errorString | 0xc0007a08f0>:
storage account log verification failed
...
occurredfail [github.com/Azure/ARO-HCP/test/e2e/shoebox_logs.go:353]: failed to verify shoebox logs reached storage account shoeboxt4mfpn and Event Hub shoebox-eh for cluster /subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/shoebox-cluster-vg2849dzv4lc/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/shoebox-hcp-cluster
Unexpected error:
<*errors.errorString | 0xc0004c67b0>:
storage account log verification failed
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/shoebox_logs.go:353]: failed to verify shoebox logs reached storage account shoeboxt4mfpn and Event Hub shoebox-eh for cluster /subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/shoebox-cluster-vg2849dzv4lc/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/shoebox-hcp-cluster
Unexpected error:
<*errors.errorString | 0xc0004c67b0>:
storage account log verification failed
...
occurredfail [github.com/Azure/ARO-HCP/test/e2e/shoebox_logs.go:353]: failed to verify shoebox logs reached storage account shoeboxchw7s6 and Event Hub shoebox-eh for cluster /subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/shoebox-cluster-f6qdhr6p7x7g/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/shoebox-hcp-cluster
Unexpected error:
<*errors.errorString | 0xc0008da0a0>:
storage account log verification failed
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/shoebox_logs.go:353]: failed to verify shoebox logs reached storage account shoeboxchw7s6 and Event Hub shoebox-eh for cluster /subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/shoebox-cluster-f6qdhr6p7x7g/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/shoebox-hcp-cluster
Unexpected error:
<*errors.errorString | 0xc0008da0a0>:
storage account log verification failed
...
occurredContributing tests (3)
Affected runs (10)
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
ERROR CODE: InvalidRequestContent; detail message Invalid value: "<ocp-nightly>": must be at least 4.20.8; provider Microsoft.RedHatOpenShiftfull failure pattern: ERROR CODE: InvalidRequestContent; detail message Invalid value: "<ocp-nightly>": must be at least 4.20.8; provider Microsoft.RedHatOpenShift Signal: IndeterminateSignal: Indeterminate — seen in 1 prior week(s); active 8 day(s) | e2e | 7 | 41.18%7 of 17 job runs affected | IndeterminateSignal: Indeterminate — seen in 1 prior week(s); active 8 day(s) | Show trend detailsSep 20: 4 · Sep 21: 4 · Sep 22: 4 · Sep 23: 4 · Sep 24: 4 · Sep 25: 4 · Sep 26: 2 · Sep 27: 4 | none | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (3)fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:768]: failed to update nodepool npdg-422-420 to downgrade target 4.20.0-0.nightly-multi-2026-09-13-024541
Unexpected error:
<*fmt.wrapError | 0xc002ba0720>:
failed to start nodepool "npdg-422-420" update in cluster "np-dg-minor-4pp72c" resourcegroup="rg-np-dg-minor-4pp72c-t57mnh5thxv5": PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-dg-minor-4pp72c-t57mnh5thxv5/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-dg-minor-4pp72c/nodePools/npdg-422-420
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"4.20.0-0.nightly-multi-2026-09-13-024541\": must be at least 4.20.8",
"target": "properties.version.id"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:768]: failed to update nodepool npdg-422-420 to downgrade target 4.20.0-0.nightly-multi-2026-09-13-024541
Unexpected error:
<*fmt.wrapError | 0xc002ba0720>:
failed to start nodepool "npdg-422-420" update in cluster "np-dg-minor-4pp72c" resourcegroup="rg-np-dg-minor-4pp72c-t57mnh5thxv5": PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-dg-minor-4pp72c-t57mnh5thxv5/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-dg-minor-4pp72c/nodePools/npdg-422-420
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"4.20.0-0.nightly-multi-2026-09-13-024541\": must be at least 4.20.8",
"target": "properties.version.id"
}
}
--------------------------------------------------------------------------------
...
occurredfail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-btngfc" for cluster "cluster-nightly-4-20-btngfc"
Unexpected error:
<*fmt.wrapError | 0xc000b4c1e0>:
failed to create NodePool np-btngfc: failed starting nodepool creation "np-btngfc" for cluster "cluster-nightly-4-20-btngfc" in resourcegroup="rg-nightly-4-20-dlkdt65gwjpj": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-nightly-4-20-dlkdt65gwjpj/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-nightly-4-20-btngfc/nodePools/np-btngfc
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"4.20.0-0.nightly-multi-2026-09-13-024541\": must be at least 4.20.8",
"target": "properties.version.id"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-btngfc" for cluster "cluster-nightly-4-20-btngfc"
Unexpected error:
<*fmt.wrapError | 0xc000b4c1e0>:
failed to create NodePool np-btngfc: failed starting nodepool creation "np-btngfc" for cluster "cluster-nightly-4-20-btngfc" in resourcegroup="rg-nightly-4-20-dlkdt65gwjpj": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-nightly-4-20-dlkdt65gwjpj/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-nightly-4-20-btngfc/nodePools/np-btngfc
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"4.20.0-0.nightly-multi-2026-09-13-024541\": must be at least 4.20.8",
"target": "properties.version.id"
}
}
--------------------------------------------------------------------------------
...
occurredfail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:165]: failed to create node pool npupgrade-4-20 with version 4.20.0-0.nightly-multi-2026-09-13-024541
Unexpected error:
<*fmt.wrapError | 0xc000a81100>:
failed to create NodePool npupgrade-4-20: failed starting nodepool creation "npupgrade-4-20" for cluster "np-version-upgrade-cluster-9zjchk" in resourcegroup="rg-np-version-upgrade-9zjchk-r4lv9m9d98td": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-version-upgrade-9zjchk-r4lv9m9d98td/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-version-upgrade-cluster-9zjchk/nodePools/npupgrade-4-20
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"4.20.0-0.nightly-multi-2026-09-13-024541\": must be at least 4.20.8",
"target": "properties.version.id"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:165]: failed to create node pool npupgrade-4-20 with version 4.20.0-0.nightly-multi-2026-09-13-024541
Unexpected error:
<*fmt.wrapError | 0xc000a81100>:
failed to create NodePool npupgrade-4-20: failed starting nodepool creation "npupgrade-4-20" for cluster "np-version-upgrade-cluster-9zjchk" in resourcegroup="rg-np-version-upgrade-9zjchk-r4lv9m9d98td": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-version-upgrade-9zjchk-r4lv9m9d98td/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-version-upgrade-cluster-9zjchk/nodePools/npupgrade-4-20
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"4.20.0-0.nightly-multi-2026-09-13-024541\": must be at least 4.20.8",
"target": "properties.version.id"
}
}
--------------------------------------------------------------------------------
...
occurredContributing tests (4)
Affected runs (7)
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
VerifyKubeAPIServerServerVersionUpgraded failed: kube-apiserver ServerVersion not updated (unchanged from pre-upgrade)full failure pattern: VerifyKubeAPIServerServerVersionUpgraded failed: kube-apiserver ServerVersion not updated (unchanged from pre-upgrade) Signal: IndeterminateSignal: Indeterminate — seen in 1 prior week(s); active 8 day(s) | e2e | 7 | 41.18%7 of 17 job runs affected | IndeterminateSignal: Indeterminate — seen in 1 prior week(s); active 8 day(s) | Show trend detailsSep 20: 1 · Sep 21: 1 · Sep 22: 1 · Sep 23: 1 · Sep 24: 1 · Sep 25: 1 · Sep 26: 2 · Sep 27: 2 | none | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (3)fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:176]: Timed out after 2700.003s.
Expected success, but got an error:
<*errors.joinError | 0xc0014b70f8>:
VerifyKubeAPIServerServerVersionUpgraded failed: kube-apiserver ServerVersion not updated (unchanged from pre-upgrade)
VerifyHostedControlPlaneYStreamUpgrade(previousMinor=4.20.0-0.nightly-multi-2026-09-13-024541, targetMinor=4.21.0-0.nightly-multi-2026-08-16-064216) failed: clusterversion status.history has no version in target minor "4.21.0-0.nightly-multi-2026-08-16-064216"
...
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:176]: Timed out after 2700.003s.
Expected success, but got an error:
<*errors.joinError | 0xc0014b70f8>:
VerifyKubeAPIServerServerVersionUpgraded failed: kube-apiserver ServerVersion not updated (unchanged from pre-upgrade)
VerifyHostedControlPlaneYStreamUpgrade(previousMinor=4.20.0-0.nightly-multi-2026-09-13-024541, targetMinor=4.21.0-0.nightly-multi-2026-08-16-064216) failed: clusterversion status.history has no version in target minor "4.21.0-0.nightly-multi-2026-08-16-064216"
...fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:176]: Timed out after 2700.000s.
Expected success, but got an error:
<*errors.joinError | 0xc0015174b8>:
VerifyKubeAPIServerServerVersionUpgraded failed: kube-apiserver ServerVersion not updated (unchanged from pre-upgrade)
VerifyHostedControlPlaneYStreamUpgrade(previousMinor=5.0.0-0.nightly-multi-2026-09-27-005925, targetMinor=5.1.0-0.nightly-multi-2026-08-10-113053) failed: clusterversion status.history has no version in target minor "5.1.0-0.nightly-multi-2026-08-10-113053"
...
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:176]: Timed out after 2700.000s.
Expected success, but got an error:
<*errors.joinError | 0xc0015174b8>:
VerifyKubeAPIServerServerVersionUpgraded failed: kube-apiserver ServerVersion not updated (unchanged from pre-upgrade)
VerifyHostedControlPlaneYStreamUpgrade(previousMinor=5.0.0-0.nightly-multi-2026-09-27-005925, targetMinor=5.1.0-0.nightly-multi-2026-08-10-113053) failed: clusterversion status.history has no version in target minor "5.1.0-0.nightly-multi-2026-08-10-113053"
...fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:176]: Timed out after 2700.006s.
Expected success, but got an error:
<*errors.joinError | 0xc00158d338>:
VerifyKubeAPIServerServerVersionUpgraded failed: kube-apiserver ServerVersion not updated (unchanged from pre-upgrade)
VerifyHostedControlPlaneYStreamUpgrade(previousMinor=4.21.0-0.nightly-multi-2026-09-24-033657, targetMinor=4.22.0-0.nightly-multi-2026-07-08-192436) failed: clusterversion status.history has no version in target minor "4.22.0-0.nightly-multi-2026-07-08-192436"
...
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:176]: Timed out after 2700.006s.
Expected success, but got an error:
<*errors.joinError | 0xc00158d338>:
VerifyKubeAPIServerServerVersionUpgraded failed: kube-apiserver ServerVersion not updated (unchanged from pre-upgrade)
VerifyHostedControlPlaneYStreamUpgrade(previousMinor=4.21.0-0.nightly-multi-2026-09-24-033657, targetMinor=4.22.0-0.nightly-multi-2026-07-08-192436) failed: clusterversion status.history has no version in target minor "4.22.0-0.nightly-multi-2026-07-08-192436"
...Contributing tests (3)
Affected runs (7)
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
ERROR CODE: InvalidRequestContent; detail message Invalid value: "4.21": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when c...full failure pattern: ERROR CODE: InvalidRequestContent; detail message Invalid value: "4.21": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is "nightly"; provider Microsoft.RedHatOpenShift Signal: IndeterminateSignal: Indeterminate — seen in 1 prior week(s); active 7 day(s) | e2e | 6 | 35.29%6 of 17 job runs affected | IndeterminateSignal: Indeterminate — seen in 1 prior week(s); active 7 day(s) | Show trend detailsSep 20: 5 · Sep 21: 5 · Sep 22: 5 · Sep 23: 4 · Sep 24: 5 · Sep 25: 5 · Sep 26: 5 · Sep 27: 0 | none | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (3)fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:173]: failed to update cluster "autoscaling-hcp-cluster" autoscaling configuration
Unexpected error:
<*exported.ResponseError | 0xc0014a40c0>:
PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/autoscaling-cluster-gx85bj8dbjwr/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/autoscaling-hcp-cluster
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"4.21\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"",
"target": "properties.version.id"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:173]: failed to update cluster "autoscaling-hcp-cluster" autoscaling configuration
Unexpected error:
<*exported.ResponseError | 0xc0014a40c0>:
PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/autoscaling-cluster-gx85bj8dbjwr/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/autoscaling-hcp-cluster
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"4.21\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"",
"target": "properties.version.id"
}
}
--------------------------------------------------------------------------------
...
occurredfail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:408]: failed to begin updating cluster "cidr-connectivity-test" authorized CIDRs
Unexpected error:
<*exported.ResponseError | 0xc0029946c0>:
PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-hxs9z5vrpbbz/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cidr-connectivity-test
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"4.21\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"",
"target": "properties.version.id"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:408]: failed to begin updating cluster "cidr-connectivity-test" authorized CIDRs
Unexpected error:
<*exported.ResponseError | 0xc0029946c0>:
PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-hxs9z5vrpbbz/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cidr-connectivity-test
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"4.21\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"",
"target": "properties.version.id"
}
}
--------------------------------------------------------------------------------
...
occurredfail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:193]: failed to update HCP cluster tags via PATCH
Unexpected error:
<*exported.ResponseError | 0xc000f0d9b0>:
PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-tags-cl4zndzgpjp5/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-tags-cluster
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"4.21\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"",
"target": "properties.version.id"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:193]: failed to update HCP cluster tags via PATCH
Unexpected error:
<*exported.ResponseError | 0xc000f0d9b0>:
PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-tags-cl4zndzgpjp5/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-tags-cluster
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"4.21\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"",
"target": "properties.version.id"
}
}
--------------------------------------------------------------------------------
...
occurredContributing tests (5)
Affected runs (6)
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
ERROR CODE: InternalServerError; detail message [cosmosCluster] .api.url is empty; [servingCABundle] ServingCABundle not yet populated; [clusterServiceClusterStatus] cluster servi...full failure pattern: ERROR CODE: InternalServerError; detail message [cosmosCluster] .api.url is empty; [servingCABundle] ServingCABundle not yet populated; [clusterServiceClusterStatus] cluster service has not been successfully created; [hypershiftHostedCluster] hosted cluster state not cached yet; provider Microsoft.RedHatOpenShift Signal: IndeterminateSignal: Indeterminate — no prior history; active 5 day(s) — New failure pattern — no prior history | e2e | 5 | 29.41%5 of 17 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 5 day(s) — New failure pattern — no prior history | Show trend detailsSep 20: 0 · Sep 21: 0 · Sep 22: 1 · Sep 23: 1 · Sep 24: 1 · Sep 25: 0 · Sep 26: 6 · Sep 27: 28 | none | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (3)fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:102]: failed to create HCP cluster np-autoscale-cluster
Unexpected error:
<*fmt.wrapError | 0xc0026392a0>:
failed to create HCP cluster np-autoscale-cluster: failed waiting for cluster="np-autoscale-cluster" in resourcegroup="np-autoscaling-pndxf7r8t9rr" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/a91e490f-050a-4c07-a58f-af347cb9f143
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/a91e490f-050a-4c07-a58f-af347cb9f143",
"name": "a91e490f-050a-4c07-a58f-af347cb9f143",
"status": "Failed",
"startTime": "2026-09-28T00:13:40.174684426Z",
"endTime": "2026-09-28T00:42:43.628005146Z",
"error": {
"code": "InternalServerError",
"message": "[cosmosCluster] .api.url is empty; [servingCABundle] ServingCABundle not yet populated; [clusterServiceClusterStatus] cluster service has not been successfully created; [hypershiftHostedCluster] hosted cluster state not cached yet"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:102]: failed to create HCP cluster np-autoscale-cluster
Unexpected error:
<*fmt.wrapError | 0xc0026392a0>:
failed to create HCP cluster np-autoscale-cluster: failed waiting for cluster="np-autoscale-cluster" in resourcegroup="np-autoscaling-pndxf7r8t9rr" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/a91e490f-050a-4c07-a58f-af347cb9f143
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/a91e490f-050a-4c07-a58f-af347cb9f143",
"name": "a91e490f-050a-4c07-a58f-af347cb9f143",
"status": "Failed",
"startTime": "2026-09-28T00:13:40.174684426Z",
"endTime": "2026-09-28T00:42:43.628005146Z",
"error": {
"code": "InternalServerError",
"message": "[cosmosCluster] .api.url is empty; [servingCABundle] ServingCABundle not yet populated; [clusterServiceClusterStatus] cluster service has not been successfully created; [hypershiftHostedCluster] hosted cluster state not cached yet"
}
}
--------------------------------------------------------------------------------
...
occurredfail [github.com/Azure/ARO-HCP/test/e2e/cluster_nsg_subnet_reuse.go:107]: failed to create seed HCP cluster basic-cluster
Unexpected error:
<*fmt.wrapError | 0xc000866340>:
failed to create HCP cluster basic-cluster: failed waiting for cluster="basic-cluster" in resourcegroup="rg-cluster-nsg-subnet-reuse-wg62gnrkcq82" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/38c68d19-2950-482f-9bf9-f70f6c7063a4
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/38c68d19-2950-482f-9bf9-f70f6c7063a4",
"name": "38c68d19-2950-482f-9bf9-f70f6c7063a4",
"status": "Failed",
"startTime": "2026-09-27T23:02:28.308542215Z",
"endTime": "2026-09-27T23:31:35.969190413Z",
"error": {
"code": "InternalServerError",
"message": "[cosmosCluster] .api.url is empty; [servingCABundle] ServingCABundle not yet populated; [clusterServiceClusterStatus] cluster service has not been successfully created; [hypershiftHostedCluster] hosted cluster state not cached yet"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_nsg_subnet_reuse.go:107]: failed to create seed HCP cluster basic-cluster
Unexpected error:
<*fmt.wrapError | 0xc000866340>:
failed to create HCP cluster basic-cluster: failed waiting for cluster="basic-cluster" in resourcegroup="rg-cluster-nsg-subnet-reuse-wg62gnrkcq82" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/38c68d19-2950-482f-9bf9-f70f6c7063a4
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/38c68d19-2950-482f-9bf9-f70f6c7063a4",
"name": "38c68d19-2950-482f-9bf9-f70f6c7063a4",
"status": "Failed",
"startTime": "2026-09-27T23:02:28.308542215Z",
"endTime": "2026-09-27T23:31:35.969190413Z",
"error": {
"code": "InternalServerError",
"message": "[cosmosCluster] .api.url is empty; [servingCABundle] ServingCABundle not yet populated; [clusterServiceClusterStatus] cluster service has not been successfully created; [hypershiftHostedCluster] hosted cluster state not cached yet"
}
}
--------------------------------------------------------------------------------
...
occurredfail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_v20260901preview.go:86]: failed to create HCP cluster "v20260901" via v20260901preview
Unexpected error:
<*fmt.wrapError | 0xc000992660>:
failed to create HCP cluster v20260901: failed waiting for cluster="v20260901" in resourcegroup="v20260901preview-nwqdmg89c6p9" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/7c1f6eed-e1f4-4411-9a1f-fe2e5ddf1e50
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/7c1f6eed-e1f4-4411-9a1f-fe2e5ddf1e50",
"name": "7c1f6eed-e1f4-4411-9a1f-fe2e5ddf1e50",
"status": "Failed",
"startTime": "2026-09-27T23:02:36.690582806Z",
"endTime": "2026-09-27T23:31:45.971437914Z",
"error": {
"code": "InternalServerError",
"message": "[cosmosCluster] .api.url is empty; [servingCABundle] ServingCABundle not yet populated; [clusterServiceClusterStatus] cluster service has not been successfully created; [hypershiftHostedCluster] hosted cluster state not cached yet"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_v20260901preview.go:86]: failed to create HCP cluster "v20260901" via v20260901preview
Unexpected error:
<*fmt.wrapError | 0xc000992660>:
failed to create HCP cluster v20260901: failed waiting for cluster="v20260901" in resourcegroup="v20260901preview-nwqdmg89c6p9" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/7c1f6eed-e1f4-4411-9a1f-fe2e5ddf1e50
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/7c1f6eed-e1f4-4411-9a1f-fe2e5ddf1e50",
"name": "7c1f6eed-e1f4-4411-9a1f-fe2e5ddf1e50",
"status": "Failed",
"startTime": "2026-09-27T23:02:36.690582806Z",
"endTime": "2026-09-27T23:31:45.971437914Z",
"error": {
"code": "InternalServerError",
"message": "[cosmosCluster] .api.url is empty; [servingCABundle] ServingCABundle not yet populated; [clusterServiceClusterStatus] cluster service has not been successfully created; [hypershiftHostedCluster] hosted cluster state not cached yet"
}
}
--------------------------------------------------------------------------------
...
occurredContributing tests (33)
Affected runs (5)
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
ERROR CODE: InvalidRequestContent; detail message Required value: field cannot be null; provider Microsoft.RedHatOpenShiftfull failure pattern: ERROR CODE: InvalidRequestContent; detail message Required value: field cannot be null; provider Microsoft.RedHatOpenShift Signal: IndeterminateSignal: Indeterminate — no prior history; active 2 day(s) — New failure pattern — no prior history | e2e | 3 | 17.65%3 of 17 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 2 day(s) — New failure pattern — no prior history | Show trend detailsSep 20: 0 · Sep 21: 0 · Sep 22: 0 · Sep 23: 14 · Sep 24: 7 · Sep 25: 0 · Sep 26: 0 · Sep 27: 0 | none | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (3)fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_ingress.go:87]: failed to create HCP cluster "private-ingress" with private ingress
Unexpected error:
<*fmt.wrapError | 0xc000ab7ea0>:
failed to create HCP cluster private-ingress: failed starting cluster creation "private-ingress" in resourcegroup="private-ingress-r6szljgv5m4p": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/private-ingress-r6szljgv5m4p/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/private-ingress
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Required value: field cannot be null",
"target": "properties.platform.vnetIntegrationSubnetId"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_ingress.go:87]: failed to create HCP cluster "private-ingress" with private ingress
Unexpected error:
<*fmt.wrapError | 0xc000ab7ea0>:
failed to create HCP cluster private-ingress: failed starting cluster creation "private-ingress" in resourcegroup="private-ingress-r6szljgv5m4p": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/private-ingress-r6szljgv5m4p/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/private-ingress
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Required value: field cannot be null",
"target": "properties.platform.vnetIntegrationSubnetId"
}
}
--------------------------------------------------------------------------------
...
occurredfail [github.com/Azure/ARO-HCP/test/e2e/idms_lifecycle.go:113]: failed to create HCP cluster with ImageDigestMirrors
Unexpected error:
<*fmt.wrapError | 0xc000e90d80>:
failed to create HCP cluster idms-e2e-hcp-cluster: failed starting cluster creation "idms-e2e-hcp-cluster" in resourcegroup="idms-pc4xbw2wvdgz": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/idms-pc4xbw2wvdgz/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/idms-e2e-hcp-cluster
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Required value: field cannot be null",
"target": "properties.platform.vnetIntegrationSubnetId"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/idms_lifecycle.go:113]: failed to create HCP cluster with ImageDigestMirrors
Unexpected error:
<*fmt.wrapError | 0xc000e90d80>:
failed to create HCP cluster idms-e2e-hcp-cluster: failed starting cluster creation "idms-e2e-hcp-cluster" in resourcegroup="idms-pc4xbw2wvdgz": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/idms-pc4xbw2wvdgz/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/idms-e2e-hcp-cluster
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Required value: field cannot be null",
"target": "properties.platform.vnetIntegrationSubnetId"
}
}
--------------------------------------------------------------------------------
...
occurredfail [github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:140]: failed to start cluster creation
Unexpected error:
<*exported.ResponseError | 0xc0015fdec0>:
PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/delayed-rbac-gk9hkpv4f4bq/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/delayed-rbac-cluster
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Required value: field cannot be null",
"target": "properties.platform.vnetIntegrationSubnetId"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:140]: failed to start cluster creation
Unexpected error:
<*exported.ResponseError | 0xc0015fdec0>:
PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/delayed-rbac-gk9hkpv4f4bq/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/delayed-rbac-cluster
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Required value: field cannot be null",
"target": "properties.platform.vnetIntegrationSubnetId"
}
}
--------------------------------------------------------------------------------
...
occurredContributing tests (14)
Affected runs (3)
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
ERROR CODE: InternalServerError; detail message [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not ...full failure pattern: ERROR CODE: InternalServerError; detail message [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated; provider Microsoft.RedHatOpenShift Signal: IndeterminateSignal: Indeterminate — no prior history; active 3 day(s) — New failure pattern — no prior history | e2e | 3 | 17.65%3 of 17 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 3 day(s) — New failure pattern — no prior history | Show trend detailsSep 20: 0 · Sep 21: 1 · Sep 22: 0 · Sep 23: 0 · Sep 24: 0 · Sep 25: 1 · Sep 26: 0 · Sep 27: 2 | none | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (3)fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:123]: HCP cluster rg-nightly-4-22-j6wfl7rfmftf/cluster-nightly-4-22-nm6bdd should provision
Unexpected error:
<*fmt.wrapError | 0xc000b18ea0>:
failed to create HCP cluster cluster-nightly-4-22-nm6bdd: failed waiting for cluster="cluster-nightly-4-22-nm6bdd" in resourcegroup="rg-nightly-4-22-j6wfl7rfmftf" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/30dbd006-3693-4ff9-83b2-2e342bb7120d
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/30dbd006-3693-4ff9-83b2-2e342bb7120d",
"name": "30dbd006-3693-4ff9-83b2-2e342bb7120d",
"status": "Failed",
"startTime": "2026-09-27T23:57:15.981848903Z",
"endTime": "2026-09-28T00:26:23.602322446Z",
"error": {
"code": "InternalServerError",
"message": "[cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:123]: HCP cluster rg-nightly-4-22-j6wfl7rfmftf/cluster-nightly-4-22-nm6bdd should provision
Unexpected error:
<*fmt.wrapError | 0xc000b18ea0>:
failed to create HCP cluster cluster-nightly-4-22-nm6bdd: failed waiting for cluster="cluster-nightly-4-22-nm6bdd" in resourcegroup="rg-nightly-4-22-j6wfl7rfmftf" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/30dbd006-3693-4ff9-83b2-2e342bb7120d
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/30dbd006-3693-4ff9-83b2-2e342bb7120d",
"name": "30dbd006-3693-4ff9-83b2-2e342bb7120d",
"status": "Failed",
"startTime": "2026-09-27T23:57:15.981848903Z",
"endTime": "2026-09-28T00:26:23.602322446Z",
"error": {
"code": "InternalServerError",
"message": "[cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated"
}
}
--------------------------------------------------------------------------------
...
occurredfail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:133]: failed to create HCP cluster "cp-ystream-upgrade-4-23-fbjrkv" at install version 4.22.0-0.nightly-multi-2026-07-08-192436
Unexpected error:
<*fmt.wrapError | 0xc000adaa00>:
failed to create HCP cluster cp-ystream-upgrade-4-23-fbjrkv: failed waiting for cluster="cp-ystream-upgrade-4-23-fbjrkv" in resourcegroup="rg-cp-ystream-upgrade-4-23-xs76x2g6dgbl" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/2c0d0d6a-5f02-484a-8b51-9407d1b552aa
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/2c0d0d6a-5f02-484a-8b51-9407d1b552aa",
"name": "2c0d0d6a-5f02-484a-8b51-9407d1b552aa",
"status": "Failed",
"startTime": "2026-09-27T23:57:39.891778199Z",
"endTime": "2026-09-28T00:26:43.608164604Z",
"error": {
"code": "InternalServerError",
"message": "[cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:133]: failed to create HCP cluster "cp-ystream-upgrade-4-23-fbjrkv" at install version 4.22.0-0.nightly-multi-2026-07-08-192436
Unexpected error:
<*fmt.wrapError | 0xc000adaa00>:
failed to create HCP cluster cp-ystream-upgrade-4-23-fbjrkv: failed waiting for cluster="cp-ystream-upgrade-4-23-fbjrkv" in resourcegroup="rg-cp-ystream-upgrade-4-23-xs76x2g6dgbl" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/2c0d0d6a-5f02-484a-8b51-9407d1b552aa
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/2c0d0d6a-5f02-484a-8b51-9407d1b552aa",
"name": "2c0d0d6a-5f02-484a-8b51-9407d1b552aa",
"status": "Failed",
"startTime": "2026-09-27T23:57:39.891778199Z",
"endTime": "2026-09-28T00:26:43.608164604Z",
"error": {
"code": "InternalServerError",
"message": "[cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated"
}
}
--------------------------------------------------------------------------------
...
occurredfail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:123]: HCP cluster rg-nightly-5-1-7q8vqcfshfst/cluster-nightly-5-1-htcp7l should provision
Unexpected error:
<*fmt.wrapError | 0xc000acaa60>:
failed to create HCP cluster cluster-nightly-5-1-htcp7l: failed waiting for cluster="cluster-nightly-5-1-htcp7l" in resourcegroup="rg-nightly-5-1-7q8vqcfshfst" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/3dbd0e68-03bb-49be-ad53-7a10f8750a36
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/3dbd0e68-03bb-49be-ad53-7a10f8750a36",
"name": "3dbd0e68-03bb-49be-ad53-7a10f8750a36",
"status": "Failed",
"startTime": "2026-09-25T23:32:03.179098884Z",
"endTime": "2026-09-26T00:01:10.002965076Z",
"error": {
"code": "InternalServerError",
"message": "[cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:123]: HCP cluster rg-nightly-5-1-7q8vqcfshfst/cluster-nightly-5-1-htcp7l should provision
Unexpected error:
<*fmt.wrapError | 0xc000acaa60>:
failed to create HCP cluster cluster-nightly-5-1-htcp7l: failed waiting for cluster="cluster-nightly-5-1-htcp7l" in resourcegroup="rg-nightly-5-1-7q8vqcfshfst" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/3dbd0e68-03bb-49be-ad53-7a10f8750a36
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/3dbd0e68-03bb-49be-ad53-7a10f8750a36",
"name": "3dbd0e68-03bb-49be-ad53-7a10f8750a36",
"status": "Failed",
"startTime": "2026-09-25T23:32:03.179098884Z",
"endTime": "2026-09-26T00:01:10.002965076Z",
"error": {
"code": "InternalServerError",
"message": "[cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated"
}
}
--------------------------------------------------------------------------------
...
occurredContributing tests (3)
Affected runs (3)
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
ERROR CODE: InvalidRequestContent; detail message Invalid value: "4.22": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when c...full failure pattern: ERROR CODE: InvalidRequestContent; detail message Invalid value: "4.22": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is "nightly"; provider Microsoft.RedHatOpenShift Signal: IndeterminateSignal: Indeterminate — seen in 1 prior week(s); active 4 day(s) | e2e | 3 | 17.65%3 of 17 job runs affected | IndeterminateSignal: Indeterminate — seen in 1 prior week(s); active 4 day(s) | Show trend detailsSep 20: 1 · Sep 21: 1 · Sep 22: 1 · Sep 23: 1 · Sep 24: 0 · Sep 25: 0 · Sep 26: 0 · Sep 27: 0 | none | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (3)fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:91]: failed to create HCP cluster "private-kas" with private KAS
Unexpected error:
<*fmt.wrapError | 0xc000587ac0>:
failed to create HCP cluster private-kas: failed starting cluster creation "private-kas" in resourcegroup="private-kas-f9strbqhzqsp": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/private-kas-f9strbqhzqsp/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/private-kas
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"4.22\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"",
"target": "properties.version.id"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:91]: failed to create HCP cluster "private-kas" with private KAS
Unexpected error:
<*fmt.wrapError | 0xc000587ac0>:
failed to create HCP cluster private-kas: failed starting cluster creation "private-kas" in resourcegroup="private-kas-f9strbqhzqsp": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/private-kas-f9strbqhzqsp/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/private-kas
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"4.22\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"",
"target": "properties.version.id"
}
}
--------------------------------------------------------------------------------
...
occurredfail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:90]: failed to create HCP cluster "private-kas" with private KAS
Unexpected error:
<*fmt.wrapError | 0xc00103f360>:
failed to create HCP cluster private-kas: failed starting cluster creation "private-kas" in resourcegroup="private-kas-xqlnpq2wtgvl": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/private-kas-xqlnpq2wtgvl/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/private-kas
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"4.22\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"",
"target": "properties.version.id"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:90]: failed to create HCP cluster "private-kas" with private KAS
Unexpected error:
<*fmt.wrapError | 0xc00103f360>:
failed to create HCP cluster private-kas: failed starting cluster creation "private-kas" in resourcegroup="private-kas-xqlnpq2wtgvl": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/private-kas-xqlnpq2wtgvl/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/private-kas
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"4.22\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"",
"target": "properties.version.id"
}
}
--------------------------------------------------------------------------------
...
occurredfail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:90]: failed to create HCP cluster "private-kas" with private KAS
Unexpected error:
<*fmt.wrapError | 0xc00062e820>:
failed to create HCP cluster private-kas: failed starting cluster creation "private-kas" in resourcegroup="private-kas-rnt89sjcb8mz": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/private-kas-rnt89sjcb8mz/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/private-kas
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"4.22\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"",
"target": "properties.version.id"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:90]: failed to create HCP cluster "private-kas" with private KAS
Unexpected error:
<*fmt.wrapError | 0xc00062e820>:
failed to create HCP cluster private-kas: failed starting cluster creation "private-kas" in resourcegroup="private-kas-rnt89sjcb8mz": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/private-kas-rnt89sjcb8mz/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/private-kas
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"4.22\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"",
"target": "properties.version.id"
}
}
--------------------------------------------------------------------------------
...
occurredContributing tests (1)
Affected runs (3)
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
cluster CA data must contain exactly one PEM blockfull failure pattern: cluster CA data must contain exactly one PEM block Signal: IndeterminateSignal: Indeterminate — no prior history; active 2 day(s) — New failure pattern — no prior history | e2e | 2 | 11.76%2 of 17 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 2 day(s) — New failure pattern — no prior history | Show trend detailsSep 20: 0 · Sep 21: 0 · Sep 22: 1 · Sep 23: 1 · Sep 24: 0 · Sep 25: 0 · Sep 26: 0 · Sep 27: 0 | none | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (1)fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:202]: cluster CA data must contain exactly one PEM block
Expected
<[]uint8 | len:6010, cap:6012>: ...
to be empty
fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:202]: cluster CA data must contain exactly one PEM block
Expected
<[]uint8 | len:6010, cap:6012>: ...
to be emptyContributing tests (2)
Affected runs (2)
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
failed to create nodepool <nodepool>: failed waiting for nodepool="<nodepool>" for cluster "<cluster>" in resourcegroup="<resource-group>" to finish creating: GET <url>full failure pattern: failed to create nodepool <nodepool>: failed waiting for nodepool="<nodepool>" for cluster "<cluster>" in resourcegroup="<resource-group>" to finish creating: GET <url> Signal: IndeterminateSignal: Indeterminate — seen in 1 prior week(s); active 2 day(s) | e2e | 2 | 11.76%2 of 17 job runs affected | IndeterminateSignal: Indeterminate — seen in 1 prior week(s); active 2 day(s) | Show trend detailsSep 20: 0 · Sep 21: 0 · Sep 22: 0 · Sep 23: 0 · Sep 24: 1 · Sep 25: 1 · Sep 26: 0 · Sep 27: 0 | none | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (2)fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_delete.go:128]: failed to create node pools
Expected success, but got an error:
<*fmt.wrapError | 0xc00078b060>:
failed to create NodePool np-deletable: failed waiting for nodepool="np-deletable" for cluster "np-delete-hcp-cluster" in resourcegroup="nodepool-delete-gb52bkffcwft" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/db1ab9c2-9577-48f8-9f15-b41d89774bb7
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/db1ab9c2-9577-48f8-9f15-b41d89774bb7",
"name": "db1ab9c2-9577-48f8-9f15-b41d89774bb7",
"status": "Failed",
"startTime": "2026-09-25T23:59:44.019265481Z",
"endTime": "2026-09-26T00:18:50.149741392Z",
"error": {
"code": "InternalServerError",
"message": "node pool creation did not complete before the deadline"
}
}
--------------------------------------------------------------------------------
...
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_delete.go:128]: failed to create node pools
Expected success, but got an error:
<*fmt.wrapError | 0xc00078b060>:
failed to create NodePool np-deletable: failed waiting for nodepool="np-deletable" for cluster "np-delete-hcp-cluster" in resourcegroup="nodepool-delete-gb52bkffcwft" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/db1ab9c2-9577-48f8-9f15-b41d89774bb7
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/db1ab9c2-9577-48f8-9f15-b41d89774bb7",
"name": "db1ab9c2-9577-48f8-9f15-b41d89774bb7",
"status": "Failed",
"startTime": "2026-09-25T23:59:44.019265481Z",
"endTime": "2026-09-26T00:18:50.149741392Z",
"error": {
"code": "InternalServerError",
"message": "node pool creation did not complete before the deadline"
}
}
--------------------------------------------------------------------------------
...fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_delete.go:128]: failed to create node pools
Expected success, but got an error:
<*fmt.wrapError | 0xc001321160>:
failed to create NodePool np-deletable: failed waiting for nodepool="np-deletable" for cluster "np-delete-hcp-cluster" in resourcegroup="nodepool-delete-l2xrb7ls9tkm" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4e3486a0-a8be-491f-b7ba-de40bdc6a4c5
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4e3486a0-a8be-491f-b7ba-de40bdc6a4c5",
"name": "4e3486a0-a8be-491f-b7ba-de40bdc6a4c5",
"status": "Failed",
"startTime": "2026-09-24T03:02:54.530721397Z",
"endTime": "2026-09-24T03:21:54.293711104Z",
"error": {
"code": "InternalServerError",
"message": "node pool creation did not complete before the deadline"
}
}
--------------------------------------------------------------------------------
...
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_delete.go:128]: failed to create node pools
Expected success, but got an error:
<*fmt.wrapError | 0xc001321160>:
failed to create NodePool np-deletable: failed waiting for nodepool="np-deletable" for cluster "np-delete-hcp-cluster" in resourcegroup="nodepool-delete-l2xrb7ls9tkm" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4e3486a0-a8be-491f-b7ba-de40bdc6a4c5
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4e3486a0-a8be-491f-b7ba-de40bdc6a4c5",
"name": "4e3486a0-a8be-491f-b7ba-de40bdc6a4c5",
"status": "Failed",
"startTime": "2026-09-24T03:02:54.530721397Z",
"endTime": "2026-09-24T03:21:54.293711104Z",
"error": {
"code": "InternalServerError",
"message": "node pool creation did not complete before the deadline"
}
}
--------------------------------------------------------------------------------
...Contributing tests (2)
Affected runs (2)
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
ERROR CODE: InternalServerError; detail message [hypershiftHostedCluster] hosted cluster is not available: KubeconfigWaitingForCreate: Waiting for hosted control plane kubeconfig ...full failure pattern: ERROR CODE: InternalServerError; detail message [hypershiftHostedCluster] hosted cluster is not available: KubeconfigWaitingForCreate: Waiting for hosted control plane kubeconfig to be created; hosted cluster degraded: UnavailableReplicas: [capi-provider]; provider Microsoft.RedHatOpenShift Signal: IndeterminateSignal: Indeterminate — seen in 1 prior week(s); active 1 day(s) | e2e | 1 | 5.88%1 of 17 job runs affected | IndeterminateSignal: Indeterminate — seen in 1 prior week(s); active 1 day(s) | Show trend detailsSep 20: 0 · Sep 21: 0 · Sep 22: 0 · Sep 23: 0 · Sep 24: 1 · Sep 25: 0 · Sep 26: 0 · Sep 27: 0 | none | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (1)fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:146]: failed to create HCP cluster np-version-upgrade-cluster-ml64wq with version 4.21.34
Unexpected error:
<*fmt.wrapError | 0xc0013c02a0>:
failed to create HCP cluster np-version-upgrade-cluster-ml64wq: failed waiting for cluster="np-version-upgrade-cluster-ml64wq" in resourcegroup="rg-np-version-upgrade-ml64wq-zkbzk7hvlvck" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/2d3c2c93-508a-4ae2-aa7a-7a7d3546ac58
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/2d3c2c93-508a-4ae2-aa7a-7a7d3546ac58",
"name": "2d3c2c93-508a-4ae2-aa7a-7a7d3546ac58",
"status": "Failed",
"startTime": "2026-09-25T00:48:56.872501334Z",
"endTime": "2026-09-25T01:18:00.947451943Z",
"error": {
"code": "InternalServerError",
"message": "[hypershiftHostedCluster] hosted cluster is not available: KubeconfigWaitingForCreate: Waiting for hosted control plane kubeconfig to be created; hosted cluster degraded: UnavailableReplicas: capi-provider deployment has 1 unavailable replicas"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:146]: failed to create HCP cluster np-version-upgrade-cluster-ml64wq with version 4.21.34
Unexpected error:
<*fmt.wrapError | 0xc0013c02a0>:
failed to create HCP cluster np-version-upgrade-cluster-ml64wq: failed waiting for cluster="np-version-upgrade-cluster-ml64wq" in resourcegroup="rg-np-version-upgrade-ml64wq-zkbzk7hvlvck" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/2d3c2c93-508a-4ae2-aa7a-7a7d3546ac58
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/2d3c2c93-508a-4ae2-aa7a-7a7d3546ac58",
"name": "2d3c2c93-508a-4ae2-aa7a-7a7d3546ac58",
"status": "Failed",
"startTime": "2026-09-25T00:48:56.872501334Z",
"endTime": "2026-09-25T01:18:00.947451943Z",
"error": {
"code": "InternalServerError",
"message": "[hypershiftHostedCluster] hosted cluster is not available: KubeconfigWaitingForCreate: Waiting for hosted control plane kubeconfig to be created; hosted cluster degraded: UnavailableReplicas: capi-provider deployment has 1 unavailable replicas"
}
}
--------------------------------------------------------------------------------
...
occurredContributing tests (1)
Affected runs (1)
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Customer should be able to test admin credentials before cluster ready, then full admin credential lifecycle: Cluster provisioning failedfull failure pattern: Customer should be able to test admin credentials before cluster ready, then full admin credential lifecycle: Cluster provisioning failed Signal: IndeterminateSignal: Indeterminate — seen in 1 prior week(s); active 1 day(s) | e2e | 1 | 5.88%1 of 17 job runs affected | IndeterminateSignal: Indeterminate — seen in 1 prior week(s); active 1 day(s) | Show trend detailsSep 20: 0 · Sep 21: 0 · Sep 22: 0 · Sep 23: 0 · Sep 24: 0 · Sep 25: 0 · Sep 26: 0 · Sep 27: 1 | none | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (1)fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:158]: Cluster provisioning failed fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:158]: Cluster provisioning failed Contributing tests (1)
Affected runs (1)
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
cluster entered terminal Failed state after role assignment deploymentfull failure pattern: cluster entered terminal Failed state after role assignment deployment Signal: IndeterminateSignal: Indeterminate — seen in 1 prior week(s); active 1 day(s) | e2e | 1 | 5.88%1 of 17 job runs affected | IndeterminateSignal: Indeterminate — seen in 1 prior week(s); active 1 day(s) | Show trend detailsSep 20: 0 · Sep 21: 0 · Sep 22: 0 · Sep 23: 0 · Sep 24: 0 · Sep 25: 0 · Sep 26: 0 · Sep 27: 1 | none | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (1)fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:227]: Timed out after 2580.002s.
cluster should eventually succeed after role assignments are created
The function passed to Eventually failed at /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:223 with:
cluster entered terminal Failed state after role assignment deployment
Expected
<armredhatopenshifthcp.ProvisioningState>: ...
not to equal
<armredhatopenshifthcp.ProvisioningState>: ...
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:227]: Timed out after 2580.002s.
cluster should eventually succeed after role assignments are created
The function passed to Eventually failed at /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:223 with:
cluster entered terminal Failed state after role assignment deployment
Expected
<armredhatopenshifthcp.ProvisioningState>: ...
not to equal
<armredhatopenshifthcp.ProvisioningState>: ...Contributing tests (1)
Affected runs (1)
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
ERROR CODE: InternalServerError; detail code OperationNotAllowed; detail message Operation 'retrieveBootDiagnosticsData' is not allowed on VM 'cilium-cluster-cilium-np-44b92-<id>'...full failure pattern: ERROR CODE: InternalServerError; detail code OperationNotAllowed; detail message Operation 'retrieveBootDiagnosticsData' is not allowed on VM 'cilium-cluster-cilium-np-44b92-<id>' since another operation is in progress.; provider Microsoft.Compute Signal: IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | e2e | 1 | 5.88%1 of 17 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | Show trend detailsSep 20: 0 · Sep 21: 0 · Sep 22: 0 · Sep 23: 0 · Sep 24: 0 · Sep 25: 1 · Sep 26: 0 · Sep 27: 0 | none | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (1)fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:205]: failed to verify nodes are Ready with Cilium CNI for cluster "cilium-cluster"
Unexpected error:
<*errors.joinError | 0xc000ff01b0>:
failed to create NodePool cilium-np: failed waiting for nodepool="cilium-np" for cluster "cilium-cluster" in resourcegroup="complex-cilium-kv-d8dncw5q6xr2" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4cccec67-451f-41c1-957a-1453bb0ac735
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4cccec67-451f-41c1-957a-1453bb0ac735",
"name": "4cccec67-451f-41c1-957a-1453bb0ac735",
"status": "Failed",
"startTime": "2026-09-25T02:15:20.351974211Z",
"endTime": "2026-09-25T02:34:26.038545442Z",
"error": {
"code": "InternalServerError",
"message": "node pool creation did not complete before the deadline"
}
}
--------------------------------------------------------------------------------
encountered errors downloading console logs:
- VM "cilium-cluster-cilium-np-44b92-qd24g": failed to retrieve boot diagnostics data for VM "cilium-cluster-cilium-np-44b92-qd24g" after 3 attempts, last error: POST https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/complex-cilium-kv-d8dncw5q6xr2--managed/providers/Microsoft.Compute/virtualMachines/cilium-cluster-cilium-np-44b92-qd24g/retrieveBootDiagnosticsData
--------------------------------------------------------------------------------
RESPONSE 409: 409 Conflict
ERROR CODE: OperationNotAllowed
--------------------------------------------------------------------------------
{
"error": {
"code": "OperationNotAllowed",
"message": "Operation 'retrieveBootDiagnosticsData' is not allowed on VM 'cilium-cluster-cilium-np-44b92-qd24g' since another operation is in progress."
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:205]: failed to verify nodes are Ready with Cilium CNI for cluster "cilium-cluster"
Unexpected error:
<*errors.joinError | 0xc000ff01b0>:
failed to create NodePool cilium-np: failed waiting for nodepool="cilium-np" for cluster "cilium-cluster" in resourcegroup="complex-cilium-kv-d8dncw5q6xr2" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4cccec67-451f-41c1-957a-1453bb0ac735
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4cccec67-451f-41c1-957a-1453bb0ac735",
"name": "4cccec67-451f-41c1-957a-1453bb0ac735",
"status": "Failed",
"startTime": "2026-09-25T02:15:20.351974211Z",
"endTime": "2026-09-25T02:34:26.038545442Z",
"error": {
"code": "InternalServerError",
"message": "node pool creation did not complete before the deadline"
}
}
--------------------------------------------------------------------------------
encountered errors downloading console logs:
- VM "cilium-cluster-cilium-np-44b92-qd24g": failed to retrieve boot diagnostics data for VM "cilium-cluster-cilium-np-44b92-qd24g" after 3 attempts, last error: POST https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/complex-cilium-kv-d8dncw5q6xr2--managed/providers/Microsoft.Compute/virtualMachines/cilium-cluster-cilium-np-44b92-qd24g/retrieveBootDiagnosticsData
--------------------------------------------------------------------------------
RESPONSE 409: 409 Conflict
ERROR CODE: OperationNotAllowed
--------------------------------------------------------------------------------
{
"error": {
"code": "OperationNotAllowed",
"message": "Operation 'retrieveBootDiagnosticsData' is not allowed on VM 'cilium-cluster-cilium-np-44b92-qd24g' since another operation is in progress."
}
}
--------------------------------------------------------------------------------
...
occurredContributing tests (1)
Affected runs (1)
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
ERROR CODE: InternalServerError; detail message [hypershiftHostedCluster] hosted cluster has no control plane endpoint host; provider Microsoft.RedHatOpenShiftfull failure pattern: ERROR CODE: InternalServerError; detail message [hypershiftHostedCluster] hosted cluster has no control plane endpoint host; provider Microsoft.RedHatOpenShift Signal: IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | e2e | 1 | 5.88%1 of 17 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | Show trend detailsSep 20: 0 · Sep 21: 0 · Sep 22: 0 · Sep 23: 0 · Sep 24: 0 · Sep 25: 0 · Sep 26: 0 · Sep 27: 1 | none | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (1)fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:133]: failed to create HCP cluster "cp-ystream-upgrade-5-0-tssqn7" at install version 4.22.0-0.nightly-multi-2026-07-08-192436
Unexpected error:
<*fmt.wrapError | 0xc0008d6f80>:
failed to create HCP cluster cp-ystream-upgrade-5-0-tssqn7: failed waiting for cluster="cp-ystream-upgrade-5-0-tssqn7" in resourcegroup="rg-cp-ystream-upgrade-5-0-s5ql4kptw4ct" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/8948f23c-1b13-4b8b-b8b8-d745d0f16492
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/8948f23c-1b13-4b8b-b8b8-d745d0f16492",
"name": "8948f23c-1b13-4b8b-b8b8-d745d0f16492",
"status": "Failed",
"startTime": "2026-09-27T23:57:40.978774448Z",
"endTime": "2026-09-28T00:26:43.612304644Z",
"error": {
"code": "InternalServerError",
"message": "[hypershiftHostedCluster] hosted cluster has no control plane endpoint host"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:133]: failed to create HCP cluster "cp-ystream-upgrade-5-0-tssqn7" at install version 4.22.0-0.nightly-multi-2026-07-08-192436
Unexpected error:
<*fmt.wrapError | 0xc0008d6f80>:
failed to create HCP cluster cp-ystream-upgrade-5-0-tssqn7: failed waiting for cluster="cp-ystream-upgrade-5-0-tssqn7" in resourcegroup="rg-cp-ystream-upgrade-5-0-s5ql4kptw4ct" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/8948f23c-1b13-4b8b-b8b8-d745d0f16492
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/8948f23c-1b13-4b8b-b8b8-d745d0f16492",
"name": "8948f23c-1b13-4b8b-b8b8-d745d0f16492",
"status": "Failed",
"startTime": "2026-09-27T23:57:40.978774448Z",
"endTime": "2026-09-28T00:26:43.612304644Z",
"error": {
"code": "InternalServerError",
"message": "[hypershiftHostedCluster] hosted cluster has no control plane endpoint host"
}
}
--------------------------------------------------------------------------------
...
occurredContributing tests (1)
Affected runs (1)
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
ERROR CODE: InvalidRequestContent; detail code Conflict; detail message Cannot request credential while credentials are being revoked; provider Microsoft.RedHatOpenShiftfull failure pattern: ERROR CODE: InvalidRequestContent; detail code Conflict; detail message Cannot request credential while credentials are being revoked; provider Microsoft.RedHatOpenShift Signal: IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | e2e | 1 | 5.88%1 of 17 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | Show trend detailsSep 20: 0 · Sep 21: 0 · Sep 22: 0 · Sep 23: 0 · Sep 24: 0 · Sep 25: 0 · Sep 26: 1 · Sep 27: 0 | none | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (1)fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:267]: failed to request new admin credentials after revocation for cluster "admin-cred-lifecycle-7t664q"
Unexpected error:
<*fmt.wrapErrors | 0xc000847980>:
20240610 credential request failed: failed to start credential request: POST https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/admin-credential-lifecycle-test-rwn8v4w9qpfg/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/admin-cred-lifecycle-7t664q/requestAdminCredential
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Required value",
"target": "certificateSigningRequest"
}
}
--------------------------------------------------------------------------------
; 2026-09-01-preview fallback failed: failed to begin admin credential request for hcpCluster="admin-cred-lifecycle-7t664q" in resourcegroup="admin-credential-lifecycle-test-rwn8v4w9qpfg": POST https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/admin-credential-lifecycle-test-rwn8v4w9qpfg/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/admin-cred-lifecycle-7t664q/requestAdminCredential
--------------------------------------------------------------------------------
RESPONSE 409: 409 Conflict
ERROR CODE: Conflict
--------------------------------------------------------------------------------
{
"error": {
"code": "Conflict",
"message": "Cannot request credential while credentials are being revoked",
"target": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/admin-credential-lifecycle-test-rwn8v4w9qpfg/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/admin-cred-lifecycle-7t664q"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:267]: failed to request new admin credentials after revocation for cluster "admin-cred-lifecycle-7t664q"
Unexpected error:
<*fmt.wrapErrors | 0xc000847980>:
20240610 credential request failed: failed to start credential request: POST https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/admin-credential-lifecycle-test-rwn8v4w9qpfg/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/admin-cred-lifecycle-7t664q/requestAdminCredential
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Required value",
"target": "certificateSigningRequest"
}
}
--------------------------------------------------------------------------------
; 2026-09-01-preview fallback failed: failed to begin admin credential request for hcpCluster="admin-cred-lifecycle-7t664q" in resourcegroup="admin-credential-lifecycle-test-rwn8v4w9qpfg": POST https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/admin-credential-lifecycle-test-rwn8v4w9qpfg/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/admin-cred-lifecycle-7t664q/requestAdminCredential
--------------------------------------------------------------------------------
RESPONSE 409: 409 Conflict
ERROR CODE: Conflict
--------------------------------------------------------------------------------
{
"error": {
"code": "Conflict",
"message": "Cannot request credential while credentials are being revoked",
"target": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/admin-credential-lifecycle-test-rwn8v4w9qpfg/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/admin-cred-lifecycle-7t664q"
}
}
--------------------------------------------------------------------------------
...
occurredContributing tests (1)
Affected runs (1)
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Operation cannot be fulfilled on nodes "<node>": the object has been modified; please apply your changes to the latest version and try againfull failure pattern: Operation cannot be fulfilled on nodes "<node>": the object has been modified; please apply your changes to the latest version and try again Signal: IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | e2e | 1 | 5.88%1 of 17 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | Show trend detailsSep 20: 0 · Sep 21: 0 · Sep 22: 0 · Sep 23: 0 · Sep 24: 1 · Sep 25: 0 · Sep 26: 0 · Sep 27: 0 | none | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (1)fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_cordon.go:346]: failed to uncordon node a8y5x1l0f9x5u8y-cordon-np-8kx7p-97f6x
Unexpected error:
<*errors.StatusError | 0xc00103ab40>:
Operation cannot be fulfilled on nodes "a8y5x1l0f9x5u8y-cordon-np-8kx7p-97f6x": the object has been modified; please apply your changes to the latest version and try again
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_cordon.go:346]: failed to uncordon node a8y5x1l0f9x5u8y-cordon-np-8kx7p-97f6x
Unexpected error:
<*errors.StatusError | 0xc00103ab40>:
Operation cannot be fulfilled on nodes "a8y5x1l0f9x5u8y-cordon-np-8kx7p-97f6x": the object has been modified; please apply your changes to the latest version and try again
...
occurredContributing tests (1)
Affected runs (1)
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||