CIHealth
<
Single Day: May 13▾
Pick a day
>
View JSON API
↗ Open Failure patterns for this day

Environment: INT

Time (UTC)JobFailed atResultPRFailed testsDetails
branch-ci-Azure-ARO-HCP-main-e2e-integration-e2e-parallel
post-goodmerged PR
No additional run metadata captured
 e2e failed n/a 7
e2e (7)
DeleteResourceGroup timed out after <minutes> minutes; context deadline exceeded
23:17:23 UTC · failed at e2e · test Customer should be able to create an HCP cluster then delete it by deleting the customer resource group · suite integration/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delete_cx_rg.go:143]: Unexpected error:
    <*fmt.wrapErrors | 0xc00115e660>: 
    failed to delete resource group, caused by: timeout '60.000000' minutes exceeded during DeleteResourceGroup for resource group cx-rg-cluster-fbqwxv, error: context deadline exceeded
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delete_cx_rg.go:143]: Unexpected error:
    <*fmt.wrapErrors | 0xc00115e660>: 
    failed to delete resource group, caused by: timeout '60.000000' minutes exceeded during DeleteResourceGroup for resource group cx-rg-cluster-fbqwxv, error: context deadline exceeded
    ...
occurred
failed to cleanup resource group: at least one HCP cluster <cluster> to delete: failed waiting for hcpcluster="<cluster>" in resourcegroup="<resource-group>" to finish deleting: context canceled
23:17:23 UTC · failed at e2e · test Customer should be able to test admin credentials before cluster ready, then full admin credential lifecycle · suite integration/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:249] A node timeout occurred and then the following failure was recorded in the timedout node before it exited:
Unexpected error:
    <*errors.joinError | 0xc001312198>: 
    failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster="admin-cred-lifecycle-9mdxlx" in resourcegroup="admin-credential-lifecycle-test-5szj8p" to finish deleting: context canceled
    ...
occurred  
fail [:0]: A node timeout occurred
fail [github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:249] A node timeout occurred and then the following failure was recorded in the timedout node before it exited:
Unexpected error:
    <*errors.joinError | 0xc001312198>: 
    failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster="admin-cred-lifecycle-9mdxlx" in resourcegroup="admin-credential-lifecycle-test-5szj8p" to finish deleting: context canceled
    ...
occurred  
fail [:0]: A node timeout occurred
failed to cleanup resource group: at least one HCP cluster <cluster> to delete: failed waiting for hcpcluster="<cluster>" in resourcegroup="<resource-group>" to finish deleting: context canceled
23:17:23 UTC · failed at e2e · test Customer should be able to create an HCP cluster with back-level version 4.19 · suite integration/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:249] A node timeout occurred and then the following failure was recorded in the timedout node before it exited:
Unexpected error:
    <*errors.joinError | 0xc000df61e0>: 
    failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster="cluster-ver-4-19" in resourcegroup="rg-cluster-back-version-hgw8jz" to finish deleting: context canceled
    ...
occurred  
fail [:0]: A node timeout occurred
fail [github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:249] A node timeout occurred and then the following failure was recorded in the timedout node before it exited:
Unexpected error:
    <*errors.joinError | 0xc000df61e0>: 
    failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster="cluster-ver-4-19" in resourcegroup="rg-cluster-back-version-hgw8jz" to finish deleting: context canceled
    ...
occurred  
fail [:0]: A node timeout occurred
context deadline exceeded
23:17:23 UTC · failed at e2e · test Customer should be able to create several HCP clusters in their customer resource group, but not in the same managed resource group · suite integration/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:139]: Unexpected error:
    <context.deadlineExceededError>: 
    context deadline exceeded
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:139]: Unexpected error:
    <context.deadlineExceededError>: 
    context deadline exceeded
    ...
occurred
failed to cleanup resource group: at least one HCP cluster <cluster> to delete: failed waiting for hcpcluster="<cluster>" in resourcegroup="<resource-group>" to finish deleting: context canceled
23:17:23 UTC · failed at e2e · test Customer should be able to create an HCP cluster and manage pull secrets · suite integration/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:249] A node timeout occurred and then the following failure was recorded in the timedout node before it exited:
Unexpected error:
    <*errors.joinError | 0xc001726438>: 
    failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster="pullsecret-hcp-cluster" in resourcegroup="pullsecret-test-whxwt2" to finish deleting: context canceled
    ...
occurred  
fail [:0]: A node timeout occurred
fail [github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:249] A node timeout occurred and then the following failure was recorded in the timedout node before it exited:
Unexpected error:
    <*errors.joinError | 0xc001726438>: 
    failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster="pullsecret-hcp-cluster" in resourcegroup="pullsecret-test-whxwt2" to finish deleting: context canceled
    ...
occurred  
fail [:0]: A node timeout occurred
couldn't get current server API group list: the server has asked for the client to provide credentials
23:17:23 UTC · failed at e2e · test Authorized CIDRs Connectivity should allow API access only from authorized VM IP · suite integration/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:362]: Timed out after 300.000s.
The function passed to Eventually failed at /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:347 with:
kubectl should execute successfully from authorized VM with OIDC token
Unexpected error:
    <*errors.errorString | 0xc000bef620>: 
    E0514 00:08:31.255742    2644 memcache.go:265] "Unhandled Error" err="couldn't get current server API group list: the server has asked for the client to provide credentials"
    E0514 00:08:31.268033    2644 memcache.go:265] "Unhandled Error" err="couldn't get current server API group list: the server has asked for the client to provide credentials"
    E0514 00:08:31.294823    2644 memcache.go:265] "Unhandled Error" err="couldn't get current server API group list: the server has asked for the client to provide credentials"
    E0514 00:08:31.307820    2644 memcache.go:265] "Unhandled Error" err="couldn't get current server API group list: the server has asked for the client to provide credentials"
    E0514 00:08:31.324327    2644 memcache.go:265] "Unhandled Error" err="couldn't get current server API group list: the server has asked for the client to provide credentials"
    error: You must be logged in to the server (the server has asked for the client to provide credentials)
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:362]: Timed out after 300.000s.
The function passed to Eventually failed at /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:347 with:
kubectl should execute successfully from authorized VM with OIDC token
Unexpected error:
    <*errors.errorString | 0xc000bef620>: 
    E0514 00:08:31.255742    2644 memcache.go:265] "Unhandled Error" err="couldn't get current server API group list: the server has asked for the client to provide credentials"
    E0514 00:08:31.268033    2644 memcache.go:265] "Unhandled Error" err="couldn't get current server API group list: the server has asked for the client to provide credentials"
    E0514 00:08:31.294823    2644 memcache.go:265] "Unhandled Error" err="couldn't get current server API group list: the server has asked for the client to provide credentials"
    E0514 00:08:31.307820    2644 memcache.go:265] "Unhandled Error" err="couldn't get current server API group list: the server has asked for the client to provide credentials"
    E0514 00:08:31.324327    2644 memcache.go:265] "Unhandled Error" err="couldn't get current server API group list: the server has asked for the client to provide credentials"
    error: You must be logged in to the server (the server has asked for the client to provide credentials)
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message timed out after 29s waiting for resolution of desired version from '4.21.15' cluster version; provider Microsoft.RedHatOpenShift
23:17:23 UTC · failed at e2e · test Customer should be able to successfully upgrade control plane minor version from 4.20 minor to 4.21 minor · suite integration/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:160]: Unexpected error:
    <*fmt.wrapError | 0xc0008449c0>: 
    failed waiting for hcpCluster="cp-ystream-upgrade-4-21-x9glvp" in resourcegroup="rg-cp-ystream-upgrade-4-21-k7tb76" to finish updating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/82580c93-80ca-4782-a780-8beefa3ca171
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/82580c93-80ca-4782-a780-8beefa3ca171",
      "name": "82580c93-80ca-4782-a780-8beefa3ca171",
      "status": "Failed",
      "startTime": "2026-05-14T00:41:19.022062955Z",
      "endTime": "2026-05-14T00:42:47.115197921Z",
      "error": {
        "code": "InvalidRequestContent",
        "message": "timed out after 29s waiting for resolution of desired version from '4.21.15' cluster version"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:160]: Unexpected error:
    <*fmt.wrapError | 0xc0008449c0>: 
    failed waiting for hcpCluster="cp-ystream-upgrade-4-21-x9glvp" in resourcegroup="rg-cp-ystream-upgrade-4-21-k7tb76" to finish updating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/82580c93-80ca-4782-a780-8beefa3ca171
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/82580c93-80ca-4782-a780-8beefa3ca171",
      "name": "82580c93-80ca-4782-a780-8beefa3ca171",
      "status": "Failed",
      "startTime": "2026-05-14T00:41:19.022062955Z",
      "endTime": "2026-05-14T00:42:47.115197921Z",
      "error": {
        "code": "InvalidRequestContent",
        "message": "timed out after 29s waiting for resolution of desired version from '4.21.15' cluster version"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
branch-ci-Azure-ARO-HCP-main-e2e-integration-e2e-parallel
post-goodmerged PR
No additional run metadata captured
 e2e failed n/a 6
e2e (6)
failed to cleanup resource group: at least one HCP cluster <cluster> to delete: failed waiting for hcpcluster="<cluster>" in resourcegroup="<resource-group>" to finish deleting: context canceled
19:53:51 UTC · failed at e2e · test Customer should be able to use workload identity via the cluster OIDC issuer URL · suite integration/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:249] A node timeout occurred and then the following failure was recorded in the timedout node before it exited:
Unexpected error:
    <*errors.joinError | 0xc00166e348>: 
    failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster="oidc-wi-cluster" in resourcegroup="oidc-wi-74ww4g" to finish deleting: context canceled
    ...
occurred  
fail [:0]: A node timeout occurred
fail [github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:249] A node timeout occurred and then the following failure was recorded in the timedout node before it exited:
Unexpected error:
    <*errors.joinError | 0xc00166e348>: 
    failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster="oidc-wi-cluster" in resourcegroup="oidc-wi-74ww4g" to finish deleting: context canceled
    ...
occurred  
fail [:0]: A node timeout occurred
failed to cleanup resource group: at least one HCP cluster <cluster> to delete: failed waiting for hcpcluster="<cluster>" in resourcegroup="<resource-group>" to finish deleting: context canceled
19:53:51 UTC · failed at e2e · test Customer should upgrade and update a nodepool from 4.20.z to 4.20.zLatest · suite integration/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:249] A node timeout occurred and then the following failure was recorded in the timedout node before it exited:
Unexpected error:
    <*errors.joinError | 0xc00106a468>: 
    failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster="np-version-upgrade-cluster-gxhjsl" in resourcegroup="rg-np-version-upgrade-gxhjsl-cphmb6" to finish deleting: context canceled
    ...
occurred  
fail [:0]: A node timeout occurred
fail [github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:249] A node timeout occurred and then the following failure was recorded in the timedout node before it exited:
Unexpected error:
    <*errors.joinError | 0xc00106a468>: 
    failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster="np-version-upgrade-cluster-gxhjsl" in resourcegroup="rg-np-version-upgrade-gxhjsl-cphmb6" to finish deleting: context canceled
    ...
occurred  
fail [:0]: A node timeout occurred
failed to cleanup resource group: at least one HCP cluster <cluster> to delete: failed waiting for hcpcluster="<cluster>" in resourcegroup="<resource-group>" to finish deleting: context canceled
19:53:51 UTC · failed at e2e · test Customer should be able to update node pool labels and taints · suite integration/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:249] A node timeout occurred and then the following failure was recorded in the timedout node before it exited:
Unexpected error:
    <*errors.joinError | 0xc0017783d8>: 
    failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster="cluster-np-labels-taints" in resourcegroup="rg-np-labels-taints-n4jghs" to finish deleting: context canceled
    ...
occurred  
fail [:0]: A node timeout occurred
fail [github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:249] A node timeout occurred and then the following failure was recorded in the timedout node before it exited:
Unexpected error:
    <*errors.joinError | 0xc0017783d8>: 
    failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster="cluster-np-labels-taints" in resourcegroup="rg-np-labels-taints-n4jghs" to finish deleting: context canceled
    ...
occurred  
fail [:0]: A node timeout occurred
failed to cleanup resource group: at least one HCP cluster <cluster> to delete: failed waiting for hcpcluster="<cluster>" in resourcegroup="<resource-group>" to finish deleting: context canceled
19:53:51 UTC · failed at e2e · test Nodepool Ephemeral OS Disk should create a nodepool with ephemeral OS disk when autoRepair is enabled · suite integration/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:249] A node timeout occurred and then the following failure was recorded in the timedout node before it exited:
Unexpected error:
    <*errors.joinError | 0xc0006d4738>: 
    failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster="ephemeral-disk" in resourcegroup="ephemeral-osdisk-fdrwjd" to finish deleting: context canceled
    ...
occurred  
fail [:0]: A node timeout occurred
fail [github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:249] A node timeout occurred and then the following failure was recorded in the timedout node before it exited:
Unexpected error:
    <*errors.joinError | 0xc0006d4738>: 
    failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster="ephemeral-disk" in resourcegroup="ephemeral-osdisk-fdrwjd" to finish deleting: context canceled
    ...
occurred  
fail [:0]: A node timeout occurred
ERROR CODE: InvalidRequestContent; detail message timed out after 29s waiting for resolution of desired version from '4.21.15' cluster version; provider Microsoft.RedHatOpenShift
19:53:51 UTC · failed at e2e · test Customer should be able to successfully upgrade control plane minor version from 4.20 minor to 4.21 minor · suite integration/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:160]: Unexpected error:
    <*fmt.wrapError | 0xc000c94560>: 
    failed waiting for hcpCluster="cp-ystream-upgrade-4-21-gjf5t9" in resourcegroup="rg-cp-ystream-upgrade-4-21-fg6lv9" to finish updating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/5f247cdd-5812-4761-b99d-f988d2cb1453
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/5f247cdd-5812-4761-b99d-f988d2cb1453",
      "name": "5f247cdd-5812-4761-b99d-f988d2cb1453",
      "status": "Failed",
      "startTime": "2026-05-13T21:07:58.388882897Z",
      "endTime": "2026-05-13T21:09:22.905598885Z",
      "error": {
        "code": "InvalidRequestContent",
        "message": "timed out after 29s waiting for resolution of desired version from '4.21.15' cluster version"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:160]: Unexpected error:
    <*fmt.wrapError | 0xc000c94560>: 
    failed waiting for hcpCluster="cp-ystream-upgrade-4-21-gjf5t9" in resourcegroup="rg-cp-ystream-upgrade-4-21-fg6lv9" to finish updating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/5f247cdd-5812-4761-b99d-f988d2cb1453
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/5f247cdd-5812-4761-b99d-f988d2cb1453",
      "name": "5f247cdd-5812-4761-b99d-f988d2cb1453",
      "status": "Failed",
      "startTime": "2026-05-13T21:07:58.388882897Z",
      "endTime": "2026-05-13T21:09:22.905598885Z",
      "error": {
        "code": "InvalidRequestContent",
        "message": "timed out after 29s waiting for resolution of desired version from '4.21.15' cluster version"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
timeout during CreateHCPClusterAndWait; context deadline exceeded
19:53:51 UTC · failed at e2e · test Customer should upgrade and update a nodepool from 4.20.z to 4.21.zLatest · suite integration/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:130]: Unexpected error:
    <*fmt.wrapErrors | 0xc00136ec60>: 
    failed to create HCP cluster np-version-upgrade-cluster-98czdp, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster np-version-upgrade-cluster-98czdp in resource group rg-np-version-upgrade-98czdp-6t5bts, error: failed waiting for cluster="np-version-upgrade-cluster-98czdp" in resourcegroup="rg-np-version-upgrade-98czdp-6t5bts" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster np-version-upgrade-cluster-98czdp in resource group rg-np-version-upgrade-98czdp-6t5bts, error: context deadline exceeded
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:130]: Unexpected error:
    <*fmt.wrapErrors | 0xc00136ec60>: 
    failed to create HCP cluster np-version-upgrade-cluster-98czdp, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster np-version-upgrade-cluster-98czdp in resource group rg-np-version-upgrade-98czdp-6t5bts, error: failed waiting for cluster="np-version-upgrade-cluster-98czdp" in resourcegroup="rg-np-version-upgrade-98czdp-6t5bts" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster np-version-upgrade-cluster-98czdp in resource group rg-np-version-upgrade-98czdp-6t5bts, error: context deadline exceeded
    ...
occurred
branch-ci-Azure-ARO-HCP-main-e2e-integration-e2e-parallel
post-goodmerged PR
No additional run metadata captured
 e2e failed n/a 2
e2e (2)
ERROR CODE: InvalidRequestContent; detail message timed out after 29s waiting for resolution of desired version from '4.21.15' cluster version; provider Microsoft.RedHatOpenShift
15:16:27 UTC · failed at e2e · test Customer should be able to successfully upgrade control plane minor version from 4.20 minor to 4.21 minor · suite integration/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:160]: Unexpected error:
    <*fmt.wrapError | 0xc000dac400>: 
    failed waiting for hcpCluster="cp-ystream-upgrade-4-21-mtqjkn" in resourcegroup="rg-cp-ystream-upgrade-4-21-n4zbgv" to finish updating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/182072e0-9b74-4cef-b677-89a453c15273
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/182072e0-9b74-4cef-b677-89a453c15273",
      "name": "182072e0-9b74-4cef-b677-89a453c15273",
      "status": "Failed",
      "startTime": "2026-05-13T16:20:21.870546037Z",
      "endTime": "2026-05-13T16:22:13.927370791Z",
      "error": {
        "code": "InvalidRequestContent",
        "message": "timed out after 29s waiting for resolution of desired version from '4.21.15' cluster version"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:160]: Unexpected error:
    <*fmt.wrapError | 0xc000dac400>: 
    failed waiting for hcpCluster="cp-ystream-upgrade-4-21-mtqjkn" in resourcegroup="rg-cp-ystream-upgrade-4-21-n4zbgv" to finish updating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/182072e0-9b74-4cef-b677-89a453c15273
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/182072e0-9b74-4cef-b677-89a453c15273",
      "name": "182072e0-9b74-4cef-b677-89a453c15273",
      "status": "Failed",
      "startTime": "2026-05-13T16:20:21.870546037Z",
      "endTime": "2026-05-13T16:22:13.927370791Z",
      "error": {
        "code": "InvalidRequestContent",
        "message": "timed out after 29s waiting for resolution of desired version from '4.21.15' cluster version"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
timeout during CreateHCPClusterAndWait; context deadline exceeded
15:16:27 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 4.20 · suite integration/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:141]: Unexpected error:
    <*fmt.wrapErrors | 0xc0014b4ff0>: 
    failed to create HCP cluster cluster-zstream-4-20-x9qmdr, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster cluster-zstream-4-20-x9qmdr in resource group rg-zstream-upgrade-4-20-w7phnh, error: failed waiting for cluster="cluster-zstream-4-20-x9qmdr" in resourcegroup="rg-zstream-upgrade-4-20-w7phnh" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster cluster-zstream-4-20-x9qmdr in resource group rg-zstream-upgrade-4-20-w7phnh, error: context deadline exceeded
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:141]: Unexpected error:
    <*fmt.wrapErrors | 0xc0014b4ff0>: 
    failed to create HCP cluster cluster-zstream-4-20-x9qmdr, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster cluster-zstream-4-20-x9qmdr in resource group rg-zstream-upgrade-4-20-w7phnh, error: failed waiting for cluster="cluster-zstream-4-20-x9qmdr" in resourcegroup="rg-zstream-upgrade-4-20-w7phnh" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster cluster-zstream-4-20-x9qmdr in resource group rg-zstream-upgrade-4-20-w7phnh, error: context deadline exceeded
    ...
occurred
branch-ci-Azure-ARO-HCP-main-e2e-integration-e2e-parallel
post-goodmerged PR
No additional run metadata captured
 e2e failed n/a 1
e2e (1)
ERROR CODE: InvalidRequestContent; detail message timed out after 29s waiting for resolution of desired version from '4.21.15' cluster version; provider Microsoft.RedHatOpenShift
10:07:37 UTC · failed at e2e · test Customer should be able to successfully upgrade control plane minor version from 4.20 minor to 4.21 minor · suite integration/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:160]: Unexpected error:
    <*fmt.wrapError | 0xc000bfcae0>: 
    failed waiting for hcpCluster="cp-ystream-upgrade-4-21-mrjt2c" in resourcegroup="rg-cp-ystream-upgrade-4-21-fpbzfg" to finish updating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/49d5cbd3-678d-4fe3-9ed2-a848d52acb56
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/49d5cbd3-678d-4fe3-9ed2-a848d52acb56",
      "name": "49d5cbd3-678d-4fe3-9ed2-a848d52acb56",
      "status": "Failed",
      "startTime": "2026-05-13T11:04:30.183820377Z",
      "endTime": "2026-05-13T11:06:12.539306454Z",
      "error": {
        "code": "InvalidRequestContent",
        "message": "timed out after 29s waiting for resolution of desired version from '4.21.15' cluster version"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:160]: Unexpected error:
    <*fmt.wrapError | 0xc000bfcae0>: 
    failed waiting for hcpCluster="cp-ystream-upgrade-4-21-mrjt2c" in resourcegroup="rg-cp-ystream-upgrade-4-21-fpbzfg" to finish updating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/49d5cbd3-678d-4fe3-9ed2-a848d52acb56
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/49d5cbd3-678d-4fe3-9ed2-a848d52acb56",
      "name": "49d5cbd3-678d-4fe3-9ed2-a848d52acb56",
      "status": "Failed",
      "startTime": "2026-05-13T11:04:30.183820377Z",
      "endTime": "2026-05-13T11:06:12.539306454Z",
      "error": {
        "code": "InvalidRequestContent",
        "message": "timed out after 29s waiting for resolution of desired version from '4.21.15' cluster version"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
branch-ci-Azure-ARO-HCP-main-e2e-integration-e2e-parallel
post-goodmerged PR
No additional run metadata captured
 e2e failed n/a 1
e2e (1)
ERROR CODE: InvalidRequestContent; detail message timed out after 29s waiting for resolution of desired version from '4.21.15' cluster version; provider Microsoft.RedHatOpenShift
07:31:30 UTC · failed at e2e · test Customer should be able to successfully upgrade control plane minor version from 4.20 minor to 4.21 minor · suite integration/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:160]: Unexpected error:
    <*fmt.wrapError | 0xc000946d00>: 
    failed waiting for hcpCluster="cp-ystream-upgrade-4-21-7nmwp5" in resourcegroup="rg-cp-ystream-upgrade-4-21-9gqgnv" to finish updating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/dd81deca-9e2f-4086-a4f5-2a70306c3bba
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/dd81deca-9e2f-4086-a4f5-2a70306c3bba",
      "name": "dd81deca-9e2f-4086-a4f5-2a70306c3bba",
      "status": "Failed",
      "startTime": "2026-05-13T08:40:18.500125613Z",
      "endTime": "2026-05-13T08:42:21.215747142Z",
      "error": {
        "code": "InvalidRequestContent",
        "message": "timed out after 29s waiting for resolution of desired version from '4.21.15' cluster version"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:160]: Unexpected error:
    <*fmt.wrapError | 0xc000946d00>: 
    failed waiting for hcpCluster="cp-ystream-upgrade-4-21-7nmwp5" in resourcegroup="rg-cp-ystream-upgrade-4-21-9gqgnv" to finish updating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/dd81deca-9e2f-4086-a4f5-2a70306c3bba
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/dd81deca-9e2f-4086-a4f5-2a70306c3bba",
      "name": "dd81deca-9e2f-4086-a4f5-2a70306c3bba",
      "status": "Failed",
      "startTime": "2026-05-13T08:40:18.500125613Z",
      "endTime": "2026-05-13T08:42:21.215747142Z",
      "error": {
        "code": "InvalidRequestContent",
        "message": "timed out after 29s waiting for resolution of desired version from '4.21.15' cluster version"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
branch-ci-Azure-ARO-HCP-main-e2e-integration-e2e-parallel
post-goodmerged PR
No additional run metadata captured
 e2e failed n/a 1
e2e (1)
ERROR CODE: InvalidRequestContent; detail message timed out after 29s waiting for resolution of desired version from '4.21.15' cluster version; provider Microsoft.RedHatOpenShift
04:03:36 UTC · failed at e2e · test Customer should be able to successfully upgrade control plane minor version from 4.20 minor to 4.21 minor · suite integration/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:160]: Unexpected error:
    <*fmt.wrapError | 0xc000133320>: 
    failed waiting for hcpCluster="cp-ystream-upgrade-4-21-7cdhcm" in resourcegroup="rg-cp-ystream-upgrade-4-21-q42bt7" to finish updating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/02423575-465c-41ec-aef3-b05e26e53f38
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/02423575-465c-41ec-aef3-b05e26e53f38",
      "name": "02423575-465c-41ec-aef3-b05e26e53f38",
      "status": "Failed",
      "startTime": "2026-05-13T06:04:25.927599638Z",
      "endTime": "2026-05-13T06:05:58.537133791Z",
      "error": {
        "code": "InvalidRequestContent",
        "message": "timed out after 29s waiting for resolution of desired version from '4.21.15' cluster version"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:160]: Unexpected error:
    <*fmt.wrapError | 0xc000133320>: 
    failed waiting for hcpCluster="cp-ystream-upgrade-4-21-7cdhcm" in resourcegroup="rg-cp-ystream-upgrade-4-21-q42bt7" to finish updating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/02423575-465c-41ec-aef3-b05e26e53f38
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/02423575-465c-41ec-aef3-b05e26e53f38",
      "name": "02423575-465c-41ec-aef3-b05e26e53f38",
      "status": "Failed",
      "startTime": "2026-05-13T06:04:25.927599638Z",
      "endTime": "2026-05-13T06:05:58.537133791Z",
      "error": {
        "code": "InvalidRequestContent",
        "message": "timed out after 29s waiting for resolution of desired version from '4.21.15' cluster version"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred