CIHealth Run Log

Environment: STG

Time (UTC)	Runtime	Job	Region	Failed at	Result	PR	Failed tests	Details
23:00:44	1h 40m 44s	periodic-ci-Azure-ARO-HCP-main-periodic-stage-e2e-parallel-ocp-nightly post-goodmerged PR No additional run metadata captured	uksouth	e2e	failed	n/a	25	e2e (25) ERROR CODE: DeadlineExceeded; detail message cluster creation did not complete before the deadline; [cosmosCluster] .api.url is empty; [servingCABundle] ServingCABundle not yet populated; [clusterServiceClusterStatus] cluster service has not been successfully created; [hypershiftHostedCluster] hosted cluster state not cached yet; provider Microsoft.RedHatOpenShift 23:00:44 UTC · failed at e2e · test Customer should upgrade a nodepool skipping one minor version (+2) from 4.20.z to 4.22.zLatest · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:443]: failed to create HCP cluster np-skip-minor-27sjww at version 4.22.0-0.nightly-multi-2026-07-08-192436 Unexpected error: <fmt.wrapError \| 0xc000d81140>: failed to create HCP cluster np-skip-minor-27sjww: failed waiting for cluster="np-skip-minor-27sjww" in resourcegroup="rg-np-skip-minor-27sjww-kgw6mbppvdmz" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/74064ef6-783f-4e23-b96a-a2576b7c7d5f -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/74064ef6-783f-4e23-b96a-a2576b7c7d5f", "name": "74064ef6-783f-4e23-b96a-a2576b7c7d5f", "status": "Failed", "startTime": "2026-10-02T23:50:41.341932888Z", "endTime": "2026-10-03T00:19:42.833586106Z", "error": { "code": "DeadlineExceeded", "message": "cluster creation did not complete before the deadline; [cosmosCluster] .api.url is empty; [servingCABundle] ServingCABundle not yet populated; [clusterServiceClusterStatus] cluster service has not been successfully created; [hypershiftHostedCluster] hosted cluster state not cached yet" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:443]: failed to create HCP cluster np-skip-minor-27sjww at version 4.22.0-0.nightly-multi-2026-07-08-192436 Unexpected error: <fmt.wrapError \| 0xc000d81140>: failed to create HCP cluster np-skip-minor-27sjww: failed waiting for cluster="np-skip-minor-27sjww" in resourcegroup="rg-np-skip-minor-27sjww-kgw6mbppvdmz" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/74064ef6-783f-4e23-b96a-a2576b7c7d5f -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/74064ef6-783f-4e23-b96a-a2576b7c7d5f", "name": "74064ef6-783f-4e23-b96a-a2576b7c7d5f", "status": "Failed", "startTime": "2026-10-02T23:50:41.341932888Z", "endTime": "2026-10-03T00:19:42.833586106Z", "error": { "code": "DeadlineExceeded", "message": "cluster creation did not complete before the deadline; [cosmosCluster] .api.url is empty; [servingCABundle] ServingCABundle not yet populated; [clusterServiceClusterStatus] cluster service has not been successfully created; [hypershiftHostedCluster] hosted cluster state not cached yet" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message cluster creation did not complete before the deadline; [cosmosCluster] .api.url is empty; [servingCABundle] ServingCABundle not yet populated; [clusterServiceClusterStatus] cluster service has not been successfully created; [hypershiftHostedCluster] hosted cluster state not cached yet; provider Microsoft.RedHatOpenShift 23:00:44 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP nightly channel for 4.22 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:123]: HCP cluster rg-nightly-4-22-8cm72gr5ql2s/cluster-nightly-4-22-65xxws should provision Unexpected error: <fmt.wrapError \| 0xc00164c880>: failed to create HCP cluster cluster-nightly-4-22-65xxws: failed waiting for cluster="cluster-nightly-4-22-65xxws" in resourcegroup="rg-nightly-4-22-8cm72gr5ql2s" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/562de0b5-c401-4c2c-b9be-9fd3eaaa9a92 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/562de0b5-c401-4c2c-b9be-9fd3eaaa9a92", "name": "562de0b5-c401-4c2c-b9be-9fd3eaaa9a92", "status": "Failed", "startTime": "2026-10-02T23:28:43.258428047Z", "endTime": "2026-10-02T23:57:43.738727793Z", "error": { "code": "DeadlineExceeded", "message": "cluster creation did not complete before the deadline; [cosmosCluster] .api.url is empty; [servingCABundle] ServingCABundle not yet populated; [clusterServiceClusterStatus] cluster service has not been successfully created; [hypershiftHostedCluster] hosted cluster state not cached yet" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:123]: HCP cluster rg-nightly-4-22-8cm72gr5ql2s/cluster-nightly-4-22-65xxws should provision Unexpected error: <fmt.wrapError \| 0xc00164c880>: failed to create HCP cluster cluster-nightly-4-22-65xxws: failed waiting for cluster="cluster-nightly-4-22-65xxws" in resourcegroup="rg-nightly-4-22-8cm72gr5ql2s" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/562de0b5-c401-4c2c-b9be-9fd3eaaa9a92 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/562de0b5-c401-4c2c-b9be-9fd3eaaa9a92", "name": "562de0b5-c401-4c2c-b9be-9fd3eaaa9a92", "status": "Failed", "startTime": "2026-10-02T23:28:43.258428047Z", "endTime": "2026-10-02T23:57:43.738727793Z", "error": { "code": "DeadlineExceeded", "message": "cluster creation did not complete before the deadline; [cosmosCluster] .api.url is empty; [servingCABundle] ServingCABundle not yet populated; [clusterServiceClusterStatus] cluster service has not been successfully created; [hypershiftHostedCluster] hosted cluster state not cached yet" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-update-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift 23:00:44 UTC · failed at e2e · test Customer should be able to create an HCP cluster and manage ImageDigestMirrors · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/idms_lifecycle.go:175]: failed to update cluster with second ImageDigestMirror set Unexpected error: <exported.ResponseError \| 0xc00106c720>: PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/idms-88jnzt6h65n4/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/idms-e2e-hcp-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.cluster.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.cluster.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/idms_lifecycle.go:175]: failed to update cluster with second ImageDigestMirror set Unexpected error: <exported.ResponseError \| 0xc00106c720>: PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/idms-88jnzt6h65n4/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/idms-e2e-hcp-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.cluster.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.cluster.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-update-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift 23:00:44 UTC · failed at e2e · test Customer should be able to successfully upgrade control plane minor version from 5.0 minor to 5.1 minor · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:167]: failed to trigger y-stream upgrade of cluster "cp-ystream-upgrade-5-1-766n78" to 5.1.0-0.nightly-multi-2026-09-30-181926 Unexpected error: <exported.ResponseError \| 0xc00149e390>: PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cp-ystream-upgrade-5-1-qb9p7vsl4pjf/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cp-ystream-upgrade-5-1-766n78 -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.cluster.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.cluster.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:167]: failed to trigger y-stream upgrade of cluster "cp-ystream-upgrade-5-1-766n78" to 5.1.0-0.nightly-multi-2026-09-30-181926 Unexpected error: <exported.ResponseError \| 0xc00149e390>: PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cp-ystream-upgrade-5-1-qb9p7vsl4pjf/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cp-ystream-upgrade-5-1-766n78 -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.cluster.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.cluster.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-update-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift 23:00:44 UTC · failed at e2e · test Customer should be able to create an HCP cluster with custom autoscaling and update the autoscaling configuration · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:173]: failed to update cluster "autoscaling-hcp-cluster" autoscaling configuration Unexpected error: <exported.ResponseError \| 0xc00182a270>: PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/autoscaling-cluster-7bxvbnqqpfz2/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/autoscaling-hcp-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.cluster.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.cluster.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:173]: failed to update cluster "autoscaling-hcp-cluster" autoscaling configuration Unexpected error: <exported.ResponseError \| 0xc00182a270>: PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/autoscaling-cluster-7bxvbnqqpfz2/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/autoscaling-hcp-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.cluster.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.cluster.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-update-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift 23:00:44 UTC · failed at e2e · test Customer should be able to successfully upgrade control plane minor version from 4.20 minor to 4.21 minor · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:167]: failed to trigger y-stream upgrade of cluster "cp-ystream-upgrade-4-21-p76rlg" to 4.21.0-0.nightly-multi-2026-10-02-021649 Unexpected error: <exported.ResponseError \| 0xc000ab88a0>: PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cp-ystream-upgrade-4-21-c4g7bpjwkcrc/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cp-ystream-upgrade-4-21-p76rlg -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.cluster.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.cluster.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:167]: failed to trigger y-stream upgrade of cluster "cp-ystream-upgrade-4-21-p76rlg" to 4.21.0-0.nightly-multi-2026-10-02-021649 Unexpected error: <exported.ResponseError \| 0xc000ab88a0>: PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cp-ystream-upgrade-4-21-c4g7bpjwkcrc/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cp-ystream-upgrade-4-21-p76rlg -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.cluster.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.cluster.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 23:00:44 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP nightly channel for 5.0 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-tqrvbk" for cluster "cluster-nightly-5-0-tqrvbk" Unexpected error: <fmt.wrapError \| 0xc000f91320>: failed to create NodePool np-tqrvbk: failed waiting for nodepool="np-tqrvbk" for cluster "cluster-nightly-5-0-tqrvbk" in resourcegroup="rg-nightly-5-0-r6nmqb54mssw" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/096e1e24-3a65-44de-9802-0a6339605f8f -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/096e1e24-3a65-44de-9802-0a6339605f8f", "name": "096e1e24-3a65-44de-9802-0a6339605f8f", "status": "Failed", "startTime": "2026-10-02T23:41:44.73235146Z", "endTime": "2026-10-03T00:00:53.838500072Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine s0b5t8z5y1y3a5r-np-tqrvbk-zdnfc-89fp6: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine s0b5t8z5y1y3a5r-np-tqrvbk-zdnfc-d94nh: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine s0b5t8z5y1y3a5r-np-tqrvbk-zdnfc-89fp6: Creating: virtualmachine creating or updating\nMachine s0b5t8z5y1y3a5r-np-tqrvbk-zdnfc-d94nh: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-tqrvbk" for cluster "cluster-nightly-5-0-tqrvbk" Unexpected error: <fmt.wrapError \| 0xc000f91320>: failed to create NodePool np-tqrvbk: failed waiting for nodepool="np-tqrvbk" for cluster "cluster-nightly-5-0-tqrvbk" in resourcegroup="rg-nightly-5-0-r6nmqb54mssw" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/096e1e24-3a65-44de-9802-0a6339605f8f -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/096e1e24-3a65-44de-9802-0a6339605f8f", "name": "096e1e24-3a65-44de-9802-0a6339605f8f", "status": "Failed", "startTime": "2026-10-02T23:41:44.73235146Z", "endTime": "2026-10-03T00:00:53.838500072Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine s0b5t8z5y1y3a5r-np-tqrvbk-zdnfc-89fp6: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine s0b5t8z5y1y3a5r-np-tqrvbk-zdnfc-d94nh: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine s0b5t8z5y1y3a5r-np-tqrvbk-zdnfc-89fp6: Creating: virtualmachine creating or updating\nMachine s0b5t8z5y1y3a5r-np-tqrvbk-zdnfc-d94nh: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message cluster creation did not complete before the deadline; [cosmosCluster] .api.url is empty; [servingCABundle] ServingCABundle not yet populated; [clusterServiceClusterStatus] cluster service has not been successfully created; [hypershiftHostedCluster] hosted cluster state not cached yet; provider Microsoft.RedHatOpenShift 23:00:44 UTC · failed at e2e · test Customer should be able to successfully upgrade control plane minor version from 4.22 minor to 5.0 minor · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:133]: failed to create HCP cluster "cp-ystream-upgrade-5-0-lg5p6f" at install version 4.22.0-0.nightly-multi-2026-07-08-192436 Unexpected error: <fmt.wrapError \| 0xc000a56dc0>: failed to create HCP cluster cp-ystream-upgrade-5-0-lg5p6f: failed waiting for cluster="cp-ystream-upgrade-5-0-lg5p6f" in resourcegroup="rg-cp-ystream-upgrade-5-0-b2ldx28lrgzd" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/9c7f2efd-8245-4407-9270-f3937d54358b -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/9c7f2efd-8245-4407-9270-f3937d54358b", "name": "9c7f2efd-8245-4407-9270-f3937d54358b", "status": "Failed", "startTime": "2026-10-02T23:32:39.04806912Z", "endTime": "2026-10-03T00:01:43.756113041Z", "error": { "code": "DeadlineExceeded", "message": "cluster creation did not complete before the deadline; [cosmosCluster] .api.url is empty; [servingCABundle] ServingCABundle not yet populated; [clusterServiceClusterStatus] cluster service has not been successfully created; [hypershiftHostedCluster] hosted cluster state not cached yet" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:133]: failed to create HCP cluster "cp-ystream-upgrade-5-0-lg5p6f" at install version 4.22.0-0.nightly-multi-2026-07-08-192436 Unexpected error: <fmt.wrapError \| 0xc000a56dc0>: failed to create HCP cluster cp-ystream-upgrade-5-0-lg5p6f: failed waiting for cluster="cp-ystream-upgrade-5-0-lg5p6f" in resourcegroup="rg-cp-ystream-upgrade-5-0-b2ldx28lrgzd" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/9c7f2efd-8245-4407-9270-f3937d54358b -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/9c7f2efd-8245-4407-9270-f3937d54358b", "name": "9c7f2efd-8245-4407-9270-f3937d54358b", "status": "Failed", "startTime": "2026-10-02T23:32:39.04806912Z", "endTime": "2026-10-03T00:01:43.756113041Z", "error": { "code": "DeadlineExceeded", "message": "cluster creation did not complete before the deadline; [cosmosCluster] .api.url is empty; [servingCABundle] ServingCABundle not yet populated; [clusterServiceClusterStatus] cluster service has not been successfully created; [hypershiftHostedCluster] hosted cluster state not cached yet" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.nodepool.max-update-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift 23:00:44 UTC · failed at e2e · test Customer should be able to update node pool labels and taints · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_labels_taints.go:186]: failed to update node pool np-1 with new taints and scale up Unexpected error: <fmt.wrapError \| 0xc00141a900>: failed to start nodepool "np-1" update in cluster "cluster-np-labels-taints" resourcegroup="rg-np-labels-taints-kmfrpjr862hw": PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-labels-taints-kmfrpjr862hw/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-np-labels-taints/nodePools/np-1 -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.nodepool.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.nodepool.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_labels_taints.go:186]: failed to update node pool np-1 with new taints and scale up Unexpected error: <fmt.wrapError \| 0xc00141a900>: failed to start nodepool "np-1" update in cluster "cluster-np-labels-taints" resourcegroup="rg-np-labels-taints-kmfrpjr862hw": PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-labels-taints-kmfrpjr862hw/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-np-labels-taints/nodePools/np-1 -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.nodepool.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.nodepool.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 23:00:44 UTC · failed at e2e · test Create HCPOpenShiftCluster with Private KeyVault should create a cluster with private keyvault using v20251223preview API · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:138]: failed to create node pool "np-1" for private keyvault cluster "private-kv-cluster" Unexpected error: <fmt.wrapError \| 0xc0010647a0>: failed to create NodePool np-1: failed waiting for nodepool="np-1" for cluster "private-kv-cluster" in resourcegroup="private-keyvault-b4pbcj49h2gk" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/56464cc9-6bea-4036-8793-830b73f9c445 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/56464cc9-6bea-4036-8793-830b73f9c445", "name": "56464cc9-6bea-4036-8793-830b73f9c445", "status": "Failed", "startTime": "2026-10-02T23:18:22.317106803Z", "endTime": "2026-10-02T23:37:23.768654646Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine z5o3s0b7q4a6b4y-np-1-zmwpp-882wt: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine z5o3s0b7q4a6b4y-np-1-zmwpp-t6wms: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine z5o3s0b7q4a6b4y-np-1-zmwpp-882wt: Creating: virtualmachine creating or updating\nMachine z5o3s0b7q4a6b4y-np-1-zmwpp-t6wms: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:138]: failed to create node pool "np-1" for private keyvault cluster "private-kv-cluster" Unexpected error: <fmt.wrapError \| 0xc0010647a0>: failed to create NodePool np-1: failed waiting for nodepool="np-1" for cluster "private-kv-cluster" in resourcegroup="private-keyvault-b4pbcj49h2gk" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/56464cc9-6bea-4036-8793-830b73f9c445 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/56464cc9-6bea-4036-8793-830b73f9c445", "name": "56464cc9-6bea-4036-8793-830b73f9c445", "status": "Failed", "startTime": "2026-10-02T23:18:22.317106803Z", "endTime": "2026-10-02T23:37:23.768654646Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine z5o3s0b7q4a6b4y-np-1-zmwpp-882wt: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine z5o3s0b7q4a6b4y-np-1-zmwpp-t6wms: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine z5o3s0b7q4a6b4y-np-1-zmwpp-882wt: Creating: virtualmachine creating or updating\nMachine z5o3s0b7q4a6b4y-np-1-zmwpp-t6wms: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message cluster creation did not complete before the deadline; [cosmosCluster] .api.url is empty; [servingCABundle] ServingCABundle not yet populated; [clusterServiceClusterStatus] cluster service has not been successfully created; [hypershiftHostedCluster] hosted cluster state not cached yet; provider Microsoft.RedHatOpenShift 23:00:44 UTC · failed at e2e · test Customer should be able to successfully upgrade control plane minor version from 4.22 minor to 4.23 minor · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:133]: failed to create HCP cluster "cp-ystream-upgrade-4-23-85ljr8" at install version 4.22.0-0.nightly-multi-2026-07-08-192436 Unexpected error: <fmt.wrapError \| 0xc001476720>: failed to create HCP cluster cp-ystream-upgrade-4-23-85ljr8: failed waiting for cluster="cp-ystream-upgrade-4-23-85ljr8" in resourcegroup="rg-cp-ystream-upgrade-4-23-xxm22554wqqw" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4209a33e-d8fd-4e71-a9e2-6938fd555838 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4209a33e-d8fd-4e71-a9e2-6938fd555838", "name": "4209a33e-d8fd-4e71-a9e2-6938fd555838", "status": "Failed", "startTime": "2026-10-02T23:32:20.614033798Z", "endTime": "2026-10-03T00:01:23.755103284Z", "error": { "code": "DeadlineExceeded", "message": "cluster creation did not complete before the deadline; [cosmosCluster] .api.url is empty; [servingCABundle] ServingCABundle not yet populated; [clusterServiceClusterStatus] cluster service has not been successfully created; [hypershiftHostedCluster] hosted cluster state not cached yet" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:133]: failed to create HCP cluster "cp-ystream-upgrade-4-23-85ljr8" at install version 4.22.0-0.nightly-multi-2026-07-08-192436 Unexpected error: <fmt.wrapError \| 0xc001476720>: failed to create HCP cluster cp-ystream-upgrade-4-23-85ljr8: failed waiting for cluster="cp-ystream-upgrade-4-23-85ljr8" in resourcegroup="rg-cp-ystream-upgrade-4-23-xxm22554wqqw" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4209a33e-d8fd-4e71-a9e2-6938fd555838 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4209a33e-d8fd-4e71-a9e2-6938fd555838", "name": "4209a33e-d8fd-4e71-a9e2-6938fd555838", "status": "Failed", "startTime": "2026-10-02T23:32:20.614033798Z", "endTime": "2026-10-03T00:01:23.755103284Z", "error": { "code": "DeadlineExceeded", "message": "cluster creation did not complete before the deadline; [cosmosCluster] .api.url is empty; [servingCABundle] ServingCABundle not yet populated; [clusterServiceClusterStatus] cluster service has not been successfully created; [hypershiftHostedCluster] hosted cluster state not cached yet" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 23:00:44 UTC · failed at e2e · test Customer should be able to create a no-CNI private cluster with a private key vault, a nodepool and install cilium CNI successfully · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:193]: failed to verify nodes are Ready with Cilium CNI for cluster "cilium-cluster" Unexpected error: <errors.joinError \| 0xc00100c018>: VerifyNodesReady failed: no nodes found in the cluster VerifyCiliumOperational failed: not all pods in kube-system namespace are running: no cilium pods found in kube-system namespace failed to create NodePool cilium-np: failed waiting for nodepool="cilium-np" for cluster "cilium-cluster" in resourcegroup="complex-cilium-kv-dbjg2z24dvcx" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/2f88e2f7-11dc-4604-9894-6c4ed35e59a2 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/2f88e2f7-11dc-4604-9894-6c4ed35e59a2", "name": "2f88e2f7-11dc-4604-9894-6c4ed35e59a2", "status": "Failed", "startTime": "2026-10-02T23:17:46.069336916Z", "endTime": "2026-10-02T23:36:53.678900847Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine cilium-cluster-cilium-np-p829p-2b2s6: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine cilium-cluster-cilium-np-p829p-t26hm: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine cilium-cluster-cilium-np-p829p-2b2s6: Creating: virtualmachine creating or updating\nMachine cilium-cluster-cilium-np-p829p-t26hm: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:193]: failed to verify nodes are Ready with Cilium CNI for cluster "cilium-cluster" Unexpected error: <errors.joinError \| 0xc00100c018>: VerifyNodesReady failed: no nodes found in the cluster VerifyCiliumOperational failed: not all pods in kube-system namespace are running: no cilium pods found in kube-system namespace failed to create NodePool cilium-np: failed waiting for nodepool="cilium-np" for cluster "cilium-cluster" in resourcegroup="complex-cilium-kv-dbjg2z24dvcx" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/2f88e2f7-11dc-4604-9894-6c4ed35e59a2 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/2f88e2f7-11dc-4604-9894-6c4ed35e59a2", "name": "2f88e2f7-11dc-4604-9894-6c4ed35e59a2", "status": "Failed", "startTime": "2026-10-02T23:17:46.069336916Z", "endTime": "2026-10-02T23:36:53.678900847Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine cilium-cluster-cilium-np-p829p-2b2s6: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine cilium-cluster-cilium-np-p829p-t26hm: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine cilium-cluster-cilium-np-p829p-2b2s6: Creating: virtualmachine creating or updating\nMachine cilium-cluster-cilium-np-p829p-t26hm: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 23:00:44 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP nightly channel for 4.21 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-fpzhbk" for cluster "cluster-nightly-4-21-fpzhbk" Unexpected error: <fmt.wrapError \| 0xc000c80140>: failed to create NodePool np-fpzhbk: failed waiting for nodepool="np-fpzhbk" for cluster "cluster-nightly-4-21-fpzhbk" in resourcegroup="rg-nightly-4-21-lf2wqrcgx8mk" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/f07e721e-1ddd-4cdc-848f-f939d97b8b6a -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/f07e721e-1ddd-4cdc-848f-f939d97b8b6a", "name": "f07e721e-1ddd-4cdc-848f-f939d97b8b6a", "status": "Failed", "startTime": "2026-10-02T23:36:50.854597667Z", "endTime": "2026-10-02T23:55:53.768834785Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine v2x8c0w5w4x9u4j-np-fpzhbk-pj8tx-g4dx8: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine v2x8c0w5w4x9u4j-np-fpzhbk-pj8tx-z5xht: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine v2x8c0w5w4x9u4j-np-fpzhbk-pj8tx-g4dx8: Creating: virtualmachine creating or updating\nMachine v2x8c0w5w4x9u4j-np-fpzhbk-pj8tx-z5xht: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-fpzhbk" for cluster "cluster-nightly-4-21-fpzhbk" Unexpected error: <fmt.wrapError \| 0xc000c80140>: failed to create NodePool np-fpzhbk: failed waiting for nodepool="np-fpzhbk" for cluster "cluster-nightly-4-21-fpzhbk" in resourcegroup="rg-nightly-4-21-lf2wqrcgx8mk" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/f07e721e-1ddd-4cdc-848f-f939d97b8b6a -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/f07e721e-1ddd-4cdc-848f-f939d97b8b6a", "name": "f07e721e-1ddd-4cdc-848f-f939d97b8b6a", "status": "Failed", "startTime": "2026-10-02T23:36:50.854597667Z", "endTime": "2026-10-02T23:55:53.768834785Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine v2x8c0w5w4x9u4j-np-fpzhbk-pj8tx-g4dx8: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine v2x8c0w5w4x9u4j-np-fpzhbk-pj8tx-z5xht: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine v2x8c0w5w4x9u4j-np-fpzhbk-pj8tx-g4dx8: Creating: virtualmachine creating or updating\nMachine v2x8c0w5w4x9u4j-np-fpzhbk-pj8tx-z5xht: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-update-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift 23:00:44 UTC · failed at e2e · test Customer should be able to successfully upgrade control plane minor version from 4.21 minor to 4.22 minor · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:167]: failed to trigger y-stream upgrade of cluster "cp-ystream-upgrade-4-22-gfwqg6" to 4.22.0-0.nightly-multi-2026-07-08-192436 Unexpected error: <exported.ResponseError \| 0xc0018044b0>: PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cp-ystream-upgrade-4-22-p7jzbr5ffl22/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cp-ystream-upgrade-4-22-gfwqg6 -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.cluster.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.cluster.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:167]: failed to trigger y-stream upgrade of cluster "cp-ystream-upgrade-4-22-gfwqg6" to 4.22.0-0.nightly-multi-2026-07-08-192436 Unexpected error: <exported.ResponseError \| 0xc0018044b0>: PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cp-ystream-upgrade-4-22-p7jzbr5ffl22/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cp-ystream-upgrade-4-22-gfwqg6 -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.cluster.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.cluster.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InvalidRequestContent; detail message Invalid value: "<ocp-nightly>": must be at least 4.20.8; provider Microsoft.RedHatOpenShift 23:00:44 UTC · failed at e2e · test Customer should upgrade and update a nodepool from 4.20.z to 4.21.zLatest · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:165]: failed to create node pool npupgrade-4-20 with version 4.20.0-0.nightly-multi-2026-10-01-125400 Unexpected error: <fmt.wrapError \| 0xc001214a20>: failed to create NodePool npupgrade-4-20: failed starting nodepool creation "npupgrade-4-20" for cluster "np-version-upgrade-cluster-r44ftv" in resourcegroup="rg-np-version-upgrade-r44ftv-dq6wfvlkhtqj": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-version-upgrade-r44ftv-dq6wfvlkhtqj/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-version-upgrade-cluster-r44ftv/nodePools/npupgrade-4-20 -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"4.20.0-0.nightly-multi-2026-10-01-125400\": must be at least 4.20.8", "target": "properties.version.id" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:165]: failed to create node pool npupgrade-4-20 with version 4.20.0-0.nightly-multi-2026-10-01-125400 Unexpected error: <fmt.wrapError \| 0xc001214a20>: failed to create NodePool npupgrade-4-20: failed starting nodepool creation "npupgrade-4-20" for cluster "np-version-upgrade-cluster-r44ftv" in resourcegroup="rg-np-version-upgrade-r44ftv-dq6wfvlkhtqj": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-version-upgrade-r44ftv-dq6wfvlkhtqj/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-version-upgrade-cluster-r44ftv/nodePools/npupgrade-4-20 -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"4.20.0-0.nightly-multi-2026-10-01-125400\": must be at least 4.20.8", "target": "properties.version.id" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-update-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift 23:00:44 UTC · failed at e2e · test Update HCPOpenShiftCluster Positive creates a cluster and updates tags with a PATCH request · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:193]: failed to update HCP cluster tags via PATCH Unexpected error: <exported.ResponseError \| 0xc0005e2c30>: PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-tags-9rcfwwnss4kp/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-tags-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.cluster.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.cluster.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:193]: failed to update HCP cluster tags via PATCH Unexpected error: <exported.ResponseError \| 0xc0005e2c30>: PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-tags-9rcfwwnss4kp/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-tags-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.cluster.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.cluster.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InvalidRequestContent; detail message Invalid value: "4.21": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is "nightly"; provider Microsoft.RedHatOpenShift 23:00:44 UTC · failed at e2e · test Authorized CIDRs Connectivity should allow API access only from authorized VM IP · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:408]: failed to begin updating cluster "cidr-connectivity-test" authorized CIDRs Unexpected error: <exported.ResponseError \| 0xc002910ff0>: PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-rbgmrr9dbzzr/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cidr-connectivity-test -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"4.21\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"", "target": "properties.version.id" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:408]: failed to begin updating cluster "cidr-connectivity-test" authorized CIDRs Unexpected error: <exported.ResponseError \| 0xc002910ff0>: PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-rbgmrr9dbzzr/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cidr-connectivity-test -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"4.21\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"", "target": "properties.version.id" } } -------------------------------------------------------------------------------- ... occurred failed to create nodepool <nodepool>: failed waiting for nodepool="<nodepool>" for cluster "<cluster>" in resourcegroup="<resource-group>" to finish creating: GET <url> 23:00:44 UTC · failed at e2e · test Customer should be able to delete a nodepool · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_delete.go:128]: failed to create node pools Expected success, but got an error: <fmt.wrapError \| 0xc002b293e0>: failed to create NodePool np-delete-main: failed waiting for nodepool="np-delete-main" for cluster "np-delete-hcp-cluster" in resourcegroup="nodepool-delete-rx57wpbdxhdh" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/811670f5-5dbe-439c-9697-edae7243b031 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/811670f5-5dbe-439c-9697-edae7243b031", "name": "811670f5-5dbe-439c-9697-edae7243b031", "status": "Failed", "startTime": "2026-10-02T23:47:39.993818098Z", "endTime": "2026-10-03T00:06:42.823230365Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine e9p1n9f9j7n4p2s-np-delete-main-ntkpd-lgvpd: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine e9p1n9f9j7n4p2s-np-delete-main-ntkpd-thccv: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine e9p1n9f9j7n4p2s-np-delete-main-ntkpd-lgvpd: Creating: virtualmachine creating or updating\nMachine e9p1n9f9j7n4p2s-np-delete-main-ntkpd-thccv: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_delete.go:128]: failed to create node pools Expected success, but got an error: <fmt.wrapError \| 0xc002b293e0>: failed to create NodePool np-delete-main: failed waiting for nodepool="np-delete-main" for cluster "np-delete-hcp-cluster" in resourcegroup="nodepool-delete-rx57wpbdxhdh" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/811670f5-5dbe-439c-9697-edae7243b031 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/811670f5-5dbe-439c-9697-edae7243b031", "name": "811670f5-5dbe-439c-9697-edae7243b031", "status": "Failed", "startTime": "2026-10-02T23:47:39.993818098Z", "endTime": "2026-10-03T00:06:42.823230365Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine e9p1n9f9j7n4p2s-np-delete-main-ntkpd-lgvpd: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine e9p1n9f9j7n4p2s-np-delete-main-ntkpd-thccv: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine e9p1n9f9j7n4p2s-np-delete-main-ntkpd-lgvpd: Creating: virtualmachine creating or updating\nMachine e9p1n9f9j7n4p2s-np-delete-main-ntkpd-thccv: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... ERROR CODE: DeadlineExceeded; detail message cluster creation did not complete before the deadline; [cosmosCluster] .api.url is empty; [servingCABundle] ServingCABundle not yet populated; [clusterServiceClusterStatus] cluster service has not been successfully created; [hypershiftHostedCluster] hosted cluster state not cached yet; provider Microsoft.RedHatOpenShift 23:00:44 UTC · failed at e2e · test Customer should downgrade a nodepool to a lower minor version from 4.22.zLatest to 4.20.zLatest · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:722]: failed to create HCP cluster Unexpected error: <fmt.wrapError \| 0xc000aae3e0>: failed to create HCP cluster np-dg-minor-t84576: failed waiting for cluster="np-dg-minor-t84576" in resourcegroup="rg-np-dg-minor-t84576-xhl45nnpkcxs" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/86a92082-52bb-4ade-8c3c-f2f5320c6001 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/86a92082-52bb-4ade-8c3c-f2f5320c6001", "name": "86a92082-52bb-4ade-8c3c-f2f5320c6001", "status": "Failed", "startTime": "2026-10-02T23:50:46.033712456Z", "endTime": "2026-10-03T00:19:52.83457796Z", "error": { "code": "DeadlineExceeded", "message": "cluster creation did not complete before the deadline; [cosmosCluster] .api.url is empty; [servingCABundle] ServingCABundle not yet populated; [clusterServiceClusterStatus] cluster service has not been successfully created; [hypershiftHostedCluster] hosted cluster state not cached yet" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:722]: failed to create HCP cluster Unexpected error: <fmt.wrapError \| 0xc000aae3e0>: failed to create HCP cluster np-dg-minor-t84576: failed waiting for cluster="np-dg-minor-t84576" in resourcegroup="rg-np-dg-minor-t84576-xhl45nnpkcxs" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/86a92082-52bb-4ade-8c3c-f2f5320c6001 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/86a92082-52bb-4ade-8c3c-f2f5320c6001", "name": "86a92082-52bb-4ade-8c3c-f2f5320c6001", "status": "Failed", "startTime": "2026-10-02T23:50:46.033712456Z", "endTime": "2026-10-03T00:19:52.83457796Z", "error": { "code": "DeadlineExceeded", "message": "cluster creation did not complete before the deadline; [cosmosCluster] .api.url is empty; [servingCABundle] ServingCABundle not yet populated; [clusterServiceClusterStatus] cluster service has not been successfully created; [hypershiftHostedCluster] hosted cluster state not cached yet" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; provider Microsoft.Compute 23:00:44 UTC · failed at e2e · test Customer should create a cluster with private ingress using v20260901preview and verify the ingress is internal · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_ingress.go:123]: failed to create node pool "np-1" for private ingress cluster "private-ingress" Unexpected error: <fmt.wrapError \| 0xc00089caa0>: failed to create NodePool np-1: failed waiting for nodepool="np-1" for cluster "private-ingress" in resourcegroup="private-ingress-b7c2z62bpd6r" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0c057ce9-73bf-4556-855a-010dcaa301e1 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0c057ce9-73bf-4556-855a-010dcaa301e1", "name": "0c057ce9-73bf-4556-855a-010dcaa301e1", "status": "Failed", "startTime": "2026-10-02T23:15:33.73277988Z", "endTime": "2026-10-02T23:34:33.89328902Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 1, want 2; node pool condition AllNodesHealthy is False: 1 of 2 machines are not healthy\nMachine private-ingress-np-1-lcxp7-kcfwr: InspectionFailed: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/private-ingress-b7c2z62bpd6r--managed/providers/Microsoft.Compute/virtualMachines/private-ingress-np-1-lcxp7-kcfwr to exist\n; node pool condition AllMachinesReady is False: 1 of 2 machines are not ready\nMachine private-ingress-np-1-lcxp7-kcfwr: ReadyUnknown: NodeHealthy: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/private-ingress-b7c2z62bpd6r--managed/providers/Microsoft.Compute/virtualMachines/private-ingress-np-1-lcxp7-kcfwr to exist\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_ingress.go:123]: failed to create node pool "np-1" for private ingress cluster "private-ingress" Unexpected error: <fmt.wrapError \| 0xc00089caa0>: failed to create NodePool np-1: failed waiting for nodepool="np-1" for cluster "private-ingress" in resourcegroup="private-ingress-b7c2z62bpd6r" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0c057ce9-73bf-4556-855a-010dcaa301e1 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0c057ce9-73bf-4556-855a-010dcaa301e1", "name": "0c057ce9-73bf-4556-855a-010dcaa301e1", "status": "Failed", "startTime": "2026-10-02T23:15:33.73277988Z", "endTime": "2026-10-02T23:34:33.89328902Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 1, want 2; node pool condition AllNodesHealthy is False: 1 of 2 machines are not healthy\nMachine private-ingress-np-1-lcxp7-kcfwr: InspectionFailed: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/private-ingress-b7c2z62bpd6r--managed/providers/Microsoft.Compute/virtualMachines/private-ingress-np-1-lcxp7-kcfwr to exist\n; node pool condition AllMachinesReady is False: 1 of 2 machines are not ready\nMachine private-ingress-np-1-lcxp7-kcfwr: ReadyUnknown: NodeHealthy: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/private-ingress-b7c2z62bpd6r--managed/providers/Microsoft.Compute/virtualMachines/private-ingress-np-1-lcxp7-kcfwr to exist\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 23:00:44 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP nightly channel for 4.23 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-vw9ss6" for cluster "cluster-nightly-4-23-vw9ss6" Unexpected error: <fmt.wrapError \| 0xc000e0f2c0>: failed to create NodePool np-vw9ss6: failed waiting for nodepool="np-vw9ss6" for cluster "cluster-nightly-4-23-vw9ss6" in resourcegroup="rg-nightly-4-23-66zfk978pxch" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/04cd216e-5e0e-4e76-a1ab-6ea26fbb70a0 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/04cd216e-5e0e-4e76-a1ab-6ea26fbb70a0", "name": "04cd216e-5e0e-4e76-a1ab-6ea26fbb70a0", "status": "Failed", "startTime": "2026-10-02T23:42:21.716380971Z", "endTime": "2026-10-03T00:01:23.832580289Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine h7d3c7e5o0i6e7s-np-vw9ss6-lw4g2-qhsr8: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine h7d3c7e5o0i6e7s-np-vw9ss6-lw4g2-wcqs9: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine h7d3c7e5o0i6e7s-np-vw9ss6-lw4g2-qhsr8: Creating: virtualmachine creating or updating\nMachine h7d3c7e5o0i6e7s-np-vw9ss6-lw4g2-wcqs9: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-vw9ss6" for cluster "cluster-nightly-4-23-vw9ss6" Unexpected error: <fmt.wrapError \| 0xc000e0f2c0>: failed to create NodePool np-vw9ss6: failed waiting for nodepool="np-vw9ss6" for cluster "cluster-nightly-4-23-vw9ss6" in resourcegroup="rg-nightly-4-23-66zfk978pxch" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/04cd216e-5e0e-4e76-a1ab-6ea26fbb70a0 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/04cd216e-5e0e-4e76-a1ab-6ea26fbb70a0", "name": "04cd216e-5e0e-4e76-a1ab-6ea26fbb70a0", "status": "Failed", "startTime": "2026-10-02T23:42:21.716380971Z", "endTime": "2026-10-03T00:01:23.832580289Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine h7d3c7e5o0i6e7s-np-vw9ss6-lw4g2-qhsr8: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine h7d3c7e5o0i6e7s-np-vw9ss6-lw4g2-wcqs9: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine h7d3c7e5o0i6e7s-np-vw9ss6-lw4g2-qhsr8: Creating: virtualmachine creating or updating\nMachine h7d3c7e5o0i6e7s-np-vw9ss6-lw4g2-wcqs9: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 23:00:44 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP nightly channel for 5.1 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-rx699z" for cluster "cluster-nightly-5-1-rx699z" Unexpected error: <fmt.wrapError \| 0xc000eb8200>: failed to create NodePool np-rx699z: failed waiting for nodepool="np-rx699z" for cluster "cluster-nightly-5-1-rx699z" in resourcegroup="rg-nightly-5-1-x6gpqlf2wmlk" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/13ac35c1-1131-4bbe-9ed8-04f87748743e -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/13ac35c1-1131-4bbe-9ed8-04f87748743e", "name": "13ac35c1-1131-4bbe-9ed8-04f87748743e", "status": "Failed", "startTime": "2026-10-02T23:40:19.915190927Z", "endTime": "2026-10-02T23:59:23.95618655Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine t2t5i8t7n0a2i6q-np-rx699z-9wkrd-d28hs: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine t2t5i8t7n0a2i6q-np-rx699z-9wkrd-kswmm: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine t2t5i8t7n0a2i6q-np-rx699z-9wkrd-d28hs: Creating: virtualmachine creating or updating\nMachine t2t5i8t7n0a2i6q-np-rx699z-9wkrd-kswmm: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-rx699z" for cluster "cluster-nightly-5-1-rx699z" Unexpected error: <fmt.wrapError \| 0xc000eb8200>: failed to create NodePool np-rx699z: failed waiting for nodepool="np-rx699z" for cluster "cluster-nightly-5-1-rx699z" in resourcegroup="rg-nightly-5-1-x6gpqlf2wmlk" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/13ac35c1-1131-4bbe-9ed8-04f87748743e -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/13ac35c1-1131-4bbe-9ed8-04f87748743e", "name": "13ac35c1-1131-4bbe-9ed8-04f87748743e", "status": "Failed", "startTime": "2026-10-02T23:40:19.915190927Z", "endTime": "2026-10-02T23:59:23.95618655Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine t2t5i8t7n0a2i6q-np-rx699z-9wkrd-d28hs: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine t2t5i8t7n0a2i6q-np-rx699z-9wkrd-kswmm: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine t2t5i8t7n0a2i6q-np-rx699z-9wkrd-d28hs: Creating: virtualmachine creating or updating\nMachine t2t5i8t7n0a2i6q-np-rx699z-9wkrd-kswmm: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InvalidRequestContent; detail message Invalid value: "4.21": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is "nightly"; provider Microsoft.RedHatOpenShift 23:00:44 UTC · failed at e2e · test Customer should be able to create a cluster and node pools with aggregated advanced features · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:275]: failed to begin cluster "agg-cluster" authorized CIDR update Unexpected error: <exported.ResponseError \| 0xc001680330>: PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/feature-aggregation-kcc2h87qnl8v/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/agg-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"4.21\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"", "target": "properties.version.id" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:275]: failed to begin cluster "agg-cluster" authorized CIDR update Unexpected error: <exported.ResponseError \| 0xc001680330>: PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/feature-aggregation-kcc2h87qnl8v/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/agg-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"4.21\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"", "target": "properties.version.id" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InvalidRequestContent; detail message Invalid value: "<ocp-nightly>": must be at least 4.20.8; provider Microsoft.RedHatOpenShift 23:00:44 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP nightly channel for 4.20 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-f6fpxn" for cluster "cluster-nightly-4-20-f6fpxn" Unexpected error: <fmt.wrapError \| 0xc000be25e0>: failed to create NodePool np-f6fpxn: failed starting nodepool creation "np-f6fpxn" for cluster "cluster-nightly-4-20-f6fpxn" in resourcegroup="rg-nightly-4-20-9z2m2dsr8b8d": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-nightly-4-20-9z2m2dsr8b8d/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-nightly-4-20-f6fpxn/nodePools/np-f6fpxn -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"4.20.0-0.nightly-multi-2026-10-01-125400\": must be at least 4.20.8", "target": "properties.version.id" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-f6fpxn" for cluster "cluster-nightly-4-20-f6fpxn" Unexpected error: <fmt.wrapError \| 0xc000be25e0>: failed to create NodePool np-f6fpxn: failed starting nodepool creation "np-f6fpxn" for cluster "cluster-nightly-4-20-f6fpxn" in resourcegroup="rg-nightly-4-20-9z2m2dsr8b8d": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-nightly-4-20-9z2m2dsr8b8d/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-nightly-4-20-f6fpxn/nodePools/np-f6fpxn -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"4.20.0-0.nightly-multi-2026-10-01-125400\": must be at least 4.20.8", "target": "properties.version.id" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.nodepool.max-update-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift 23:00:44 UTC · failed at e2e · test Customer should be able to create a cluster with default autoscaling and a nodepool with autoscaling enabled up to replica limits · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:189]: failed to update AZ nodepool autoscale-az max replicas to 4 Unexpected error: <fmt.wrapError \| 0xc0010a36c0>: failed to start nodepool "autoscale-az" update in cluster "np-autoscale-cluster" resourcegroup="np-autoscaling-tsckb2km58j7": PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/np-autoscaling-tsckb2km58j7/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-autoscale-cluster/nodePools/autoscale-az -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.nodepool.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.nodepool.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:189]: failed to update AZ nodepool autoscale-az max replicas to 4 Unexpected error: <fmt.wrapError \| 0xc0010a36c0>: failed to start nodepool "autoscale-az" update in cluster "np-autoscale-cluster" resourcegroup="np-autoscaling-tsckb2km58j7": PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/np-autoscaling-tsckb2km58j7/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-autoscale-cluster/nodePools/autoscale-az -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"aro-hcp.experimental.nodepool.max-update-duration\": unrecognized experimental tag", "target": "tags[aro-hcp.experimental.nodepool.max-update-duration]" } } -------------------------------------------------------------------------------- ... occurred
20:18:59	2h 14m 12s	branch-ci-Azure-ARO-HCP-main-e2e-stage-e2e-parallel post-goodmerged PR No additional run metadata captured	uksouth	e2e	failed	n/a	12	e2e (12) ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift node pool not cached yet; provider Microsoft.RedHatOpenShift 20:18:59 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 5.1 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-k6xcnn" for cluster "cluster-candidate-5-1-k6xcnn" Unexpected error: <fmt.wrapError \| 0xc000a95460>: failed to create NodePool np-k6xcnn: failed waiting for nodepool="np-k6xcnn" for cluster "cluster-candidate-5-1-k6xcnn" in resourcegroup="rg-candidate-5-1-xcbcjc2xvxnz" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/f4a6f0d3-f2f8-4021-bdab-85f57fe8a1f8 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/f4a6f0d3-f2f8-4021-bdab-85f57fe8a1f8", "name": "f4a6f0d3-f2f8-4021-bdab-85f57fe8a1f8", "status": "Failed", "startTime": "2026-10-02T21:05:55.776603892Z", "endTime": "2026-10-02T21:24:58.213196373Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift node pool not cached yet" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-k6xcnn" for cluster "cluster-candidate-5-1-k6xcnn" Unexpected error: <fmt.wrapError \| 0xc000a95460>: failed to create NodePool np-k6xcnn: failed waiting for nodepool="np-k6xcnn" for cluster "cluster-candidate-5-1-k6xcnn" in resourcegroup="rg-candidate-5-1-xcbcjc2xvxnz" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/f4a6f0d3-f2f8-4021-bdab-85f57fe8a1f8 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/f4a6f0d3-f2f8-4021-bdab-85f57fe8a1f8", "name": "f4a6f0d3-f2f8-4021-bdab-85f57fe8a1f8", "status": "Failed", "startTime": "2026-10-02T21:05:55.776603892Z", "endTime": "2026-10-02T21:24:58.213196373Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift node pool not cached yet" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 20:18:59 UTC · failed at e2e · test Customer should be able to create a no-CNI private cluster with a private key vault, a nodepool and install cilium CNI successfully · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:193]: failed to verify nodes are Ready with Cilium CNI for cluster "cilium-cluster" Unexpected error: <errors.joinError \| 0xc001880060>: VerifyNodesReady failed: no nodes found in the cluster VerifyCiliumOperational failed: not all pods in kube-system namespace are running: failed to list pods in kube-system namespace: client rate limiter Wait returned an error: context deadline exceeded failed to create NodePool cilium-np: failed waiting for nodepool="cilium-np" for cluster "cilium-cluster" in resourcegroup="complex-cilium-kv-flb2g5s6xxjr" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/c3afe0f8-5077-4a5f-b1fe-943343d11f7a -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/c3afe0f8-5077-4a5f-b1fe-943343d11f7a", "name": "c3afe0f8-5077-4a5f-b1fe-943343d11f7a", "status": "Failed", "startTime": "2026-10-02T20:40:40.455432211Z", "endTime": "2026-10-02T20:59:49.144387256Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine cilium-cluster-cilium-np-4kgx8-km75d: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine cilium-cluster-cilium-np-4kgx8-wjql6: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine cilium-cluster-cilium-np-4kgx8-km75d: Creating: virtualmachine creating or updating\nMachine cilium-cluster-cilium-np-4kgx8-wjql6: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:193]: failed to verify nodes are Ready with Cilium CNI for cluster "cilium-cluster" Unexpected error: <errors.joinError \| 0xc001880060>: VerifyNodesReady failed: no nodes found in the cluster VerifyCiliumOperational failed: not all pods in kube-system namespace are running: failed to list pods in kube-system namespace: client rate limiter Wait returned an error: context deadline exceeded failed to create NodePool cilium-np: failed waiting for nodepool="cilium-np" for cluster "cilium-cluster" in resourcegroup="complex-cilium-kv-flb2g5s6xxjr" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/c3afe0f8-5077-4a5f-b1fe-943343d11f7a -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/c3afe0f8-5077-4a5f-b1fe-943343d11f7a", "name": "c3afe0f8-5077-4a5f-b1fe-943343d11f7a", "status": "Failed", "startTime": "2026-10-02T20:40:40.455432211Z", "endTime": "2026-10-02T20:59:49.144387256Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine cilium-cluster-cilium-np-4kgx8-km75d: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine cilium-cluster-cilium-np-4kgx8-wjql6: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine cilium-cluster-cilium-np-4kgx8-km75d: Creating: virtualmachine creating or updating\nMachine cilium-cluster-cilium-np-4kgx8-wjql6: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 20:18:59 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.22 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-rfrkgm" for cluster "cluster-candidate-4-22-rfrkgm" Unexpected error: <fmt.wrapError \| 0xc000a0efa0>: failed to create NodePool np-rfrkgm: failed waiting for nodepool="np-rfrkgm" for cluster "cluster-candidate-4-22-rfrkgm" in resourcegroup="rg-candidate-4-22-4dnctl8qsdmb" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/bd66a171-5708-426c-a085-dc7fbb337a5f -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/bd66a171-5708-426c-a085-dc7fbb337a5f", "name": "bd66a171-5708-426c-a085-dc7fbb337a5f", "status": "Failed", "startTime": "2026-10-02T21:02:30.361632741Z", "endTime": "2026-10-02T21:21:38.229702877Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine z0t1i4f5k3c2g3w-np-rfrkgm-drz4w-dgcdt: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine z0t1i4f5k3c2g3w-np-rfrkgm-drz4w-z5pzp: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine z0t1i4f5k3c2g3w-np-rfrkgm-drz4w-dgcdt: Creating: virtualmachine creating or updating\nMachine z0t1i4f5k3c2g3w-np-rfrkgm-drz4w-z5pzp: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-rfrkgm" for cluster "cluster-candidate-4-22-rfrkgm" Unexpected error: <fmt.wrapError \| 0xc000a0efa0>: failed to create NodePool np-rfrkgm: failed waiting for nodepool="np-rfrkgm" for cluster "cluster-candidate-4-22-rfrkgm" in resourcegroup="rg-candidate-4-22-4dnctl8qsdmb" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/bd66a171-5708-426c-a085-dc7fbb337a5f -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/bd66a171-5708-426c-a085-dc7fbb337a5f", "name": "bd66a171-5708-426c-a085-dc7fbb337a5f", "status": "Failed", "startTime": "2026-10-02T21:02:30.361632741Z", "endTime": "2026-10-02T21:21:38.229702877Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine z0t1i4f5k3c2g3w-np-rfrkgm-drz4w-dgcdt: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine z0t1i4f5k3c2g3w-np-rfrkgm-drz4w-z5pzp: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine z0t1i4f5k3c2g3w-np-rfrkgm-drz4w-dgcdt: Creating: virtualmachine creating or updating\nMachine z0t1i4f5k3c2g3w-np-rfrkgm-drz4w-z5pzp: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; provider Microsoft.Compute 20:18:59 UTC · failed at e2e · test Customer should be able to create an HCP cluster then delete it by deleting the customer resource group · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delete_cx_rg.go:141]: failed to create node pools in parallel for cluster "cx-rg-hcp-cluster" Unexpected error: <fmt.wrapError \| 0xc000fbb780>: failed to create NodePool np-1: failed waiting for nodepool="np-1" for cluster "cx-rg-hcp-cluster" in resourcegroup="cx-rg-cluster-9v5cc2pldl4b" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/64c53475-70e9-4ead-ac9d-b255c14e111e -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/64c53475-70e9-4ead-ac9d-b255c14e111e", "name": "64c53475-70e9-4ead-ac9d-b255c14e111e", "status": "Failed", "startTime": "2026-10-02T20:37:10.986126021Z", "endTime": "2026-10-02T20:56:19.371811745Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 1, want 2; node pool condition AllNodesHealthy is False: 1 of 2 machines are not healthy\nMachine v2z0r1j6g1i1v5p-np-1-pkgzx-qrzhh: InspectionFailed: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/cx-rg-cluster-9v5cc2pldl4b--managed/providers/Microsoft.Compute/virtualMachines/v2z0r1j6g1i1v5p-np-1-pkgzx-qrzhh to exist\n; node pool condition AllMachinesReady is False: 1 of 2 machines are not ready\nMachine v2z0r1j6g1i1v5p-np-1-pkgzx-qrzhh: ReadyUnknown: NodeHealthy: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/cx-rg-cluster-9v5cc2pldl4b--managed/providers/Microsoft.Compute/virtualMachines/v2z0r1j6g1i1v5p-np-1-pkgzx-qrzhh to exist\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delete_cx_rg.go:141]: failed to create node pools in parallel for cluster "cx-rg-hcp-cluster" Unexpected error: <fmt.wrapError \| 0xc000fbb780>: failed to create NodePool np-1: failed waiting for nodepool="np-1" for cluster "cx-rg-hcp-cluster" in resourcegroup="cx-rg-cluster-9v5cc2pldl4b" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/64c53475-70e9-4ead-ac9d-b255c14e111e -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/64c53475-70e9-4ead-ac9d-b255c14e111e", "name": "64c53475-70e9-4ead-ac9d-b255c14e111e", "status": "Failed", "startTime": "2026-10-02T20:37:10.986126021Z", "endTime": "2026-10-02T20:56:19.371811745Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 1, want 2; node pool condition AllNodesHealthy is False: 1 of 2 machines are not healthy\nMachine v2z0r1j6g1i1v5p-np-1-pkgzx-qrzhh: InspectionFailed: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/cx-rg-cluster-9v5cc2pldl4b--managed/providers/Microsoft.Compute/virtualMachines/v2z0r1j6g1i1v5p-np-1-pkgzx-qrzhh to exist\n; node pool condition AllMachinesReady is False: 1 of 2 machines are not ready\nMachine v2z0r1j6g1i1v5p-np-1-pkgzx-qrzhh: ReadyUnknown: NodeHealthy: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/cx-rg-cluster-9v5cc2pldl4b--managed/providers/Microsoft.Compute/virtualMachines/v2z0r1j6g1i1v5p-np-1-pkgzx-qrzhh to exist\n" } } -------------------------------------------------------------------------------- ... occurred failed to create nodepool <nodepool>: failed waiting for nodepool="<nodepool>" for cluster "<cluster>" in resourcegroup="<resource-group>" to finish creating: GET <url> 20:18:59 UTC · failed at e2e · test Customer should be able to delete a nodepool · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_delete.go:128]: failed to create node pools Expected success, but got an error: <fmt.wrapError \| 0xc000c2b720>: failed to create NodePool np-deletable: failed waiting for nodepool="np-deletable" for cluster "np-delete-hcp-cluster" in resourcegroup="nodepool-delete-fgzfw6brhrkb" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/6270f4b8-a6c4-4356-a32a-feb00b0c9403 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/6270f4b8-a6c4-4356-a32a-feb00b0c9403", "name": "6270f4b8-a6c4-4356-a32a-feb00b0c9403", "status": "Failed", "startTime": "2026-10-02T21:34:15.426377086Z", "endTime": "2026-10-02T21:53:18.348312497Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine p8k4o5c9o8v2d0a-np-deletable-9k5ld-8sclt: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine p8k4o5c9o8v2d0a-np-deletable-9k5ld-ksnwd: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine p8k4o5c9o8v2d0a-np-deletable-9k5ld-8sclt: Creating: virtualmachine creating or updating\nMachine p8k4o5c9o8v2d0a-np-deletable-9k5ld-ksnwd: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_delete.go:128]: failed to create node pools Expected success, but got an error: <fmt.wrapError \| 0xc000c2b720>: failed to create NodePool np-deletable: failed waiting for nodepool="np-deletable" for cluster "np-delete-hcp-cluster" in resourcegroup="nodepool-delete-fgzfw6brhrkb" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/6270f4b8-a6c4-4356-a32a-feb00b0c9403 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/6270f4b8-a6c4-4356-a32a-feb00b0c9403", "name": "6270f4b8-a6c4-4356-a32a-feb00b0c9403", "status": "Failed", "startTime": "2026-10-02T21:34:15.426377086Z", "endTime": "2026-10-02T21:53:18.348312497Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine p8k4o5c9o8v2d0a-np-deletable-9k5ld-8sclt: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine p8k4o5c9o8v2d0a-np-deletable-9k5ld-ksnwd: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine p8k4o5c9o8v2d0a-np-deletable-9k5ld-8sclt: Creating: virtualmachine creating or updating\nMachine p8k4o5c9o8v2d0a-np-deletable-9k5ld-ksnwd: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 20:18:59 UTC · failed at e2e · test Customer should create a cluster with private KAS and verify API server is only reachable from VNet while ingress remains public · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:159]: failed to create node pool "np-1" for private KAS cluster "private-kas" Unexpected error: <fmt.wrapError \| 0xc000f04d60>: failed to create NodePool np-1: failed waiting for nodepool="np-1" for cluster "private-kas" in resourcegroup="private-kas-j6tfrrstctwj" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/2237f1ea-36ac-4925-ab4c-3bf34b150641 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/2237f1ea-36ac-4925-ab4c-3bf34b150641", "name": "2237f1ea-36ac-4925-ab4c-3bf34b150641", "status": "Failed", "startTime": "2026-10-02T20:40:29.897644584Z", "endTime": "2026-10-02T20:59:29.402047255Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 1; node pool condition AllNodesHealthy is False: 1 of 1 machines are not healthy\nMachine private-kas-np-1-krs6w-gdbz8: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 1 of 1 machines are not ready\nMachine private-kas-np-1-krs6w-gdbz8: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:159]: failed to create node pool "np-1" for private KAS cluster "private-kas" Unexpected error: <fmt.wrapError \| 0xc000f04d60>: failed to create NodePool np-1: failed waiting for nodepool="np-1" for cluster "private-kas" in resourcegroup="private-kas-j6tfrrstctwj" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/2237f1ea-36ac-4925-ab4c-3bf34b150641 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/2237f1ea-36ac-4925-ab4c-3bf34b150641", "name": "2237f1ea-36ac-4925-ab4c-3bf34b150641", "status": "Failed", "startTime": "2026-10-02T20:40:29.897644584Z", "endTime": "2026-10-02T20:59:29.402047255Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 1; node pool condition AllNodesHealthy is False: 1 of 1 machines are not healthy\nMachine private-kas-np-1-krs6w-gdbz8: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 1 of 1 machines are not ready\nMachine private-kas-np-1-krs6w-gdbz8: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 20:18:59 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.20 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-qpmglj" for cluster "cluster-candidate-4-20-qpmglj" Unexpected error: <fmt.wrapError \| 0xc00128af20>: failed to create NodePool np-qpmglj: failed waiting for nodepool="np-qpmglj" for cluster "cluster-candidate-4-20-qpmglj" in resourcegroup="rg-candidate-4-20-lrjrfzqq46lq" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/8f59c352-b15a-44fe-8eb1-052155b6abc3 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/8f59c352-b15a-44fe-8eb1-052155b6abc3", "name": "8f59c352-b15a-44fe-8eb1-052155b6abc3", "status": "Failed", "startTime": "2026-10-02T21:01:08.060624103Z", "endTime": "2026-10-02T21:20:08.251497411Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine a7s6c5x0k3i6u9a-np-qpmglj-tcqmc-pngl6: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine a7s6c5x0k3i6u9a-np-qpmglj-tcqmc-rh562: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine a7s6c5x0k3i6u9a-np-qpmglj-tcqmc-pngl6: Creating: virtualmachine creating or updating\nMachine a7s6c5x0k3i6u9a-np-qpmglj-tcqmc-rh562: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-qpmglj" for cluster "cluster-candidate-4-20-qpmglj" Unexpected error: <fmt.wrapError \| 0xc00128af20>: failed to create NodePool np-qpmglj: failed waiting for nodepool="np-qpmglj" for cluster "cluster-candidate-4-20-qpmglj" in resourcegroup="rg-candidate-4-20-lrjrfzqq46lq" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/8f59c352-b15a-44fe-8eb1-052155b6abc3 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/8f59c352-b15a-44fe-8eb1-052155b6abc3", "name": "8f59c352-b15a-44fe-8eb1-052155b6abc3", "status": "Failed", "startTime": "2026-10-02T21:01:08.060624103Z", "endTime": "2026-10-02T21:20:08.251497411Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine a7s6c5x0k3i6u9a-np-qpmglj-tcqmc-pngl6: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine a7s6c5x0k3i6u9a-np-qpmglj-tcqmc-rh562: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine a7s6c5x0k3i6u9a-np-qpmglj-tcqmc-pngl6: Creating: virtualmachine creating or updating\nMachine a7s6c5x0k3i6u9a-np-qpmglj-tcqmc-rh562: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want >= <count> (autoscaling min); node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 20:18:59 UTC · failed at e2e · test Customer should be able to create a cluster and node pools with aggregated advanced features · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:404]: failed to verify cilium is running and nodes are Ready for cluster "agg-cluster" Unexpected error: <errors.joinError \| 0xc001172048>: VerifyNodesReady failed: no nodes found in the cluster VerifyCiliumOperational failed: not all pods in kube-system namespace are running: failed to list pods in kube-system namespace: client rate limiter Wait returned an error: context deadline exceeded failed to create NodePool agg-np-a: failed waiting for nodepool="agg-np-a" for cluster "agg-cluster" in resourcegroup="feature-aggregation-5sg2h42pm8fv" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4ac0d101-3fd0-42ea-ae32-a6afba54d453 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4ac0d101-3fd0-42ea-ae32-a6afba54d453", "name": "4ac0d101-3fd0-42ea-ae32-a6afba54d453", "status": "Failed", "startTime": "2026-10-02T20:43:49.738242165Z", "endTime": "2026-10-02T21:02:49.398233298Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want \u003e= 2 (autoscaling min); node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine agg-cluster-agg-np-a-wg9bd-twwb6: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine agg-cluster-agg-np-a-wg9bd-z9q58: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine agg-cluster-agg-np-a-wg9bd-twwb6: Creating: virtualmachine creating or updating\nMachine agg-cluster-agg-np-a-wg9bd-z9q58: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:404]: failed to verify cilium is running and nodes are Ready for cluster "agg-cluster" Unexpected error: <errors.joinError \| 0xc001172048>: VerifyNodesReady failed: no nodes found in the cluster VerifyCiliumOperational failed: not all pods in kube-system namespace are running: failed to list pods in kube-system namespace: client rate limiter Wait returned an error: context deadline exceeded failed to create NodePool agg-np-a: failed waiting for nodepool="agg-np-a" for cluster "agg-cluster" in resourcegroup="feature-aggregation-5sg2h42pm8fv" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4ac0d101-3fd0-42ea-ae32-a6afba54d453 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4ac0d101-3fd0-42ea-ae32-a6afba54d453", "name": "4ac0d101-3fd0-42ea-ae32-a6afba54d453", "status": "Failed", "startTime": "2026-10-02T20:43:49.738242165Z", "endTime": "2026-10-02T21:02:49.398233298Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want \u003e= 2 (autoscaling min); node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine agg-cluster-agg-np-a-wg9bd-twwb6: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine agg-cluster-agg-np-a-wg9bd-z9q58: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine agg-cluster-agg-np-a-wg9bd-twwb6: Creating: virtualmachine creating or updating\nMachine agg-cluster-agg-np-a-wg9bd-z9q58: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 20:18:59 UTC · failed at e2e · test Create HCPOpenShiftCluster with Private KeyVault should create a cluster with private keyvault using v20251223preview API · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:138]: failed to create node pool "np-1" for private keyvault cluster "private-kv-cluster" Unexpected error: <fmt.wrapError \| 0xc000f6f0a0>: failed to create NodePool np-1: failed waiting for nodepool="np-1" for cluster "private-kv-cluster" in resourcegroup="private-keyvault-cccnsncfs2r4" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/c87a1bea-d45c-40f0-b36e-76c1418b3181 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/c87a1bea-d45c-40f0-b36e-76c1418b3181", "name": "c87a1bea-d45c-40f0-b36e-76c1418b3181", "status": "Failed", "startTime": "2026-10-02T20:39:19.772177296Z", "endTime": "2026-10-02T20:58:19.197731682Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine m9d7f6f9z0s9j7k-np-1-m4ndt-mcr5z: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine m9d7f6f9z0s9j7k-np-1-m4ndt-pn7ft: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine m9d7f6f9z0s9j7k-np-1-m4ndt-mcr5z: Creating: virtualmachine creating or updating\nMachine m9d7f6f9z0s9j7k-np-1-m4ndt-pn7ft: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:138]: failed to create node pool "np-1" for private keyvault cluster "private-kv-cluster" Unexpected error: <fmt.wrapError \| 0xc000f6f0a0>: failed to create NodePool np-1: failed waiting for nodepool="np-1" for cluster "private-kv-cluster" in resourcegroup="private-keyvault-cccnsncfs2r4" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/c87a1bea-d45c-40f0-b36e-76c1418b3181 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/c87a1bea-d45c-40f0-b36e-76c1418b3181", "name": "c87a1bea-d45c-40f0-b36e-76c1418b3181", "status": "Failed", "startTime": "2026-10-02T20:39:19.772177296Z", "endTime": "2026-10-02T20:58:19.197731682Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine m9d7f6f9z0s9j7k-np-1-m4ndt-mcr5z: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine m9d7f6f9z0s9j7k-np-1-m4ndt-pn7ft: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine m9d7f6f9z0s9j7k-np-1-m4ndt-mcr5z: Creating: virtualmachine creating or updating\nMachine m9d7f6f9z0s9j7k-np-1-m4ndt-pn7ft: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 20:18:59 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 5.0 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-4rsvdk" for cluster "cluster-candidate-5-0-4rsvdk" Unexpected error: <fmt.wrapError \| 0xc000ced660>: failed to create NodePool np-4rsvdk: failed waiting for nodepool="np-4rsvdk" for cluster "cluster-candidate-5-0-4rsvdk" in resourcegroup="rg-candidate-5-0-mr9dnz4m68mv" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/fe99a24d-45b0-4c02-b399-8a58c33f760f -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/fe99a24d-45b0-4c02-b399-8a58c33f760f", "name": "fe99a24d-45b0-4c02-b399-8a58c33f760f", "status": "Failed", "startTime": "2026-10-02T21:03:59.509943483Z", "endTime": "2026-10-02T21:23:08.337275316Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine g6y4a6l5u7h6m2s-np-4rsvdk-jgm75-gjgrx: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine g6y4a6l5u7h6m2s-np-4rsvdk-jgm75-gkprg: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine g6y4a6l5u7h6m2s-np-4rsvdk-jgm75-gjgrx: Creating: virtualmachine creating or updating\nMachine g6y4a6l5u7h6m2s-np-4rsvdk-jgm75-gkprg: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-4rsvdk" for cluster "cluster-candidate-5-0-4rsvdk" Unexpected error: <fmt.wrapError \| 0xc000ced660>: failed to create NodePool np-4rsvdk: failed waiting for nodepool="np-4rsvdk" for cluster "cluster-candidate-5-0-4rsvdk" in resourcegroup="rg-candidate-5-0-mr9dnz4m68mv" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/fe99a24d-45b0-4c02-b399-8a58c33f760f -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/fe99a24d-45b0-4c02-b399-8a58c33f760f", "name": "fe99a24d-45b0-4c02-b399-8a58c33f760f", "status": "Failed", "startTime": "2026-10-02T21:03:59.509943483Z", "endTime": "2026-10-02T21:23:08.337275316Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine g6y4a6l5u7h6m2s-np-4rsvdk-jgm75-gjgrx: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine g6y4a6l5u7h6m2s-np-4rsvdk-jgm75-gkprg: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine g6y4a6l5u7h6m2s-np-4rsvdk-jgm75-gjgrx: Creating: virtualmachine creating or updating\nMachine g6y4a6l5u7h6m2s-np-4rsvdk-jgm75-gkprg: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift node pool not cached yet; provider Microsoft.RedHatOpenShift 20:18:59 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.21 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-7lqqwg" for cluster "cluster-candidate-4-21-7lqqwg" Unexpected error: <fmt.wrapError \| 0xc000d194a0>: failed to create NodePool np-7lqqwg: failed waiting for nodepool="np-7lqqwg" for cluster "cluster-candidate-4-21-7lqqwg" in resourcegroup="rg-candidate-4-21-fsj5jdg9tljp" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/d395f2bf-88ca-4f84-8998-c863978caf3f -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/d395f2bf-88ca-4f84-8998-c863978caf3f", "name": "d395f2bf-88ca-4f84-8998-c863978caf3f", "status": "Failed", "startTime": "2026-10-02T21:03:18.868079791Z", "endTime": "2026-10-02T21:22:18.599909683Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift node pool not cached yet" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-7lqqwg" for cluster "cluster-candidate-4-21-7lqqwg" Unexpected error: <fmt.wrapError \| 0xc000d194a0>: failed to create NodePool np-7lqqwg: failed waiting for nodepool="np-7lqqwg" for cluster "cluster-candidate-4-21-7lqqwg" in resourcegroup="rg-candidate-4-21-fsj5jdg9tljp" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/d395f2bf-88ca-4f84-8998-c863978caf3f -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/d395f2bf-88ca-4f84-8998-c863978caf3f", "name": "d395f2bf-88ca-4f84-8998-c863978caf3f", "status": "Failed", "startTime": "2026-10-02T21:03:18.868079791Z", "endTime": "2026-10-02T21:22:18.599909683Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift node pool not cached yet" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 20:18:59 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.23 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-878fr2" for cluster "cluster-candidate-4-23-878fr2" Unexpected error: <fmt.wrapError \| 0xc000bb09a0>: failed to create NodePool np-878fr2: failed waiting for nodepool="np-878fr2" for cluster "cluster-candidate-4-23-878fr2" in resourcegroup="rg-candidate-4-23-tq4tlj8sn8rt" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/d39da2e3-e8ed-4ecb-a802-520caf1d680a -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/d39da2e3-e8ed-4ecb-a802-520caf1d680a", "name": "d39da2e3-e8ed-4ecb-a802-520caf1d680a", "status": "Failed", "startTime": "2026-10-02T21:05:40.949801842Z", "endTime": "2026-10-02T21:24:48.213673897Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine g4g6i1d1r4t2m8l-np-878fr2-58lpw-7czvr: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine g4g6i1d1r4t2m8l-np-878fr2-58lpw-wcjsm: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine g4g6i1d1r4t2m8l-np-878fr2-58lpw-7czvr: Creating: virtualmachine creating or updating\nMachine g4g6i1d1r4t2m8l-np-878fr2-58lpw-wcjsm: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-878fr2" for cluster "cluster-candidate-4-23-878fr2" Unexpected error: <fmt.wrapError \| 0xc000bb09a0>: failed to create NodePool np-878fr2: failed waiting for nodepool="np-878fr2" for cluster "cluster-candidate-4-23-878fr2" in resourcegroup="rg-candidate-4-23-tq4tlj8sn8rt" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/d39da2e3-e8ed-4ecb-a802-520caf1d680a -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/d39da2e3-e8ed-4ecb-a802-520caf1d680a", "name": "d39da2e3-e8ed-4ecb-a802-520caf1d680a", "status": "Failed", "startTime": "2026-10-02T21:05:40.949801842Z", "endTime": "2026-10-02T21:24:48.213673897Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine g4g6i1d1r4t2m8l-np-878fr2-58lpw-7czvr: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine g4g6i1d1r4t2m8l-np-878fr2-58lpw-wcjsm: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine g4g6i1d1r4t2m8l-np-878fr2-58lpw-7czvr: Creating: virtualmachine creating or updating\nMachine g4g6i1d1r4t2m8l-np-878fr2-58lpw-wcjsm: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred
17:45:08	2h 29m 39s	branch-ci-Azure-ARO-HCP-main-e2e-stage-e2e-parallel post-goodmerged PR No additional run metadata captured	uksouth	e2e	failed	n/a	13	e2e (13) ERROR CODE: DeadlineExceeded; provider Microsoft.Compute 17:45:08 UTC · failed at e2e · test Customer should upgrade a nodepool to a version without Cincinnati upgrade edge z-stream upgrade without Cincinnati edge in 4.20 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:321]: failed to create nodepool npnoedge-4-20 at version 4.20.40 Unexpected error: <fmt.wrapError \| 0xc000ee2740>: failed to create NodePool npnoedge-4-20: failed waiting for nodepool="npnoedge-4-20" for cluster "np-noedge-ht5gkd" in resourcegroup="rg-np-noedge-ht5gkd-s9tjwqwzr9v2" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/2685dbe2-74b7-44fa-8866-0542656a2cbe -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/2685dbe2-74b7-44fa-8866-0542656a2cbe", "name": "2685dbe2-74b7-44fa-8866-0542656a2cbe", "status": "Failed", "startTime": "2026-10-02T19:26:52.969260057Z", "endTime": "2026-10-02T19:45:54.294162813Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 1, want 2; node pool condition AllNodesHealthy is False: 1 of 2 machines are not healthy\nMachine s8l8q3n7s8a7r2q-npnoedge-4-20-gplmr-p7qxl: InspectionFailed: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-noedge-ht5gkd-s9tjwqwzr9v2-np-ne-ht5gkd--managed/providers/Microsoft.Compute/virtualMachines/s8l8q3n7s8a7r2q-npnoedge-4-20-gplmr-p7qxl to exist\n; node pool condition AllMachinesReady is False: 1 of 2 machines are not ready\nMachine s8l8q3n7s8a7r2q-npnoedge-4-20-gplmr-p7qxl: ReadyUnknown: NodeHealthy: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-noedge-ht5gkd-s9tjwqwzr9v2-np-ne-ht5gkd--managed/providers/Microsoft.Compute/virtualMachines/s8l8q3n7s8a7r2q-npnoedge-4-20-gplmr-p7qxl to exist\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:321]: failed to create nodepool npnoedge-4-20 at version 4.20.40 Unexpected error: <fmt.wrapError \| 0xc000ee2740>: failed to create NodePool npnoedge-4-20: failed waiting for nodepool="npnoedge-4-20" for cluster "np-noedge-ht5gkd" in resourcegroup="rg-np-noedge-ht5gkd-s9tjwqwzr9v2" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/2685dbe2-74b7-44fa-8866-0542656a2cbe -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/2685dbe2-74b7-44fa-8866-0542656a2cbe", "name": "2685dbe2-74b7-44fa-8866-0542656a2cbe", "status": "Failed", "startTime": "2026-10-02T19:26:52.969260057Z", "endTime": "2026-10-02T19:45:54.294162813Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 1, want 2; node pool condition AllNodesHealthy is False: 1 of 2 machines are not healthy\nMachine s8l8q3n7s8a7r2q-npnoedge-4-20-gplmr-p7qxl: InspectionFailed: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-noedge-ht5gkd-s9tjwqwzr9v2-np-ne-ht5gkd--managed/providers/Microsoft.Compute/virtualMachines/s8l8q3n7s8a7r2q-npnoedge-4-20-gplmr-p7qxl to exist\n; node pool condition AllMachinesReady is False: 1 of 2 machines are not ready\nMachine s8l8q3n7s8a7r2q-npnoedge-4-20-gplmr-p7qxl: ReadyUnknown: NodeHealthy: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-noedge-ht5gkd-s9tjwqwzr9v2-np-ne-ht5gkd--managed/providers/Microsoft.Compute/virtualMachines/s8l8q3n7s8a7r2q-npnoedge-4-20-gplmr-p7qxl to exist\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 17:45:08 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.22 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-97jnb4" for cluster "cluster-candidate-4-22-97jnb4" Unexpected error: <fmt.wrapError \| 0xc000a61960>: failed to create NodePool np-97jnb4: failed waiting for nodepool="np-97jnb4" for cluster "cluster-candidate-4-22-97jnb4" in resourcegroup="rg-candidate-4-22-gqvl8n7h4bmq" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/d14fd998-aa3e-49c8-a405-91cc8c02a377 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/d14fd998-aa3e-49c8-a405-91cc8c02a377", "name": "d14fd998-aa3e-49c8-a405-91cc8c02a377", "status": "Failed", "startTime": "2026-10-02T18:49:32.578309403Z", "endTime": "2026-10-02T19:08:33.241090855Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine v5r4w0l0u4k4n4f-np-97jnb4-k5gv8-dkh4p: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine v5r4w0l0u4k4n4f-np-97jnb4-k5gv8-hz2bh: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine v5r4w0l0u4k4n4f-np-97jnb4-k5gv8-dkh4p: Creating: virtualmachine creating or updating\nMachine v5r4w0l0u4k4n4f-np-97jnb4-k5gv8-hz2bh: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-97jnb4" for cluster "cluster-candidate-4-22-97jnb4" Unexpected error: <fmt.wrapError \| 0xc000a61960>: failed to create NodePool np-97jnb4: failed waiting for nodepool="np-97jnb4" for cluster "cluster-candidate-4-22-97jnb4" in resourcegroup="rg-candidate-4-22-gqvl8n7h4bmq" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/d14fd998-aa3e-49c8-a405-91cc8c02a377 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/d14fd998-aa3e-49c8-a405-91cc8c02a377", "name": "d14fd998-aa3e-49c8-a405-91cc8c02a377", "status": "Failed", "startTime": "2026-10-02T18:49:32.578309403Z", "endTime": "2026-10-02T19:08:33.241090855Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine v5r4w0l0u4k4n4f-np-97jnb4-k5gv8-dkh4p: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine v5r4w0l0u4k4n4f-np-97jnb4-k5gv8-hz2bh: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine v5r4w0l0u4k4n4f-np-97jnb4-k5gv8-dkh4p: Creating: virtualmachine creating or updating\nMachine v5r4w0l0u4k4n4f-np-97jnb4-k5gv8-hz2bh: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want >= <count> (autoscaling min); node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 17:45:08 UTC · failed at e2e · test Customer should be able to create a cluster and node pools with aggregated advanced features · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:404]: failed to verify cilium is running and nodes are Ready for cluster "agg-cluster" Unexpected error: <errors.joinError \| 0xc001522060>: VerifyNodesReady failed: no nodes found in the cluster VerifyCiliumOperational failed: not all pods in kube-system namespace are running: failed to list pods in kube-system namespace: client rate limiter Wait returned an error: context deadline exceeded failed to create NodePool agg-np-a: failed waiting for nodepool="agg-np-a" for cluster "agg-cluster" in resourcegroup="feature-aggregation-nl8vdbhxwtvx" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/31e50407-d006-4e42-9498-e3bbc2ee0de1 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/31e50407-d006-4e42-9498-e3bbc2ee0de1", "name": "31e50407-d006-4e42-9498-e3bbc2ee0de1", "status": "Failed", "startTime": "2026-10-02T18:27:19.35370587Z", "endTime": "2026-10-02T18:46:22.521838138Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want \u003e= 2 (autoscaling min); node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine agg-cluster-agg-np-a-h6sjn-cnj2s: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine agg-cluster-agg-np-a-h6sjn-h8wr9: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine agg-cluster-agg-np-a-h6sjn-cnj2s: Creating: virtualmachine creating or updating\nMachine agg-cluster-agg-np-a-h6sjn-h8wr9: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:404]: failed to verify cilium is running and nodes are Ready for cluster "agg-cluster" Unexpected error: <errors.joinError \| 0xc001522060>: VerifyNodesReady failed: no nodes found in the cluster VerifyCiliumOperational failed: not all pods in kube-system namespace are running: failed to list pods in kube-system namespace: client rate limiter Wait returned an error: context deadline exceeded failed to create NodePool agg-np-a: failed waiting for nodepool="agg-np-a" for cluster "agg-cluster" in resourcegroup="feature-aggregation-nl8vdbhxwtvx" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/31e50407-d006-4e42-9498-e3bbc2ee0de1 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/31e50407-d006-4e42-9498-e3bbc2ee0de1", "name": "31e50407-d006-4e42-9498-e3bbc2ee0de1", "status": "Failed", "startTime": "2026-10-02T18:27:19.35370587Z", "endTime": "2026-10-02T18:46:22.521838138Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want \u003e= 2 (autoscaling min); node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine agg-cluster-agg-np-a-h6sjn-cnj2s: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine agg-cluster-agg-np-a-h6sjn-h8wr9: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine agg-cluster-agg-np-a-h6sjn-cnj2s: Creating: virtualmachine creating or updating\nMachine agg-cluster-agg-np-a-h6sjn-h8wr9: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 17:45:08 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.21 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-kfvn25" for cluster "cluster-candidate-4-21-kfvn25" Unexpected error: <fmt.wrapError \| 0xc000b3b220>: failed to create NodePool np-kfvn25: failed waiting for nodepool="np-kfvn25" for cluster "cluster-candidate-4-21-kfvn25" in resourcegroup="rg-candidate-4-21-85wnrgqqbd2h" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b12e98e2-7f93-4004-87fa-b863b0f0044f -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b12e98e2-7f93-4004-87fa-b863b0f0044f", "name": "b12e98e2-7f93-4004-87fa-b863b0f0044f", "status": "Failed", "startTime": "2026-10-02T18:49:07.875106668Z", "endTime": "2026-10-02T19:08:13.588598197Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine l6p5f7e6i7t3n7m-np-kfvn25-4sz9p-f65j4: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine l6p5f7e6i7t3n7m-np-kfvn25-4sz9p-g5bl8: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine l6p5f7e6i7t3n7m-np-kfvn25-4sz9p-f65j4: Creating: virtualmachine creating or updating\nMachine l6p5f7e6i7t3n7m-np-kfvn25-4sz9p-g5bl8: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-kfvn25" for cluster "cluster-candidate-4-21-kfvn25" Unexpected error: <fmt.wrapError \| 0xc000b3b220>: failed to create NodePool np-kfvn25: failed waiting for nodepool="np-kfvn25" for cluster "cluster-candidate-4-21-kfvn25" in resourcegroup="rg-candidate-4-21-85wnrgqqbd2h" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b12e98e2-7f93-4004-87fa-b863b0f0044f -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b12e98e2-7f93-4004-87fa-b863b0f0044f", "name": "b12e98e2-7f93-4004-87fa-b863b0f0044f", "status": "Failed", "startTime": "2026-10-02T18:49:07.875106668Z", "endTime": "2026-10-02T19:08:13.588598197Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine l6p5f7e6i7t3n7m-np-kfvn25-4sz9p-f65j4: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine l6p5f7e6i7t3n7m-np-kfvn25-4sz9p-g5bl8: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine l6p5f7e6i7t3n7m-np-kfvn25-4sz9p-f65j4: Creating: virtualmachine creating or updating\nMachine l6p5f7e6i7t3n7m-np-kfvn25-4sz9p-g5bl8: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 17:45:08 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.20 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-vvnbz4" for cluster "cluster-candidate-4-20-vvnbz4" Unexpected error: <fmt.wrapError \| 0xc000c04c80>: failed to create NodePool np-vvnbz4: failed waiting for nodepool="np-vvnbz4" for cluster "cluster-candidate-4-20-vvnbz4" in resourcegroup="rg-candidate-4-20-6rrzp6smfk8r" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/a84f10bd-1000-4ff5-b696-ec0471f172d5 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/a84f10bd-1000-4ff5-b696-ec0471f172d5", "name": "a84f10bd-1000-4ff5-b696-ec0471f172d5", "status": "Failed", "startTime": "2026-10-02T18:45:19.007012696Z", "endTime": "2026-10-02T19:04:23.174354297Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine j3l2b9y9v8f1o9h-np-vvnbz4-llscx-jqrxn: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine j3l2b9y9v8f1o9h-np-vvnbz4-llscx-rd5xk: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine j3l2b9y9v8f1o9h-np-vvnbz4-llscx-jqrxn: Creating: virtualmachine creating or updating\nMachine j3l2b9y9v8f1o9h-np-vvnbz4-llscx-rd5xk: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-vvnbz4" for cluster "cluster-candidate-4-20-vvnbz4" Unexpected error: <fmt.wrapError \| 0xc000c04c80>: failed to create NodePool np-vvnbz4: failed waiting for nodepool="np-vvnbz4" for cluster "cluster-candidate-4-20-vvnbz4" in resourcegroup="rg-candidate-4-20-6rrzp6smfk8r" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/a84f10bd-1000-4ff5-b696-ec0471f172d5 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/a84f10bd-1000-4ff5-b696-ec0471f172d5", "name": "a84f10bd-1000-4ff5-b696-ec0471f172d5", "status": "Failed", "startTime": "2026-10-02T18:45:19.007012696Z", "endTime": "2026-10-02T19:04:23.174354297Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine j3l2b9y9v8f1o9h-np-vvnbz4-llscx-jqrxn: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine j3l2b9y9v8f1o9h-np-vvnbz4-llscx-rd5xk: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine j3l2b9y9v8f1o9h-np-vvnbz4-llscx-jqrxn: Creating: virtualmachine creating or updating\nMachine j3l2b9y9v8f1o9h-np-vvnbz4-llscx-rd5xk: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 17:45:08 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.23 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-qv99kb" for cluster "cluster-candidate-4-23-qv99kb" Unexpected error: <fmt.wrapError \| 0xc000c78ba0>: failed to create NodePool np-qv99kb: failed waiting for nodepool="np-qv99kb" for cluster "cluster-candidate-4-23-qv99kb" in resourcegroup="rg-candidate-4-23-xhtfbmhw8nkt" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/93ec2ac0-8140-4df7-861f-8f5e69bd2825 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/93ec2ac0-8140-4df7-861f-8f5e69bd2825", "name": "93ec2ac0-8140-4df7-861f-8f5e69bd2825", "status": "Failed", "startTime": "2026-10-02T18:47:56.273213854Z", "endTime": "2026-10-02T19:07:03.234946252Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine o0z3a3e4t8t4l0z-np-qv99kb-rmmkl-nhvhz: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine o0z3a3e4t8t4l0z-np-qv99kb-rmmkl-xq4s5: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine o0z3a3e4t8t4l0z-np-qv99kb-rmmkl-nhvhz: Creating: virtualmachine creating or updating\nMachine o0z3a3e4t8t4l0z-np-qv99kb-rmmkl-xq4s5: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-qv99kb" for cluster "cluster-candidate-4-23-qv99kb" Unexpected error: <fmt.wrapError \| 0xc000c78ba0>: failed to create NodePool np-qv99kb: failed waiting for nodepool="np-qv99kb" for cluster "cluster-candidate-4-23-qv99kb" in resourcegroup="rg-candidate-4-23-xhtfbmhw8nkt" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/93ec2ac0-8140-4df7-861f-8f5e69bd2825 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/93ec2ac0-8140-4df7-861f-8f5e69bd2825", "name": "93ec2ac0-8140-4df7-861f-8f5e69bd2825", "status": "Failed", "startTime": "2026-10-02T18:47:56.273213854Z", "endTime": "2026-10-02T19:07:03.234946252Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine o0z3a3e4t8t4l0z-np-qv99kb-rmmkl-nhvhz: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine o0z3a3e4t8t4l0z-np-qv99kb-rmmkl-xq4s5: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine o0z3a3e4t8t4l0z-np-qv99kb-rmmkl-nhvhz: Creating: virtualmachine creating or updating\nMachine o0z3a3e4t8t4l0z-np-qv99kb-rmmkl-xq4s5: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 17:45:08 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 5.0 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-xf2nmx" for cluster "cluster-candidate-5-0-xf2nmx" Unexpected error: <fmt.wrapError \| 0xc000995320>: failed to create NodePool np-xf2nmx: failed waiting for nodepool="np-xf2nmx" for cluster "cluster-candidate-5-0-xf2nmx" in resourcegroup="rg-candidate-5-0-7jr6wdpsfvz4" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/992a0c66-9c4a-4dc5-8b0f-2c46d65526fe -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/992a0c66-9c4a-4dc5-8b0f-2c46d65526fe", "name": "992a0c66-9c4a-4dc5-8b0f-2c46d65526fe", "status": "Failed", "startTime": "2026-10-02T18:49:51.032351879Z", "endTime": "2026-10-02T19:08:53.234722812Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine b1t7y5a4d6l6s5p-np-xf2nmx-bnzwr-kqt4r: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine b1t7y5a4d6l6s5p-np-xf2nmx-bnzwr-nwmrg: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine b1t7y5a4d6l6s5p-np-xf2nmx-bnzwr-kqt4r: Creating: virtualmachine creating or updating\nMachine b1t7y5a4d6l6s5p-np-xf2nmx-bnzwr-nwmrg: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-xf2nmx" for cluster "cluster-candidate-5-0-xf2nmx" Unexpected error: <fmt.wrapError \| 0xc000995320>: failed to create NodePool np-xf2nmx: failed waiting for nodepool="np-xf2nmx" for cluster "cluster-candidate-5-0-xf2nmx" in resourcegroup="rg-candidate-5-0-7jr6wdpsfvz4" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/992a0c66-9c4a-4dc5-8b0f-2c46d65526fe -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/992a0c66-9c4a-4dc5-8b0f-2c46d65526fe", "name": "992a0c66-9c4a-4dc5-8b0f-2c46d65526fe", "status": "Failed", "startTime": "2026-10-02T18:49:51.032351879Z", "endTime": "2026-10-02T19:08:53.234722812Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine b1t7y5a4d6l6s5p-np-xf2nmx-bnzwr-kqt4r: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine b1t7y5a4d6l6s5p-np-xf2nmx-bnzwr-nwmrg: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine b1t7y5a4d6l6s5p-np-xf2nmx-bnzwr-kqt4r: Creating: virtualmachine creating or updating\nMachine b1t7y5a4d6l6s5p-np-xf2nmx-bnzwr-nwmrg: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 17:45:08 UTC · failed at e2e · test Create HCPOpenShiftCluster with Private KeyVault should create a cluster with private keyvault using v20251223preview API · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:138]: failed to create node pool "np-1" for private keyvault cluster "private-kv-cluster" Unexpected error: <fmt.wrapError \| 0xc000fcce00>: failed to create NodePool np-1: failed waiting for nodepool="np-1" for cluster "private-kv-cluster" in resourcegroup="private-keyvault-kndcxqxwg5m4" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/9940d584-67e5-4b2a-939b-305a5a782054 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/9940d584-67e5-4b2a-939b-305a5a782054", "name": "9940d584-67e5-4b2a-939b-305a5a782054", "status": "Failed", "startTime": "2026-10-02T18:26:54.435762769Z", "endTime": "2026-10-02T18:46:02.862827665Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine l1e8j9h2p9p7t8t-np-1-dsx99-6stxx: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine l1e8j9h2p9p7t8t-np-1-dsx99-fvklg: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine l1e8j9h2p9p7t8t-np-1-dsx99-6stxx: Creating: virtualmachine creating or updating\nMachine l1e8j9h2p9p7t8t-np-1-dsx99-fvklg: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:138]: failed to create node pool "np-1" for private keyvault cluster "private-kv-cluster" Unexpected error: <fmt.wrapError \| 0xc000fcce00>: failed to create NodePool np-1: failed waiting for nodepool="np-1" for cluster "private-kv-cluster" in resourcegroup="private-keyvault-kndcxqxwg5m4" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/9940d584-67e5-4b2a-939b-305a5a782054 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/9940d584-67e5-4b2a-939b-305a5a782054", "name": "9940d584-67e5-4b2a-939b-305a5a782054", "status": "Failed", "startTime": "2026-10-02T18:26:54.435762769Z", "endTime": "2026-10-02T18:46:02.862827665Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine l1e8j9h2p9p7t8t-np-1-dsx99-6stxx: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine l1e8j9h2p9p7t8t-np-1-dsx99-fvklg: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine l1e8j9h2p9p7t8t-np-1-dsx99-6stxx: Creating: virtualmachine creating or updating\nMachine l1e8j9h2p9p7t8t-np-1-dsx99-fvklg: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 17:45:08 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 5.1 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-lvw97g" for cluster "cluster-candidate-5-1-lvw97g" Unexpected error: <fmt.wrapError \| 0xc000e66f00>: failed to create NodePool np-lvw97g: failed waiting for nodepool="np-lvw97g" for cluster "cluster-candidate-5-1-lvw97g" in resourcegroup="rg-candidate-5-1-qrgrjlf6rkm9" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4a97c7bb-57af-45b7-8fb7-5b981cf7570f -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4a97c7bb-57af-45b7-8fb7-5b981cf7570f", "name": "4a97c7bb-57af-45b7-8fb7-5b981cf7570f", "status": "Failed", "startTime": "2026-10-02T18:50:51.431353811Z", "endTime": "2026-10-02T19:09:53.217243745Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine l3n7x5q5h0o7l6w-np-lvw97g-9smn8-95ljv: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine l3n7x5q5h0o7l6w-np-lvw97g-9smn8-cfkc9: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine l3n7x5q5h0o7l6w-np-lvw97g-9smn8-95ljv: Creating: virtualmachine creating or updating\nMachine l3n7x5q5h0o7l6w-np-lvw97g-9smn8-cfkc9: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-lvw97g" for cluster "cluster-candidate-5-1-lvw97g" Unexpected error: <fmt.wrapError \| 0xc000e66f00>: failed to create NodePool np-lvw97g: failed waiting for nodepool="np-lvw97g" for cluster "cluster-candidate-5-1-lvw97g" in resourcegroup="rg-candidate-5-1-qrgrjlf6rkm9" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4a97c7bb-57af-45b7-8fb7-5b981cf7570f -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4a97c7bb-57af-45b7-8fb7-5b981cf7570f", "name": "4a97c7bb-57af-45b7-8fb7-5b981cf7570f", "status": "Failed", "startTime": "2026-10-02T18:50:51.431353811Z", "endTime": "2026-10-02T19:09:53.217243745Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine l3n7x5q5h0o7l6w-np-lvw97g-9smn8-95ljv: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine l3n7x5q5h0o7l6w-np-lvw97g-9smn8-cfkc9: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine l3n7x5q5h0o7l6w-np-lvw97g-9smn8-95ljv: Creating: virtualmachine creating or updating\nMachine l3n7x5q5h0o7l6w-np-lvw97g-9smn8-cfkc9: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; provider Microsoft.Compute 17:45:08 UTC · failed at e2e · test Customer should downgrade a nodepool version z-stream downgrade from 4.21.zLatest to 4.21.zPrevious · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:600]: failed to create nodepool npdg-4-21 Unexpected error: <fmt.wrapError \| 0xc000c9c280>: failed to create NodePool npdg-4-21: failed waiting for nodepool="npdg-4-21" for cluster "np-downgrade-gg8s9j" in resourcegroup="rg-np-downgrade-gg8s9j-fnkpnt9xd47t" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/a4186f06-3e4f-4125-8780-405aa920bd3f -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/a4186f06-3e4f-4125-8780-405aa920bd3f", "name": "a4186f06-3e4f-4125-8780-405aa920bd3f", "status": "Failed", "startTime": "2026-10-02T19:29:21.050247186Z", "endTime": "2026-10-02T19:48:24.533281016Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 1, want 2; node pool condition AllNodesHealthy is False: 1 of 2 machines are not healthy\nMachine a7o8k0t2r9e8d6s-npdg-4-21-xzxrt-5htvd: InspectionFailed: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-downgrade-gg8s9j-fnkpnt9xd47t-np-dg-gg8s9j--managed/providers/Microsoft.Compute/virtualMachines/a7o8k0t2r9e8d6s-npdg-4-21-xzxrt-5htvd to exist\n; node pool condition AllMachinesReady is False: 1 of 2 machines are not ready\nMachine a7o8k0t2r9e8d6s-npdg-4-21-xzxrt-5htvd: ReadyUnknown: NodeHealthy: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-downgrade-gg8s9j-fnkpnt9xd47t-np-dg-gg8s9j--managed/providers/Microsoft.Compute/virtualMachines/a7o8k0t2r9e8d6s-npdg-4-21-xzxrt-5htvd to exist\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:600]: failed to create nodepool npdg-4-21 Unexpected error: <fmt.wrapError \| 0xc000c9c280>: failed to create NodePool npdg-4-21: failed waiting for nodepool="npdg-4-21" for cluster "np-downgrade-gg8s9j" in resourcegroup="rg-np-downgrade-gg8s9j-fnkpnt9xd47t" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/a4186f06-3e4f-4125-8780-405aa920bd3f -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/a4186f06-3e4f-4125-8780-405aa920bd3f", "name": "a4186f06-3e4f-4125-8780-405aa920bd3f", "status": "Failed", "startTime": "2026-10-02T19:29:21.050247186Z", "endTime": "2026-10-02T19:48:24.533281016Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 1, want 2; node pool condition AllNodesHealthy is False: 1 of 2 machines are not healthy\nMachine a7o8k0t2r9e8d6s-npdg-4-21-xzxrt-5htvd: InspectionFailed: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-downgrade-gg8s9j-fnkpnt9xd47t-np-dg-gg8s9j--managed/providers/Microsoft.Compute/virtualMachines/a7o8k0t2r9e8d6s-npdg-4-21-xzxrt-5htvd to exist\n; node pool condition AllMachinesReady is False: 1 of 2 machines are not ready\nMachine a7o8k0t2r9e8d6s-npdg-4-21-xzxrt-5htvd: ReadyUnknown: NodeHealthy: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-downgrade-gg8s9j-fnkpnt9xd47t-np-dg-gg8s9j--managed/providers/Microsoft.Compute/virtualMachines/a7o8k0t2r9e8d6s-npdg-4-21-xzxrt-5htvd to exist\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 17:45:08 UTC · failed at e2e · test Customer should create a cluster with private KAS and verify API server is only reachable from VNet while ingress remains public · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:159]: failed to create node pool "np-1" for private KAS cluster "private-kas" Unexpected error: <fmt.wrapError \| 0xc001085220>: failed to create NodePool np-1: failed waiting for nodepool="np-1" for cluster "private-kas" in resourcegroup="private-kas-dk96qxfcglkz" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/a2c0d616-d3e1-4467-a097-edfb164f91f8 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/a2c0d616-d3e1-4467-a097-edfb164f91f8", "name": "a2c0d616-d3e1-4467-a097-edfb164f91f8", "status": "Failed", "startTime": "2026-10-02T18:26:30.147930745Z", "endTime": "2026-10-02T18:45:32.745630715Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 1; node pool condition AllNodesHealthy is False: 1 of 1 machines are not healthy\nMachine private-kas-np-1-jbbww-kglw7: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 1 of 1 machines are not ready\nMachine private-kas-np-1-jbbww-kglw7: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:159]: failed to create node pool "np-1" for private KAS cluster "private-kas" Unexpected error: <fmt.wrapError \| 0xc001085220>: failed to create NodePool np-1: failed waiting for nodepool="np-1" for cluster "private-kas" in resourcegroup="private-kas-dk96qxfcglkz" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/a2c0d616-d3e1-4467-a097-edfb164f91f8 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/a2c0d616-d3e1-4467-a097-edfb164f91f8", "name": "a2c0d616-d3e1-4467-a097-edfb164f91f8", "status": "Failed", "startTime": "2026-10-02T18:26:30.147930745Z", "endTime": "2026-10-02T18:45:32.745630715Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 1; node pool condition AllNodesHealthy is False: 1 of 1 machines are not healthy\nMachine private-kas-np-1-jbbww-kglw7: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 1 of 1 machines are not ready\nMachine private-kas-np-1-jbbww-kglw7: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred failed to create nodepool <nodepool>: failed waiting for nodepool="<nodepool>" for cluster "<cluster>" in resourcegroup="<resource-group>" to finish creating: GET <url> 17:45:08 UTC · failed at e2e · test Customer should be able to delete a nodepool · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_delete.go:128]: failed to create node pools Expected success, but got an error: <fmt.wrapError \| 0xc00341d8a0>: failed to create NodePool np-delete-main: failed waiting for nodepool="np-delete-main" for cluster "np-delete-hcp-cluster" in resourcegroup="nodepool-delete-8s9pgvgwlgbj" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/e4e9a75c-b715-4a5f-8f78-888a4044e76e -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/e4e9a75c-b715-4a5f-8f78-888a4044e76e", "name": "e4e9a75c-b715-4a5f-8f78-888a4044e76e", "status": "Failed", "startTime": "2026-10-02T19:20:19.695876004Z", "endTime": "2026-10-02T19:39:24.271020967Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine k8i8k9n2m6a4a0n-np-delete-main-pkm89-4mb96: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine k8i8k9n2m6a4a0n-np-delete-main-pkm89-8f5lf: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine k8i8k9n2m6a4a0n-np-delete-main-pkm89-4mb96: Creating: virtualmachine creating or updating\nMachine k8i8k9n2m6a4a0n-np-delete-main-pkm89-8f5lf: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_delete.go:128]: failed to create node pools Expected success, but got an error: <fmt.wrapError \| 0xc00341d8a0>: failed to create NodePool np-delete-main: failed waiting for nodepool="np-delete-main" for cluster "np-delete-hcp-cluster" in resourcegroup="nodepool-delete-8s9pgvgwlgbj" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/e4e9a75c-b715-4a5f-8f78-888a4044e76e -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/e4e9a75c-b715-4a5f-8f78-888a4044e76e", "name": "e4e9a75c-b715-4a5f-8f78-888a4044e76e", "status": "Failed", "startTime": "2026-10-02T19:20:19.695876004Z", "endTime": "2026-10-02T19:39:24.271020967Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine k8i8k9n2m6a4a0n-np-delete-main-pkm89-4mb96: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine k8i8k9n2m6a4a0n-np-delete-main-pkm89-8f5lf: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine k8i8k9n2m6a4a0n-np-delete-main-pkm89-4mb96: Creating: virtualmachine creating or updating\nMachine k8i8k9n2m6a4a0n-np-delete-main-pkm89-8f5lf: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 17:45:08 UTC · failed at e2e · test Customer should be able to create a no-CNI private cluster with a private key vault, a nodepool and install cilium CNI successfully · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:193]: failed to verify nodes are Ready with Cilium CNI for cluster "cilium-cluster" Unexpected error: <errors.joinError \| 0xc001f12060>: VerifyNodesReady failed: no nodes found in the cluster VerifyCiliumOperational failed: not all pods in kube-system namespace are running: failed to list pods in kube-system namespace: client rate limiter Wait returned an error: context deadline exceeded failed to create NodePool cilium-np: failed waiting for nodepool="cilium-np" for cluster "cilium-cluster" in resourcegroup="complex-cilium-kv-5x984bppgxmk" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/f889fa2b-1853-4b53-b0a5-8af8bb1664c4 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/f889fa2b-1853-4b53-b0a5-8af8bb1664c4", "name": "f889fa2b-1853-4b53-b0a5-8af8bb1664c4", "status": "Failed", "startTime": "2026-10-02T18:28:21.353402223Z", "endTime": "2026-10-02T18:47:22.523072731Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine cilium-cluster-cilium-np-rvbvq-47m2n: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine cilium-cluster-cilium-np-rvbvq-dncqc: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine cilium-cluster-cilium-np-rvbvq-47m2n: Creating: virtualmachine creating or updating\nMachine cilium-cluster-cilium-np-rvbvq-dncqc: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:193]: failed to verify nodes are Ready with Cilium CNI for cluster "cilium-cluster" Unexpected error: <errors.joinError \| 0xc001f12060>: VerifyNodesReady failed: no nodes found in the cluster VerifyCiliumOperational failed: not all pods in kube-system namespace are running: failed to list pods in kube-system namespace: client rate limiter Wait returned an error: context deadline exceeded failed to create NodePool cilium-np: failed waiting for nodepool="cilium-np" for cluster "cilium-cluster" in resourcegroup="complex-cilium-kv-5x984bppgxmk" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/f889fa2b-1853-4b53-b0a5-8af8bb1664c4 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/f889fa2b-1853-4b53-b0a5-8af8bb1664c4", "name": "f889fa2b-1853-4b53-b0a5-8af8bb1664c4", "status": "Failed", "startTime": "2026-10-02T18:28:21.353402223Z", "endTime": "2026-10-02T18:47:22.523072731Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine cilium-cluster-cilium-np-rvbvq-47m2n: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine cilium-cluster-cilium-np-rvbvq-dncqc: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine cilium-cluster-cilium-np-rvbvq-47m2n: Creating: virtualmachine creating or updating\nMachine cilium-cluster-cilium-np-rvbvq-dncqc: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred
07:30:54	2h 20m 8s	branch-ci-Azure-ARO-HCP-main-e2e-stage-e2e-parallel post-goodmerged PR No additional run metadata captured	uksouth	e2e	failed	n/a	12	e2e (12) ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 07:30:54 UTC · failed at e2e · test Customer should create a cluster with private KAS and verify API server is only reachable from VNet while ingress remains public · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:159]: failed to create node pool "np-1" for private KAS cluster "private-kas" Unexpected error: <fmt.wrapError \| 0xc000a9d580>: failed to create NodePool np-1: failed waiting for nodepool="np-1" for cluster "private-kas" in resourcegroup="private-kas-vrplznt5v54r" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/85d24adc-6f21-47c1-8812-f58f8001bb49 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/85d24adc-6f21-47c1-8812-f58f8001bb49", "name": "85d24adc-6f21-47c1-8812-f58f8001bb49", "status": "Failed", "startTime": "2026-10-02T07:56:13.127412921Z", "endTime": "2026-10-02T08:15:18.247774488Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 1; node pool condition AllNodesHealthy is False: 1 of 1 machines are not healthy\nMachine private-kas-np-1-t8dkq-6vlv4: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 1 of 1 machines are not ready\nMachine private-kas-np-1-t8dkq-6vlv4: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:159]: failed to create node pool "np-1" for private KAS cluster "private-kas" Unexpected error: <fmt.wrapError \| 0xc000a9d580>: failed to create NodePool np-1: failed waiting for nodepool="np-1" for cluster "private-kas" in resourcegroup="private-kas-vrplznt5v54r" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/85d24adc-6f21-47c1-8812-f58f8001bb49 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/85d24adc-6f21-47c1-8812-f58f8001bb49", "name": "85d24adc-6f21-47c1-8812-f58f8001bb49", "status": "Failed", "startTime": "2026-10-02T07:56:13.127412921Z", "endTime": "2026-10-02T08:15:18.247774488Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 1; node pool condition AllNodesHealthy is False: 1 of 1 machines are not healthy\nMachine private-kas-np-1-t8dkq-6vlv4: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 1 of 1 machines are not ready\nMachine private-kas-np-1-t8dkq-6vlv4: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred failed to create nodepool <nodepool>: failed waiting for nodepool="<nodepool>" for cluster "<cluster>" in resourcegroup="<resource-group>" to finish creating: GET <url> 07:30:54 UTC · failed at e2e · test Customer should be able to delete a nodepool · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_delete.go:128]: failed to create node pools Expected success, but got an error: <fmt.wrapError \| 0xc000666c60>: failed to create NodePool np-delete-main: failed waiting for nodepool="np-delete-main" for cluster "np-delete-hcp-cluster" in resourcegroup="nodepool-delete-bgj7nclctwxw" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/29fe846e-0932-41fe-a2dd-ff17d6cc5ac0 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/29fe846e-0932-41fe-a2dd-ff17d6cc5ac0", "name": "29fe846e-0932-41fe-a2dd-ff17d6cc5ac0", "status": "Failed", "startTime": "2026-10-02T08:52:01.290279416Z", "endTime": "2026-10-02T09:11:09.436811147Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine h3r3f2u9k1p8i1c-np-delete-main-hwgw9-4k649: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine h3r3f2u9k1p8i1c-np-delete-main-hwgw9-p69q6: InspectionFailed: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/nodepool-delete-bgj7nclctwxw--managed/providers/Microsoft.Compute/virtualMachines/h3r3f2u9k1p8i1c-np-delete-main-hwgw9-p69q6 to exist\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine h3r3f2u9k1p8i1c-np-delete-main-hwgw9-4k649: Creating: virtualmachine creating or updating\nMachine h3r3f2u9k1p8i1c-np-delete-main-hwgw9-p69q6: ReadyUnknown: NodeHealthy: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/nodepool-delete-bgj7nclctwxw--managed/providers/Microsoft.Compute/virtualMachines/h3r3f2u9k1p8i1c-np-delete-main-hwgw9-p69q6 to exist\n" } } -------------------------------------------------------------------------------- ... fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_delete.go:128]: failed to create node pools Expected success, but got an error: <fmt.wrapError \| 0xc000666c60>: failed to create NodePool np-delete-main: failed waiting for nodepool="np-delete-main" for cluster "np-delete-hcp-cluster" in resourcegroup="nodepool-delete-bgj7nclctwxw" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/29fe846e-0932-41fe-a2dd-ff17d6cc5ac0 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/29fe846e-0932-41fe-a2dd-ff17d6cc5ac0", "name": "29fe846e-0932-41fe-a2dd-ff17d6cc5ac0", "status": "Failed", "startTime": "2026-10-02T08:52:01.290279416Z", "endTime": "2026-10-02T09:11:09.436811147Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine h3r3f2u9k1p8i1c-np-delete-main-hwgw9-4k649: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine h3r3f2u9k1p8i1c-np-delete-main-hwgw9-p69q6: InspectionFailed: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/nodepool-delete-bgj7nclctwxw--managed/providers/Microsoft.Compute/virtualMachines/h3r3f2u9k1p8i1c-np-delete-main-hwgw9-p69q6 to exist\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine h3r3f2u9k1p8i1c-np-delete-main-hwgw9-4k649: Creating: virtualmachine creating or updating\nMachine h3r3f2u9k1p8i1c-np-delete-main-hwgw9-p69q6: ReadyUnknown: NodeHealthy: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/nodepool-delete-bgj7nclctwxw--managed/providers/Microsoft.Compute/virtualMachines/h3r3f2u9k1p8i1c-np-delete-main-hwgw9-p69q6 to exist\n" } } -------------------------------------------------------------------------------- ... ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 07:30:54 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.23 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-q645ss" for cluster "cluster-candidate-4-23-q645ss" Unexpected error: <fmt.wrapError \| 0xc00147edc0>: failed to create NodePool np-q645ss: failed waiting for nodepool="np-q645ss" for cluster "cluster-candidate-4-23-q645ss" in resourcegroup="rg-candidate-4-23-97lbmpnc95ml" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/e7c6ae1c-be71-48cb-b2ba-de56bb6c16d5 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/e7c6ae1c-be71-48cb-b2ba-de56bb6c16d5", "name": "e7c6ae1c-be71-48cb-b2ba-de56bb6c16d5", "status": "Failed", "startTime": "2026-10-02T08:22:25.175276447Z", "endTime": "2026-10-02T08:41:25.163899948Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine x4b4k7o1x7w4m8z-np-q645ss-zvxb8-b9h5q: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine x4b4k7o1x7w4m8z-np-q645ss-zvxb8-rcd4l: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine x4b4k7o1x7w4m8z-np-q645ss-zvxb8-b9h5q: Creating: virtualmachine creating or updating\nMachine x4b4k7o1x7w4m8z-np-q645ss-zvxb8-rcd4l: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-q645ss" for cluster "cluster-candidate-4-23-q645ss" Unexpected error: <fmt.wrapError \| 0xc00147edc0>: failed to create NodePool np-q645ss: failed waiting for nodepool="np-q645ss" for cluster "cluster-candidate-4-23-q645ss" in resourcegroup="rg-candidate-4-23-97lbmpnc95ml" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/e7c6ae1c-be71-48cb-b2ba-de56bb6c16d5 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/e7c6ae1c-be71-48cb-b2ba-de56bb6c16d5", "name": "e7c6ae1c-be71-48cb-b2ba-de56bb6c16d5", "status": "Failed", "startTime": "2026-10-02T08:22:25.175276447Z", "endTime": "2026-10-02T08:41:25.163899948Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine x4b4k7o1x7w4m8z-np-q645ss-zvxb8-b9h5q: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine x4b4k7o1x7w4m8z-np-q645ss-zvxb8-rcd4l: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine x4b4k7o1x7w4m8z-np-q645ss-zvxb8-b9h5q: Creating: virtualmachine creating or updating\nMachine x4b4k7o1x7w4m8z-np-q645ss-zvxb8-rcd4l: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; provider Microsoft.Compute 07:30:54 UTC · failed at e2e · test Customer should be able to create an HCP cluster with custom autoscaling and update the autoscaling configuration · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:139]: failed to create nodepool "np-1" for autoscaling cluster Unexpected error: <fmt.wrapError \| 0xc000ea0360>: failed to create NodePool np-1: failed waiting for nodepool="np-1" for cluster "autoscaling-hcp-cluster" in resourcegroup="autoscaling-cluster-5w2whf62mkf6" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/ffd107ed-298a-4008-9f25-801165f1c37e -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/ffd107ed-298a-4008-9f25-801165f1c37e", "name": "ffd107ed-298a-4008-9f25-801165f1c37e", "status": "Failed", "startTime": "2026-10-02T07:56:52.717139839Z", "endTime": "2026-10-02T08:15:58.568270167Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 1, want 2; node pool condition AllNodesHealthy is False: 1 of 2 machines are not healthy\nMachine v5b2u0b6b5e2c6j-np-1-rtdvc-2drvl: InspectionFailed: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/autoscaling-cluster-5w2whf62mkf6--managed/providers/Microsoft.Compute/virtualMachines/v5b2u0b6b5e2c6j-np-1-rtdvc-2drvl to exist\n; node pool condition AllMachinesReady is False: 1 of 2 machines are not ready\nMachine v5b2u0b6b5e2c6j-np-1-rtdvc-2drvl: ReadyUnknown: NodeHealthy: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/autoscaling-cluster-5w2whf62mkf6--managed/providers/Microsoft.Compute/virtualMachines/v5b2u0b6b5e2c6j-np-1-rtdvc-2drvl to exist\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:139]: failed to create nodepool "np-1" for autoscaling cluster Unexpected error: <fmt.wrapError \| 0xc000ea0360>: failed to create NodePool np-1: failed waiting for nodepool="np-1" for cluster "autoscaling-hcp-cluster" in resourcegroup="autoscaling-cluster-5w2whf62mkf6" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/ffd107ed-298a-4008-9f25-801165f1c37e -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/ffd107ed-298a-4008-9f25-801165f1c37e", "name": "ffd107ed-298a-4008-9f25-801165f1c37e", "status": "Failed", "startTime": "2026-10-02T07:56:52.717139839Z", "endTime": "2026-10-02T08:15:58.568270167Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 1, want 2; node pool condition AllNodesHealthy is False: 1 of 2 machines are not healthy\nMachine v5b2u0b6b5e2c6j-np-1-rtdvc-2drvl: InspectionFailed: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/autoscaling-cluster-5w2whf62mkf6--managed/providers/Microsoft.Compute/virtualMachines/v5b2u0b6b5e2c6j-np-1-rtdvc-2drvl to exist\n; node pool condition AllMachinesReady is False: 1 of 2 machines are not ready\nMachine v5b2u0b6b5e2c6j-np-1-rtdvc-2drvl: ReadyUnknown: NodeHealthy: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/autoscaling-cluster-5w2whf62mkf6--managed/providers/Microsoft.Compute/virtualMachines/v5b2u0b6b5e2c6j-np-1-rtdvc-2drvl to exist\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 07:30:54 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.22 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-kf44kx" for cluster "cluster-candidate-4-22-kf44kx" Unexpected error: <fmt.wrapError \| 0xc001670be0>: failed to create NodePool np-kf44kx: failed waiting for nodepool="np-kf44kx" for cluster "cluster-candidate-4-22-kf44kx" in resourcegroup="rg-candidate-4-22-p972hm7v767t" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/f5cc41ef-2098-4080-80d0-52d4a6c605b2 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/f5cc41ef-2098-4080-80d0-52d4a6c605b2", "name": "f5cc41ef-2098-4080-80d0-52d4a6c605b2", "status": "Failed", "startTime": "2026-10-02T08:24:18.844576711Z", "endTime": "2026-10-02T08:43:25.00034882Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine j5o2i4r2x6i1i6v-np-kf44kx-5lnfn-ff2wq: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine j5o2i4r2x6i1i6v-np-kf44kx-5lnfn-p7lf6: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine j5o2i4r2x6i1i6v-np-kf44kx-5lnfn-ff2wq: Creating: virtualmachine creating or updating\nMachine j5o2i4r2x6i1i6v-np-kf44kx-5lnfn-p7lf6: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-kf44kx" for cluster "cluster-candidate-4-22-kf44kx" Unexpected error: <fmt.wrapError \| 0xc001670be0>: failed to create NodePool np-kf44kx: failed waiting for nodepool="np-kf44kx" for cluster "cluster-candidate-4-22-kf44kx" in resourcegroup="rg-candidate-4-22-p972hm7v767t" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/f5cc41ef-2098-4080-80d0-52d4a6c605b2 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/f5cc41ef-2098-4080-80d0-52d4a6c605b2", "name": "f5cc41ef-2098-4080-80d0-52d4a6c605b2", "status": "Failed", "startTime": "2026-10-02T08:24:18.844576711Z", "endTime": "2026-10-02T08:43:25.00034882Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine j5o2i4r2x6i1i6v-np-kf44kx-5lnfn-ff2wq: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine j5o2i4r2x6i1i6v-np-kf44kx-5lnfn-p7lf6: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine j5o2i4r2x6i1i6v-np-kf44kx-5lnfn-ff2wq: Creating: virtualmachine creating or updating\nMachine j5o2i4r2x6i1i6v-np-kf44kx-5lnfn-p7lf6: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 07:30:54 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 5.1 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-qbwjq7" for cluster "cluster-candidate-5-1-qbwjq7" Unexpected error: <fmt.wrapError \| 0xc000f28640>: failed to create NodePool np-qbwjq7: failed waiting for nodepool="np-qbwjq7" for cluster "cluster-candidate-5-1-qbwjq7" in resourcegroup="rg-candidate-5-1-cxt8gqtcm56h" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/411adf6f-822a-4f98-bc7d-64851f61532c -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/411adf6f-822a-4f98-bc7d-64851f61532c", "name": "411adf6f-822a-4f98-bc7d-64851f61532c", "status": "Failed", "startTime": "2026-10-02T08:23:47.400908917Z", "endTime": "2026-10-02T08:42:55.012095583Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine x5o3w3y8k8n9n2p-np-qbwjq7-vv8c8-dhvsm: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine x5o3w3y8k8n9n2p-np-qbwjq7-vv8c8-zlljx: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine x5o3w3y8k8n9n2p-np-qbwjq7-vv8c8-dhvsm: Creating: virtualmachine creating or updating\nMachine x5o3w3y8k8n9n2p-np-qbwjq7-vv8c8-zlljx: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-qbwjq7" for cluster "cluster-candidate-5-1-qbwjq7" Unexpected error: <fmt.wrapError \| 0xc000f28640>: failed to create NodePool np-qbwjq7: failed waiting for nodepool="np-qbwjq7" for cluster "cluster-candidate-5-1-qbwjq7" in resourcegroup="rg-candidate-5-1-cxt8gqtcm56h" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/411adf6f-822a-4f98-bc7d-64851f61532c -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/411adf6f-822a-4f98-bc7d-64851f61532c", "name": "411adf6f-822a-4f98-bc7d-64851f61532c", "status": "Failed", "startTime": "2026-10-02T08:23:47.400908917Z", "endTime": "2026-10-02T08:42:55.012095583Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine x5o3w3y8k8n9n2p-np-qbwjq7-vv8c8-dhvsm: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine x5o3w3y8k8n9n2p-np-qbwjq7-vv8c8-zlljx: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine x5o3w3y8k8n9n2p-np-qbwjq7-vv8c8-dhvsm: Creating: virtualmachine creating or updating\nMachine x5o3w3y8k8n9n2p-np-qbwjq7-vv8c8-zlljx: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want >= <count> (autoscaling min); node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 07:30:54 UTC · failed at e2e · test Customer should be able to create a cluster and node pools with aggregated advanced features · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:404]: failed to verify cilium is running and nodes are Ready for cluster "agg-cluster" Unexpected error: <errors.joinError \| 0xc000e761b0>: VerifyNodesReady failed: no nodes found in the cluster VerifyCiliumOperational failed: not all pods in kube-system namespace are running: no cilium pods found in kube-system namespace failed to create NodePool agg-np-a: failed waiting for nodepool="agg-np-a" for cluster "agg-cluster" in resourcegroup="feature-aggregation-dwl8n7k5d292" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0cd68833-1cd8-4477-9f36-56a0fe7d00ba -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0cd68833-1cd8-4477-9f36-56a0fe7d00ba", "name": "0cd68833-1cd8-4477-9f36-56a0fe7d00ba", "status": "Failed", "startTime": "2026-10-02T07:58:36.615555909Z", "endTime": "2026-10-02T08:17:38.532193005Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want \u003e= 2 (autoscaling min); node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine agg-cluster-agg-np-a-c9phq-b96vq: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine agg-cluster-agg-np-a-c9phq-gw2rm: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine agg-cluster-agg-np-a-c9phq-b96vq: Creating: virtualmachine creating or updating\nMachine agg-cluster-agg-np-a-c9phq-gw2rm: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:404]: failed to verify cilium is running and nodes are Ready for cluster "agg-cluster" Unexpected error: <errors.joinError \| 0xc000e761b0>: VerifyNodesReady failed: no nodes found in the cluster VerifyCiliumOperational failed: not all pods in kube-system namespace are running: no cilium pods found in kube-system namespace failed to create NodePool agg-np-a: failed waiting for nodepool="agg-np-a" for cluster "agg-cluster" in resourcegroup="feature-aggregation-dwl8n7k5d292" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0cd68833-1cd8-4477-9f36-56a0fe7d00ba -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0cd68833-1cd8-4477-9f36-56a0fe7d00ba", "name": "0cd68833-1cd8-4477-9f36-56a0fe7d00ba", "status": "Failed", "startTime": "2026-10-02T07:58:36.615555909Z", "endTime": "2026-10-02T08:17:38.532193005Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want \u003e= 2 (autoscaling min); node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine agg-cluster-agg-np-a-c9phq-b96vq: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine agg-cluster-agg-np-a-c9phq-gw2rm: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine agg-cluster-agg-np-a-c9phq-b96vq: Creating: virtualmachine creating or updating\nMachine agg-cluster-agg-np-a-c9phq-gw2rm: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 07:30:54 UTC · failed at e2e · test Create HCPOpenShiftCluster with Private KeyVault should create a cluster with private keyvault using v20251223preview API · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:138]: failed to create node pool "np-1" for private keyvault cluster "private-kv-cluster" Unexpected error: <fmt.wrapError \| 0xc0008aece0>: failed to create NodePool np-1: failed waiting for nodepool="np-1" for cluster "private-kv-cluster" in resourcegroup="private-keyvault-plwp75qbhvcq" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0d68804b-aa1f-441d-9c44-b697ee5be0e1 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0d68804b-aa1f-441d-9c44-b697ee5be0e1", "name": "0d68804b-aa1f-441d-9c44-b697ee5be0e1", "status": "Failed", "startTime": "2026-10-02T07:58:27.095320904Z", "endTime": "2026-10-02T08:17:28.456252233Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine v5t1z8u9w4x9o4c-np-1-qj5gs-j6stv: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine v5t1z8u9w4x9o4c-np-1-qj5gs-jcmz5: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine v5t1z8u9w4x9o4c-np-1-qj5gs-j6stv: Creating: virtualmachine creating or updating\nMachine v5t1z8u9w4x9o4c-np-1-qj5gs-jcmz5: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:138]: failed to create node pool "np-1" for private keyvault cluster "private-kv-cluster" Unexpected error: <fmt.wrapError \| 0xc0008aece0>: failed to create NodePool np-1: failed waiting for nodepool="np-1" for cluster "private-kv-cluster" in resourcegroup="private-keyvault-plwp75qbhvcq" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0d68804b-aa1f-441d-9c44-b697ee5be0e1 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0d68804b-aa1f-441d-9c44-b697ee5be0e1", "name": "0d68804b-aa1f-441d-9c44-b697ee5be0e1", "status": "Failed", "startTime": "2026-10-02T07:58:27.095320904Z", "endTime": "2026-10-02T08:17:28.456252233Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine v5t1z8u9w4x9o4c-np-1-qj5gs-j6stv: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine v5t1z8u9w4x9o4c-np-1-qj5gs-jcmz5: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine v5t1z8u9w4x9o4c-np-1-qj5gs-j6stv: Creating: virtualmachine creating or updating\nMachine v5t1z8u9w4x9o4c-np-1-qj5gs-jcmz5: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 07:30:54 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.21 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-65j846" for cluster "cluster-candidate-4-21-65j846" Unexpected error: <fmt.wrapError \| 0xc000dd8540>: failed to create NodePool np-65j846: failed waiting for nodepool="np-65j846" for cluster "cluster-candidate-4-21-65j846" in resourcegroup="rg-candidate-4-21-8dj7spdjvcps" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/d4b395e6-2b88-4e54-814b-bcbf478cdffc -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/d4b395e6-2b88-4e54-814b-bcbf478cdffc", "name": "d4b395e6-2b88-4e54-814b-bcbf478cdffc", "status": "Failed", "startTime": "2026-10-02T08:23:11.897579976Z", "endTime": "2026-10-02T08:42:15.234899325Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine p4r1l1y9v2e2n2b-np-65j846-2t8dj-dczz6: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine p4r1l1y9v2e2n2b-np-65j846-2t8dj-hp4tn: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine p4r1l1y9v2e2n2b-np-65j846-2t8dj-dczz6: Creating: virtualmachine creating or updating\nMachine p4r1l1y9v2e2n2b-np-65j846-2t8dj-hp4tn: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-65j846" for cluster "cluster-candidate-4-21-65j846" Unexpected error: <fmt.wrapError \| 0xc000dd8540>: failed to create NodePool np-65j846: failed waiting for nodepool="np-65j846" for cluster "cluster-candidate-4-21-65j846" in resourcegroup="rg-candidate-4-21-8dj7spdjvcps" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/d4b395e6-2b88-4e54-814b-bcbf478cdffc -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/d4b395e6-2b88-4e54-814b-bcbf478cdffc", "name": "d4b395e6-2b88-4e54-814b-bcbf478cdffc", "status": "Failed", "startTime": "2026-10-02T08:23:11.897579976Z", "endTime": "2026-10-02T08:42:15.234899325Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine p4r1l1y9v2e2n2b-np-65j846-2t8dj-dczz6: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine p4r1l1y9v2e2n2b-np-65j846-2t8dj-hp4tn: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine p4r1l1y9v2e2n2b-np-65j846-2t8dj-dczz6: Creating: virtualmachine creating or updating\nMachine p4r1l1y9v2e2n2b-np-65j846-2t8dj-hp4tn: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 07:30:54 UTC · failed at e2e · test Customer should be able to create a no-CNI private cluster with a private key vault, a nodepool and install cilium CNI successfully · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:193]: failed to verify nodes are Ready with Cilium CNI for cluster "cilium-cluster" Unexpected error: <errors.joinError \| 0xc002194198>: VerifyNodesReady failed: no nodes found in the cluster VerifyCiliumOperational failed: not all pods in kube-system namespace are running: failed to list pods in kube-system namespace: client rate limiter Wait returned an error: context deadline exceeded failed to create NodePool cilium-np: failed waiting for nodepool="cilium-np" for cluster "cilium-cluster" in resourcegroup="complex-cilium-kv-kkklwf69d6rr" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/f6596056-a431-4061-9fcc-922385eb88b7 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/f6596056-a431-4061-9fcc-922385eb88b7", "name": "f6596056-a431-4061-9fcc-922385eb88b7", "status": "Failed", "startTime": "2026-10-02T07:58:26.598790316Z", "endTime": "2026-10-02T08:17:28.2532697Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine cilium-cluster-cilium-np-t9npj-5lpcd: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine cilium-cluster-cilium-np-t9npj-z8x9w: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine cilium-cluster-cilium-np-t9npj-5lpcd: Creating: virtualmachine creating or updating\nMachine cilium-cluster-cilium-np-t9npj-z8x9w: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:193]: failed to verify nodes are Ready with Cilium CNI for cluster "cilium-cluster" Unexpected error: <errors.joinError \| 0xc002194198>: VerifyNodesReady failed: no nodes found in the cluster VerifyCiliumOperational failed: not all pods in kube-system namespace are running: failed to list pods in kube-system namespace: client rate limiter Wait returned an error: context deadline exceeded failed to create NodePool cilium-np: failed waiting for nodepool="cilium-np" for cluster "cilium-cluster" in resourcegroup="complex-cilium-kv-kkklwf69d6rr" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/f6596056-a431-4061-9fcc-922385eb88b7 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/f6596056-a431-4061-9fcc-922385eb88b7", "name": "f6596056-a431-4061-9fcc-922385eb88b7", "status": "Failed", "startTime": "2026-10-02T07:58:26.598790316Z", "endTime": "2026-10-02T08:17:28.2532697Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine cilium-cluster-cilium-np-t9npj-5lpcd: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine cilium-cluster-cilium-np-t9npj-z8x9w: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine cilium-cluster-cilium-np-t9npj-5lpcd: Creating: virtualmachine creating or updating\nMachine cilium-cluster-cilium-np-t9npj-z8x9w: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 07:30:54 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 5.0 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-96gwgz" for cluster "cluster-candidate-5-0-96gwgz" Unexpected error: <fmt.wrapError \| 0xc000cf2260>: failed to create NodePool np-96gwgz: failed waiting for nodepool="np-96gwgz" for cluster "cluster-candidate-5-0-96gwgz" in resourcegroup="rg-candidate-5-0-84kwhzj8r7w4" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/c7df18c5-1f74-4f58-93d8-951ddd66b332 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/c7df18c5-1f74-4f58-93d8-951ddd66b332", "name": "c7df18c5-1f74-4f58-93d8-951ddd66b332", "status": "Failed", "startTime": "2026-10-02T08:23:19.198498666Z", "endTime": "2026-10-02T08:42:25.18491611Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine m9p3c0m3i2a9q0y-np-96gwgz-x5zk7-4frj5: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine m9p3c0m3i2a9q0y-np-96gwgz-x5zk7-d5fmk: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine m9p3c0m3i2a9q0y-np-96gwgz-x5zk7-4frj5: Creating: virtualmachine creating or updating\nMachine m9p3c0m3i2a9q0y-np-96gwgz-x5zk7-d5fmk: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-96gwgz" for cluster "cluster-candidate-5-0-96gwgz" Unexpected error: <fmt.wrapError \| 0xc000cf2260>: failed to create NodePool np-96gwgz: failed waiting for nodepool="np-96gwgz" for cluster "cluster-candidate-5-0-96gwgz" in resourcegroup="rg-candidate-5-0-84kwhzj8r7w4" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/c7df18c5-1f74-4f58-93d8-951ddd66b332 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/c7df18c5-1f74-4f58-93d8-951ddd66b332", "name": "c7df18c5-1f74-4f58-93d8-951ddd66b332", "status": "Failed", "startTime": "2026-10-02T08:23:19.198498666Z", "endTime": "2026-10-02T08:42:25.18491611Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine m9p3c0m3i2a9q0y-np-96gwgz-x5zk7-4frj5: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine m9p3c0m3i2a9q0y-np-96gwgz-x5zk7-d5fmk: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine m9p3c0m3i2a9q0y-np-96gwgz-x5zk7-4frj5: Creating: virtualmachine creating or updating\nMachine m9p3c0m3i2a9q0y-np-96gwgz-x5zk7-d5fmk: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 07:30:54 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.20 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-tgr2wf" for cluster "cluster-candidate-4-20-tgr2wf" Unexpected error: <fmt.wrapError \| 0xc000e1d940>: failed to create NodePool np-tgr2wf: failed waiting for nodepool="np-tgr2wf" for cluster "cluster-candidate-4-20-tgr2wf" in resourcegroup="rg-candidate-4-20-r2lrfp2tbsqc" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0734784e-738a-4b64-ad9c-e373ea6e9682 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0734784e-738a-4b64-ad9c-e373ea6e9682", "name": "0734784e-738a-4b64-ad9c-e373ea6e9682", "status": "Failed", "startTime": "2026-10-02T08:23:20.483388473Z", "endTime": "2026-10-02T08:42:25.166235902Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine x6z3k1g0g0y1m3q-np-tgr2wf-4vbp4-jmm8m: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine x6z3k1g0g0y1m3q-np-tgr2wf-4vbp4-k7sxl: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine x6z3k1g0g0y1m3q-np-tgr2wf-4vbp4-jmm8m: Creating: virtualmachine creating or updating\nMachine x6z3k1g0g0y1m3q-np-tgr2wf-4vbp4-k7sxl: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-tgr2wf" for cluster "cluster-candidate-4-20-tgr2wf" Unexpected error: <fmt.wrapError \| 0xc000e1d940>: failed to create NodePool np-tgr2wf: failed waiting for nodepool="np-tgr2wf" for cluster "cluster-candidate-4-20-tgr2wf" in resourcegroup="rg-candidate-4-20-r2lrfp2tbsqc" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0734784e-738a-4b64-ad9c-e373ea6e9682 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0734784e-738a-4b64-ad9c-e373ea6e9682", "name": "0734784e-738a-4b64-ad9c-e373ea6e9682", "status": "Failed", "startTime": "2026-10-02T08:23:20.483388473Z", "endTime": "2026-10-02T08:42:25.166235902Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine x6z3k1g0g0y1m3q-np-tgr2wf-4vbp4-jmm8m: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine x6z3k1g0g0y1m3q-np-tgr2wf-4vbp4-k7sxl: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine x6z3k1g0g0y1m3q-np-tgr2wf-4vbp4-jmm8m: Creating: virtualmachine creating or updating\nMachine x6z3k1g0g0y1m3q-np-tgr2wf-4vbp4-k7sxl: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred
02:00:29	3h 11m 28s	periodic-ci-Azure-ARO-HCP-main-periodic-stage-e2e-parallel post-goodmerged PR No additional run metadata captured	uksouth	e2e	failed	n/a	11	e2e (11) failed to create nodepool <nodepool>: failed waiting for nodepool="<nodepool>" for cluster "<cluster>" in resourcegroup="<resource-group>" to finish creating: GET <url> 02:00:29 UTC · failed at e2e · test Customer should be able to delete a nodepool · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_delete.go:128]: failed to create node pools Expected success, but got an error: <fmt.wrapError \| 0xc001276700>: failed to create NodePool np-deletable: failed waiting for nodepool="np-deletable" for cluster "np-delete-hcp-cluster" in resourcegroup="nodepool-delete-5rf62rf4p9wv" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/d53fa45c-a9c4-43a3-b87a-c16df786eeb4 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/d53fa45c-a9c4-43a3-b87a-c16df786eeb4", "name": "d53fa45c-a9c4-43a3-b87a-c16df786eeb4", "status": "Failed", "startTime": "2026-10-02T04:09:42.171003862Z", "endTime": "2026-10-02T04:28:46.827990048Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine y8h6g1h7a1t1v0i-np-deletable-c6pz2-n4l4p: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine y8h6g1h7a1t1v0i-np-deletable-c6pz2-qtmzk: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine y8h6g1h7a1t1v0i-np-deletable-c6pz2-n4l4p: Creating: virtualmachine creating or updating\nMachine y8h6g1h7a1t1v0i-np-deletable-c6pz2-qtmzk: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_delete.go:128]: failed to create node pools Expected success, but got an error: <fmt.wrapError \| 0xc001276700>: failed to create NodePool np-deletable: failed waiting for nodepool="np-deletable" for cluster "np-delete-hcp-cluster" in resourcegroup="nodepool-delete-5rf62rf4p9wv" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/d53fa45c-a9c4-43a3-b87a-c16df786eeb4 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/d53fa45c-a9c4-43a3-b87a-c16df786eeb4", "name": "d53fa45c-a9c4-43a3-b87a-c16df786eeb4", "status": "Failed", "startTime": "2026-10-02T04:09:42.171003862Z", "endTime": "2026-10-02T04:28:46.827990048Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine y8h6g1h7a1t1v0i-np-deletable-c6pz2-n4l4p: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine y8h6g1h7a1t1v0i-np-deletable-c6pz2-qtmzk: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine y8h6g1h7a1t1v0i-np-deletable-c6pz2-n4l4p: Creating: virtualmachine creating or updating\nMachine y8h6g1h7a1t1v0i-np-deletable-c6pz2-qtmzk: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 02:00:29 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 5.1 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-9rfwql" for cluster "cluster-candidate-5-1-9rfwql" Unexpected error: <fmt.wrapError \| 0xc001103040>: failed to create NodePool np-9rfwql: failed waiting for nodepool="np-9rfwql" for cluster "cluster-candidate-5-1-9rfwql" in resourcegroup="rg-candidate-5-1-jznzltj2rxxg" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/34fd61fa-5104-45ad-bf92-0d32ab5a44e5 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/34fd61fa-5104-45ad-bf92-0d32ab5a44e5", "name": "34fd61fa-5104-45ad-bf92-0d32ab5a44e5", "status": "Failed", "startTime": "2026-10-02T03:44:17.773093204Z", "endTime": "2026-10-02T04:03:23.679415683Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine s2j2b3a4i7k6f9u-np-9rfwql-2xh25-9ptnl: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine s2j2b3a4i7k6f9u-np-9rfwql-2xh25-gscjf: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine s2j2b3a4i7k6f9u-np-9rfwql-2xh25-9ptnl: Creating: virtualmachine creating or updating\nMachine s2j2b3a4i7k6f9u-np-9rfwql-2xh25-gscjf: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-9rfwql" for cluster "cluster-candidate-5-1-9rfwql" Unexpected error: <fmt.wrapError \| 0xc001103040>: failed to create NodePool np-9rfwql: failed waiting for nodepool="np-9rfwql" for cluster "cluster-candidate-5-1-9rfwql" in resourcegroup="rg-candidate-5-1-jznzltj2rxxg" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/34fd61fa-5104-45ad-bf92-0d32ab5a44e5 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/34fd61fa-5104-45ad-bf92-0d32ab5a44e5", "name": "34fd61fa-5104-45ad-bf92-0d32ab5a44e5", "status": "Failed", "startTime": "2026-10-02T03:44:17.773093204Z", "endTime": "2026-10-02T04:03:23.679415683Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine s2j2b3a4i7k6f9u-np-9rfwql-2xh25-9ptnl: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine s2j2b3a4i7k6f9u-np-9rfwql-2xh25-gscjf: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine s2j2b3a4i7k6f9u-np-9rfwql-2xh25-9ptnl: Creating: virtualmachine creating or updating\nMachine s2j2b3a4i7k6f9u-np-9rfwql-2xh25-gscjf: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want >= <count> (autoscaling min); node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 02:00:29 UTC · failed at e2e · test Customer should be able to create a cluster and node pools with aggregated advanced features · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:404]: failed to verify cilium is running and nodes are Ready for cluster "agg-cluster" Unexpected error: <errors.joinError \| 0xc002984078>: VerifyNodesReady failed: no nodes found in the cluster VerifyCiliumOperational failed: not all pods in kube-system namespace are running: failed to list pods in kube-system namespace: client rate limiter Wait returned an error: context deadline exceeded failed to create NodePool agg-np-a: failed waiting for nodepool="agg-np-a" for cluster "agg-cluster" in resourcegroup="feature-aggregation-4httpsjkvppc" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/27c9b728-c34e-4afa-93f4-b5118810ce82 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/27c9b728-c34e-4afa-93f4-b5118810ce82", "name": "27c9b728-c34e-4afa-93f4-b5118810ce82", "status": "Failed", "startTime": "2026-10-02T03:19:57.766414352Z", "endTime": "2026-10-02T03:39:03.235272547Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want \u003e= 2 (autoscaling min); node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine agg-cluster-agg-np-a-lszrc-8znc6: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine agg-cluster-agg-np-a-lszrc-nxxm5: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine agg-cluster-agg-np-a-lszrc-8znc6: Creating: virtualmachine creating or updating\nMachine agg-cluster-agg-np-a-lszrc-nxxm5: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:404]: failed to verify cilium is running and nodes are Ready for cluster "agg-cluster" Unexpected error: <errors.joinError \| 0xc002984078>: VerifyNodesReady failed: no nodes found in the cluster VerifyCiliumOperational failed: not all pods in kube-system namespace are running: failed to list pods in kube-system namespace: client rate limiter Wait returned an error: context deadline exceeded failed to create NodePool agg-np-a: failed waiting for nodepool="agg-np-a" for cluster "agg-cluster" in resourcegroup="feature-aggregation-4httpsjkvppc" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/27c9b728-c34e-4afa-93f4-b5118810ce82 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/27c9b728-c34e-4afa-93f4-b5118810ce82", "name": "27c9b728-c34e-4afa-93f4-b5118810ce82", "status": "Failed", "startTime": "2026-10-02T03:19:57.766414352Z", "endTime": "2026-10-02T03:39:03.235272547Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want \u003e= 2 (autoscaling min); node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine agg-cluster-agg-np-a-lszrc-8znc6: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine agg-cluster-agg-np-a-lszrc-nxxm5: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine agg-cluster-agg-np-a-lszrc-8znc6: Creating: virtualmachine creating or updating\nMachine agg-cluster-agg-np-a-lszrc-nxxm5: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 02:00:29 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.22 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-fs2tl4" for cluster "cluster-candidate-4-22-fs2tl4" Unexpected error: <fmt.wrapError \| 0xc000dde140>: failed to create NodePool np-fs2tl4: failed waiting for nodepool="np-fs2tl4" for cluster "cluster-candidate-4-22-fs2tl4" in resourcegroup="rg-candidate-4-22-jzw6chr4d7m4" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/af47af07-70e2-47bd-9306-0c7f478d186e -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/af47af07-70e2-47bd-9306-0c7f478d186e", "name": "af47af07-70e2-47bd-9306-0c7f478d186e", "status": "Failed", "startTime": "2026-10-02T03:39:14.039068534Z", "endTime": "2026-10-02T03:58:23.505666093Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine g1v3q3i3v9q4u8m-np-fs2tl4-ttfnq-hwrpc: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine g1v3q3i3v9q4u8m-np-fs2tl4-ttfnq-mh58m: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine g1v3q3i3v9q4u8m-np-fs2tl4-ttfnq-hwrpc: Creating: virtualmachine creating or updating\nMachine g1v3q3i3v9q4u8m-np-fs2tl4-ttfnq-mh58m: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-fs2tl4" for cluster "cluster-candidate-4-22-fs2tl4" Unexpected error: <fmt.wrapError \| 0xc000dde140>: failed to create NodePool np-fs2tl4: failed waiting for nodepool="np-fs2tl4" for cluster "cluster-candidate-4-22-fs2tl4" in resourcegroup="rg-candidate-4-22-jzw6chr4d7m4" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/af47af07-70e2-47bd-9306-0c7f478d186e -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/af47af07-70e2-47bd-9306-0c7f478d186e", "name": "af47af07-70e2-47bd-9306-0c7f478d186e", "status": "Failed", "startTime": "2026-10-02T03:39:14.039068534Z", "endTime": "2026-10-02T03:58:23.505666093Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine g1v3q3i3v9q4u8m-np-fs2tl4-ttfnq-hwrpc: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine g1v3q3i3v9q4u8m-np-fs2tl4-ttfnq-mh58m: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine g1v3q3i3v9q4u8m-np-fs2tl4-ttfnq-hwrpc: Creating: virtualmachine creating or updating\nMachine g1v3q3i3v9q4u8m-np-fs2tl4-ttfnq-mh58m: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 02:00:29 UTC · failed at e2e · test Create HCPOpenShiftCluster with Private KeyVault should create a cluster with private keyvault using v20251223preview API · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:138]: failed to create node pool "np-1" for private keyvault cluster "private-kv-cluster" Unexpected error: <fmt.wrapError \| 0xc000943f60>: failed to create NodePool np-1: failed waiting for nodepool="np-1" for cluster "private-kv-cluster" in resourcegroup="private-keyvault-6sjr57crkxhh" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/3a2572ed-7c03-46c2-98bb-89a19d5f777a -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/3a2572ed-7c03-46c2-98bb-89a19d5f777a", "name": "3a2572ed-7c03-46c2-98bb-89a19d5f777a", "status": "Failed", "startTime": "2026-10-02T03:19:47.10992898Z", "endTime": "2026-10-02T03:38:53.263922354Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine d3m7o6a3u1p6z3i-np-1-vf5g4-9k9hg: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine d3m7o6a3u1p6z3i-np-1-vf5g4-fjdfs: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine d3m7o6a3u1p6z3i-np-1-vf5g4-9k9hg: Creating: virtualmachine creating or updating\nMachine d3m7o6a3u1p6z3i-np-1-vf5g4-fjdfs: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:138]: failed to create node pool "np-1" for private keyvault cluster "private-kv-cluster" Unexpected error: <fmt.wrapError \| 0xc000943f60>: failed to create NodePool np-1: failed waiting for nodepool="np-1" for cluster "private-kv-cluster" in resourcegroup="private-keyvault-6sjr57crkxhh" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/3a2572ed-7c03-46c2-98bb-89a19d5f777a -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/3a2572ed-7c03-46c2-98bb-89a19d5f777a", "name": "3a2572ed-7c03-46c2-98bb-89a19d5f777a", "status": "Failed", "startTime": "2026-10-02T03:19:47.10992898Z", "endTime": "2026-10-02T03:38:53.263922354Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine d3m7o6a3u1p6z3i-np-1-vf5g4-9k9hg: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine d3m7o6a3u1p6z3i-np-1-vf5g4-fjdfs: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine d3m7o6a3u1p6z3i-np-1-vf5g4-9k9hg: Creating: virtualmachine creating or updating\nMachine d3m7o6a3u1p6z3i-np-1-vf5g4-fjdfs: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 02:00:29 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.23 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-2rhqzm" for cluster "cluster-candidate-4-23-2rhqzm" Unexpected error: <fmt.wrapError \| 0xc000f90900>: failed to create NodePool np-2rhqzm: failed waiting for nodepool="np-2rhqzm" for cluster "cluster-candidate-4-23-2rhqzm" in resourcegroup="rg-candidate-4-23-hvftzsslsv5w" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/83438069-4881-4a7c-99c3-04167cb9898c -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/83438069-4881-4a7c-99c3-04167cb9898c", "name": "83438069-4881-4a7c-99c3-04167cb9898c", "status": "Failed", "startTime": "2026-10-02T03:40:49.053626001Z", "endTime": "2026-10-02T03:59:53.321378994Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine o2x4w0t0l5t3o4o-np-2rhqzm-ghd9l-jdv5b: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine o2x4w0t0l5t3o4o-np-2rhqzm-ghd9l-r825r: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine o2x4w0t0l5t3o4o-np-2rhqzm-ghd9l-jdv5b: Creating: virtualmachine creating or updating\nMachine o2x4w0t0l5t3o4o-np-2rhqzm-ghd9l-r825r: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-2rhqzm" for cluster "cluster-candidate-4-23-2rhqzm" Unexpected error: <fmt.wrapError \| 0xc000f90900>: failed to create NodePool np-2rhqzm: failed waiting for nodepool="np-2rhqzm" for cluster "cluster-candidate-4-23-2rhqzm" in resourcegroup="rg-candidate-4-23-hvftzsslsv5w" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/83438069-4881-4a7c-99c3-04167cb9898c -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/83438069-4881-4a7c-99c3-04167cb9898c", "name": "83438069-4881-4a7c-99c3-04167cb9898c", "status": "Failed", "startTime": "2026-10-02T03:40:49.053626001Z", "endTime": "2026-10-02T03:59:53.321378994Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine o2x4w0t0l5t3o4o-np-2rhqzm-ghd9l-jdv5b: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine o2x4w0t0l5t3o4o-np-2rhqzm-ghd9l-r825r: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine o2x4w0t0l5t3o4o-np-2rhqzm-ghd9l-jdv5b: Creating: virtualmachine creating or updating\nMachine o2x4w0t0l5t3o4o-np-2rhqzm-ghd9l-r825r: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 02:00:29 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.21 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-lfrdtx" for cluster "cluster-candidate-4-21-lfrdtx" Unexpected error: <fmt.wrapError \| 0xc000a86ae0>: failed to create NodePool np-lfrdtx: failed waiting for nodepool="np-lfrdtx" for cluster "cluster-candidate-4-21-lfrdtx" in resourcegroup="rg-candidate-4-21-v7j8q5wv69rf" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/a8487946-6d32-4b69-bbc8-793ffc91ce5f -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/a8487946-6d32-4b69-bbc8-793ffc91ce5f", "name": "a8487946-6d32-4b69-bbc8-793ffc91ce5f", "status": "Failed", "startTime": "2026-10-02T03:40:32.227275462Z", "endTime": "2026-10-02T03:59:33.308103816Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine z7f1i8t4o9f5w4c-np-lfrdtx-fqngr-58v7d: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine z7f1i8t4o9f5w4c-np-lfrdtx-fqngr-5gbq8: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine z7f1i8t4o9f5w4c-np-lfrdtx-fqngr-58v7d: Creating: virtualmachine creating or updating\nMachine z7f1i8t4o9f5w4c-np-lfrdtx-fqngr-5gbq8: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-lfrdtx" for cluster "cluster-candidate-4-21-lfrdtx" Unexpected error: <fmt.wrapError \| 0xc000a86ae0>: failed to create NodePool np-lfrdtx: failed waiting for nodepool="np-lfrdtx" for cluster "cluster-candidate-4-21-lfrdtx" in resourcegroup="rg-candidate-4-21-v7j8q5wv69rf" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/a8487946-6d32-4b69-bbc8-793ffc91ce5f -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/a8487946-6d32-4b69-bbc8-793ffc91ce5f", "name": "a8487946-6d32-4b69-bbc8-793ffc91ce5f", "status": "Failed", "startTime": "2026-10-02T03:40:32.227275462Z", "endTime": "2026-10-02T03:59:33.308103816Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine z7f1i8t4o9f5w4c-np-lfrdtx-fqngr-58v7d: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine z7f1i8t4o9f5w4c-np-lfrdtx-fqngr-5gbq8: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine z7f1i8t4o9f5w4c-np-lfrdtx-fqngr-58v7d: Creating: virtualmachine creating or updating\nMachine z7f1i8t4o9f5w4c-np-lfrdtx-fqngr-5gbq8: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 02:00:29 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.20 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-s4kd9s" for cluster "cluster-candidate-4-20-s4kd9s" Unexpected error: <fmt.wrapError \| 0xc000b0b4a0>: failed to create NodePool np-s4kd9s: failed waiting for nodepool="np-s4kd9s" for cluster "cluster-candidate-4-20-s4kd9s" in resourcegroup="rg-candidate-4-20-zlm28jv4jtcs" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/7e5e7aaa-eb3f-4153-b0cf-40dab5c32152 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/7e5e7aaa-eb3f-4153-b0cf-40dab5c32152", "name": "7e5e7aaa-eb3f-4153-b0cf-40dab5c32152", "status": "Failed", "startTime": "2026-10-02T03:38:55.251415601Z", "endTime": "2026-10-02T03:58:03.48228814Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine j1u6v8i1i7q3s1b-np-s4kd9s-nmqgx-mn8wj: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine j1u6v8i1i7q3s1b-np-s4kd9s-nmqgx-st4n7: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine j1u6v8i1i7q3s1b-np-s4kd9s-nmqgx-mn8wj: Creating: virtualmachine creating or updating\nMachine j1u6v8i1i7q3s1b-np-s4kd9s-nmqgx-st4n7: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-s4kd9s" for cluster "cluster-candidate-4-20-s4kd9s" Unexpected error: <fmt.wrapError \| 0xc000b0b4a0>: failed to create NodePool np-s4kd9s: failed waiting for nodepool="np-s4kd9s" for cluster "cluster-candidate-4-20-s4kd9s" in resourcegroup="rg-candidate-4-20-zlm28jv4jtcs" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/7e5e7aaa-eb3f-4153-b0cf-40dab5c32152 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/7e5e7aaa-eb3f-4153-b0cf-40dab5c32152", "name": "7e5e7aaa-eb3f-4153-b0cf-40dab5c32152", "status": "Failed", "startTime": "2026-10-02T03:38:55.251415601Z", "endTime": "2026-10-02T03:58:03.48228814Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine j1u6v8i1i7q3s1b-np-s4kd9s-nmqgx-mn8wj: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine j1u6v8i1i7q3s1b-np-s4kd9s-nmqgx-st4n7: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine j1u6v8i1i7q3s1b-np-s4kd9s-nmqgx-mn8wj: Creating: virtualmachine creating or updating\nMachine j1u6v8i1i7q3s1b-np-s4kd9s-nmqgx-st4n7: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 02:00:29 UTC · failed at e2e · test Customer should create a cluster with private KAS and verify API server is only reachable from VNet while ingress remains public · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:159]: failed to create node pool "np-1" for private KAS cluster "private-kas" Unexpected error: <fmt.wrapError \| 0xc00064c1e0>: failed to create NodePool np-1: failed waiting for nodepool="np-1" for cluster "private-kas" in resourcegroup="private-kas-gs752pln4txj" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/9d6e4c14-f5a6-4fd9-881e-2a42f87e3555 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/9d6e4c14-f5a6-4fd9-881e-2a42f87e3555", "name": "9d6e4c14-f5a6-4fd9-881e-2a42f87e3555", "status": "Failed", "startTime": "2026-10-02T03:18:04.846590038Z", "endTime": "2026-10-02T03:37:13.231851998Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 1; node pool condition AllNodesHealthy is False: 1 of 1 machines are not healthy\nMachine private-kas-np-1-5dbrz-khxf4: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 1 of 1 machines are not ready\nMachine private-kas-np-1-5dbrz-khxf4: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:159]: failed to create node pool "np-1" for private KAS cluster "private-kas" Unexpected error: <fmt.wrapError \| 0xc00064c1e0>: failed to create NodePool np-1: failed waiting for nodepool="np-1" for cluster "private-kas" in resourcegroup="private-kas-gs752pln4txj" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/9d6e4c14-f5a6-4fd9-881e-2a42f87e3555 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/9d6e4c14-f5a6-4fd9-881e-2a42f87e3555", "name": "9d6e4c14-f5a6-4fd9-881e-2a42f87e3555", "status": "Failed", "startTime": "2026-10-02T03:18:04.846590038Z", "endTime": "2026-10-02T03:37:13.231851998Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 1; node pool condition AllNodesHealthy is False: 1 of 1 machines are not healthy\nMachine private-kas-np-1-5dbrz-khxf4: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 1 of 1 machines are not ready\nMachine private-kas-np-1-5dbrz-khxf4: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 02:00:29 UTC · failed at e2e · test Customer should be able to create a no-CNI private cluster with a private key vault, a nodepool and install cilium CNI successfully · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:193]: failed to verify nodes are Ready with Cilium CNI for cluster "cilium-cluster" Unexpected error: <errors.joinError \| 0xc0011d4048>: VerifyNodesReady failed: no nodes found in the cluster VerifyCiliumOperational failed: not all pods in kube-system namespace are running: failed to list pods in kube-system namespace: client rate limiter Wait returned an error: context deadline exceeded failed to create NodePool cilium-np: failed waiting for nodepool="cilium-np" for cluster "cilium-cluster" in resourcegroup="complex-cilium-kv-rrrmbd9jmmdl" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/6d52358f-8309-4bc7-af9f-a5884f9307e0 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/6d52358f-8309-4bc7-af9f-a5884f9307e0", "name": "6d52358f-8309-4bc7-af9f-a5884f9307e0", "status": "Failed", "startTime": "2026-10-02T03:16:58.467567062Z", "endTime": "2026-10-02T03:36:03.226032745Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine cilium-cluster-cilium-np-sqrfn-pjdbq: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine cilium-cluster-cilium-np-sqrfn-vtbbr: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine cilium-cluster-cilium-np-sqrfn-pjdbq: Creating: virtualmachine creating or updating\nMachine cilium-cluster-cilium-np-sqrfn-vtbbr: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:193]: failed to verify nodes are Ready with Cilium CNI for cluster "cilium-cluster" Unexpected error: <errors.joinError \| 0xc0011d4048>: VerifyNodesReady failed: no nodes found in the cluster VerifyCiliumOperational failed: not all pods in kube-system namespace are running: failed to list pods in kube-system namespace: client rate limiter Wait returned an error: context deadline exceeded failed to create NodePool cilium-np: failed waiting for nodepool="cilium-np" for cluster "cilium-cluster" in resourcegroup="complex-cilium-kv-rrrmbd9jmmdl" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/6d52358f-8309-4bc7-af9f-a5884f9307e0 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/6d52358f-8309-4bc7-af9f-a5884f9307e0", "name": "6d52358f-8309-4bc7-af9f-a5884f9307e0", "status": "Failed", "startTime": "2026-10-02T03:16:58.467567062Z", "endTime": "2026-10-02T03:36:03.226032745Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine cilium-cluster-cilium-np-sqrfn-pjdbq: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine cilium-cluster-cilium-np-sqrfn-vtbbr: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine cilium-cluster-cilium-np-sqrfn-pjdbq: Creating: virtualmachine creating or updating\nMachine cilium-cluster-cilium-np-sqrfn-vtbbr: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 02:00:29 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 5.0 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-2pqt8t" for cluster "cluster-candidate-5-0-2pqt8t" Unexpected error: <fmt.wrapError \| 0xc000e6c920>: failed to create NodePool np-2pqt8t: failed waiting for nodepool="np-2pqt8t" for cluster "cluster-candidate-5-0-2pqt8t" in resourcegroup="rg-candidate-5-0-5cmhl28njm2n" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/1671f2aa-76df-4172-89e0-297ac668f021 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/1671f2aa-76df-4172-89e0-297ac668f021", "name": "1671f2aa-76df-4172-89e0-297ac668f021", "status": "Failed", "startTime": "2026-10-02T03:42:21.546109728Z", "endTime": "2026-10-02T04:01:23.351194166Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine w7u4t7y3s9c5g5m-np-2pqt8t-xxc4f-8bg9m: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine w7u4t7y3s9c5g5m-np-2pqt8t-xxc4f-v5mdw: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine w7u4t7y3s9c5g5m-np-2pqt8t-xxc4f-8bg9m: Creating: virtualmachine creating or updating\nMachine w7u4t7y3s9c5g5m-np-2pqt8t-xxc4f-v5mdw: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-2pqt8t" for cluster "cluster-candidate-5-0-2pqt8t" Unexpected error: <fmt.wrapError \| 0xc000e6c920>: failed to create NodePool np-2pqt8t: failed waiting for nodepool="np-2pqt8t" for cluster "cluster-candidate-5-0-2pqt8t" in resourcegroup="rg-candidate-5-0-5cmhl28njm2n" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/1671f2aa-76df-4172-89e0-297ac668f021 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/1671f2aa-76df-4172-89e0-297ac668f021", "name": "1671f2aa-76df-4172-89e0-297ac668f021", "status": "Failed", "startTime": "2026-10-02T03:42:21.546109728Z", "endTime": "2026-10-02T04:01:23.351194166Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine w7u4t7y3s9c5g5m-np-2pqt8t-xxc4f-8bg9m: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine w7u4t7y3s9c5g5m-np-2pqt8t-xxc4f-v5mdw: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine w7u4t7y3s9c5g5m-np-2pqt8t-xxc4f-8bg9m: Creating: virtualmachine creating or updating\nMachine w7u4t7y3s9c5g5m-np-2pqt8t-xxc4f-v5mdw: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred
00:53:40	2h 6m 10s	branch-ci-Azure-ARO-HCP-main-e2e-stage-e2e-parallel post-goodmerged PR No additional run metadata captured	uksouth	e2e	failed	n/a	11	e2e (11) ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 00:53:40 UTC · failed at e2e · test Customer should be able to create a no-CNI private cluster with a private key vault, a nodepool and install cilium CNI successfully · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:193]: failed to verify nodes are Ready with Cilium CNI for cluster "cilium-cluster" Unexpected error: <errors.joinError \| 0xc002434018>: VerifyNodesReady failed: no nodes found in the cluster VerifyCiliumOperational failed: not all pods in kube-system namespace are running: no cilium pods found in kube-system namespace failed to create NodePool cilium-np: failed waiting for nodepool="cilium-np" for cluster "cilium-cluster" in resourcegroup="complex-cilium-kv-9gqhrxhspmxj" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/1e2689a9-2918-4b62-b602-77629a36d12f -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/1e2689a9-2918-4b62-b602-77629a36d12f", "name": "1e2689a9-2918-4b62-b602-77629a36d12f", "status": "Failed", "startTime": "2026-10-02T01:12:51.230795814Z", "endTime": "2026-10-02T01:31:55.658354454Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine cilium-cluster-cilium-np-zw5cd-7wmwm: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine cilium-cluster-cilium-np-zw5cd-t5w4x: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine cilium-cluster-cilium-np-zw5cd-7wmwm: Creating: virtualmachine creating or updating\nMachine cilium-cluster-cilium-np-zw5cd-t5w4x: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:193]: failed to verify nodes are Ready with Cilium CNI for cluster "cilium-cluster" Unexpected error: <errors.joinError \| 0xc002434018>: VerifyNodesReady failed: no nodes found in the cluster VerifyCiliumOperational failed: not all pods in kube-system namespace are running: no cilium pods found in kube-system namespace failed to create NodePool cilium-np: failed waiting for nodepool="cilium-np" for cluster "cilium-cluster" in resourcegroup="complex-cilium-kv-9gqhrxhspmxj" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/1e2689a9-2918-4b62-b602-77629a36d12f -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/1e2689a9-2918-4b62-b602-77629a36d12f", "name": "1e2689a9-2918-4b62-b602-77629a36d12f", "status": "Failed", "startTime": "2026-10-02T01:12:51.230795814Z", "endTime": "2026-10-02T01:31:55.658354454Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine cilium-cluster-cilium-np-zw5cd-7wmwm: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine cilium-cluster-cilium-np-zw5cd-t5w4x: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine cilium-cluster-cilium-np-zw5cd-7wmwm: Creating: virtualmachine creating or updating\nMachine cilium-cluster-cilium-np-zw5cd-t5w4x: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 00:53:40 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 5.0 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-tgw4bw" for cluster "cluster-candidate-5-0-tgw4bw" Unexpected error: <fmt.wrapError \| 0xc0013d7020>: failed to create NodePool np-tgw4bw: failed waiting for nodepool="np-tgw4bw" for cluster "cluster-candidate-5-0-tgw4bw" in resourcegroup="rg-candidate-5-0-c5dplrlznljs" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/e4804927-bde6-4d56-b2a2-861fb62752d1 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/e4804927-bde6-4d56-b2a2-861fb62752d1", "name": "e4804927-bde6-4d56-b2a2-861fb62752d1", "status": "Failed", "startTime": "2026-10-02T01:36:37.66754756Z", "endTime": "2026-10-02T01:55:45.209455595Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine h0a6a6p2r2u4e6h-np-tgw4bw-s7jjh-h9x7v: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine h0a6a6p2r2u4e6h-np-tgw4bw-s7jjh-q4chj: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine h0a6a6p2r2u4e6h-np-tgw4bw-s7jjh-h9x7v: Creating: virtualmachine creating or updating\nMachine h0a6a6p2r2u4e6h-np-tgw4bw-s7jjh-q4chj: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-tgw4bw" for cluster "cluster-candidate-5-0-tgw4bw" Unexpected error: <fmt.wrapError \| 0xc0013d7020>: failed to create NodePool np-tgw4bw: failed waiting for nodepool="np-tgw4bw" for cluster "cluster-candidate-5-0-tgw4bw" in resourcegroup="rg-candidate-5-0-c5dplrlznljs" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/e4804927-bde6-4d56-b2a2-861fb62752d1 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/e4804927-bde6-4d56-b2a2-861fb62752d1", "name": "e4804927-bde6-4d56-b2a2-861fb62752d1", "status": "Failed", "startTime": "2026-10-02T01:36:37.66754756Z", "endTime": "2026-10-02T01:55:45.209455595Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine h0a6a6p2r2u4e6h-np-tgw4bw-s7jjh-h9x7v: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine h0a6a6p2r2u4e6h-np-tgw4bw-s7jjh-q4chj: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine h0a6a6p2r2u4e6h-np-tgw4bw-s7jjh-h9x7v: Creating: virtualmachine creating or updating\nMachine h0a6a6p2r2u4e6h-np-tgw4bw-s7jjh-q4chj: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 00:53:40 UTC · failed at e2e · test Create HCPOpenShiftCluster with Private KeyVault should create a cluster with private keyvault using v20251223preview API · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:138]: failed to create node pool "np-1" for private keyvault cluster "private-kv-cluster" Unexpected error: <fmt.wrapError \| 0xc00101d200>: failed to create NodePool np-1: failed waiting for nodepool="np-1" for cluster "private-kv-cluster" in resourcegroup="private-keyvault-2lqkfn8tvwwp" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/f4098a97-bd38-463d-bee6-54c33fcd21ea -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/f4098a97-bd38-463d-bee6-54c33fcd21ea", "name": "f4098a97-bd38-463d-bee6-54c33fcd21ea", "status": "Failed", "startTime": "2026-10-02T01:12:48.719133172Z", "endTime": "2026-10-02T01:31:55.837797735Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine b6g9d8r2h2w9x0t-np-1-qhfgz-kdjwf: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine b6g9d8r2h2w9x0t-np-1-qhfgz-q7z6x: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine b6g9d8r2h2w9x0t-np-1-qhfgz-kdjwf: Creating: virtualmachine creating or updating\nMachine b6g9d8r2h2w9x0t-np-1-qhfgz-q7z6x: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:138]: failed to create node pool "np-1" for private keyvault cluster "private-kv-cluster" Unexpected error: <fmt.wrapError \| 0xc00101d200>: failed to create NodePool np-1: failed waiting for nodepool="np-1" for cluster "private-kv-cluster" in resourcegroup="private-keyvault-2lqkfn8tvwwp" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/f4098a97-bd38-463d-bee6-54c33fcd21ea -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/f4098a97-bd38-463d-bee6-54c33fcd21ea", "name": "f4098a97-bd38-463d-bee6-54c33fcd21ea", "status": "Failed", "startTime": "2026-10-02T01:12:48.719133172Z", "endTime": "2026-10-02T01:31:55.837797735Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine b6g9d8r2h2w9x0t-np-1-qhfgz-kdjwf: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine b6g9d8r2h2w9x0t-np-1-qhfgz-q7z6x: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine b6g9d8r2h2w9x0t-np-1-qhfgz-kdjwf: Creating: virtualmachine creating or updating\nMachine b6g9d8r2h2w9x0t-np-1-qhfgz-q7z6x: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift node pool not cached yet; provider Microsoft.RedHatOpenShift 00:53:40 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.20 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-2dtcq8" for cluster "cluster-candidate-4-20-2dtcq8" Unexpected error: <fmt.wrapError \| 0xc0018500e0>: failed to create NodePool np-2dtcq8: failed waiting for nodepool="np-2dtcq8" for cluster "cluster-candidate-4-20-2dtcq8" in resourcegroup="rg-candidate-4-20-wsgq7rfg4j9c" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4f8b062f-d3f0-4efa-9aa5-be9e1c5218c2 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4f8b062f-d3f0-4efa-9aa5-be9e1c5218c2", "name": "4f8b062f-d3f0-4efa-9aa5-be9e1c5218c2", "status": "Failed", "startTime": "2026-10-02T01:35:03.016964221Z", "endTime": "2026-10-02T01:54:05.336017308Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift node pool not cached yet" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-2dtcq8" for cluster "cluster-candidate-4-20-2dtcq8" Unexpected error: <fmt.wrapError \| 0xc0018500e0>: failed to create NodePool np-2dtcq8: failed waiting for nodepool="np-2dtcq8" for cluster "cluster-candidate-4-20-2dtcq8" in resourcegroup="rg-candidate-4-20-wsgq7rfg4j9c" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4f8b062f-d3f0-4efa-9aa5-be9e1c5218c2 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4f8b062f-d3f0-4efa-9aa5-be9e1c5218c2", "name": "4f8b062f-d3f0-4efa-9aa5-be9e1c5218c2", "status": "Failed", "startTime": "2026-10-02T01:35:03.016964221Z", "endTime": "2026-10-02T01:54:05.336017308Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift node pool not cached yet" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 00:53:40 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.23 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-z9vgpc" for cluster "cluster-candidate-4-23-z9vgpc" Unexpected error: <fmt.wrapError \| 0xc000d433c0>: failed to create NodePool np-z9vgpc: failed waiting for nodepool="np-z9vgpc" for cluster "cluster-candidate-4-23-z9vgpc" in resourcegroup="rg-candidate-4-23-k2q8jrl7bhxd" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/a5150081-901b-4236-a201-530c0a2b188a -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/a5150081-901b-4236-a201-530c0a2b188a", "name": "a5150081-901b-4236-a201-530c0a2b188a", "status": "Failed", "startTime": "2026-10-02T01:35:33.904818612Z", "endTime": "2026-10-02T01:54:35.210883513Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine y0w7y2a8c7t4f6j-np-z9vgpc-4fpk5-54zr9: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine y0w7y2a8c7t4f6j-np-z9vgpc-4fpk5-tfmvd: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine y0w7y2a8c7t4f6j-np-z9vgpc-4fpk5-54zr9: Creating: virtualmachine creating or updating\nMachine y0w7y2a8c7t4f6j-np-z9vgpc-4fpk5-tfmvd: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-z9vgpc" for cluster "cluster-candidate-4-23-z9vgpc" Unexpected error: <fmt.wrapError \| 0xc000d433c0>: failed to create NodePool np-z9vgpc: failed waiting for nodepool="np-z9vgpc" for cluster "cluster-candidate-4-23-z9vgpc" in resourcegroup="rg-candidate-4-23-k2q8jrl7bhxd" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/a5150081-901b-4236-a201-530c0a2b188a -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/a5150081-901b-4236-a201-530c0a2b188a", "name": "a5150081-901b-4236-a201-530c0a2b188a", "status": "Failed", "startTime": "2026-10-02T01:35:33.904818612Z", "endTime": "2026-10-02T01:54:35.210883513Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine y0w7y2a8c7t4f6j-np-z9vgpc-4fpk5-54zr9: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine y0w7y2a8c7t4f6j-np-z9vgpc-4fpk5-tfmvd: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine y0w7y2a8c7t4f6j-np-z9vgpc-4fpk5-54zr9: Creating: virtualmachine creating or updating\nMachine y0w7y2a8c7t4f6j-np-z9vgpc-4fpk5-tfmvd: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift node pool not cached yet; provider Microsoft.RedHatOpenShift 00:53:40 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 5.1 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-ljll7c" for cluster "cluster-candidate-5-1-ljll7c" Unexpected error: <fmt.wrapError \| 0xc002d92dc0>: failed to create NodePool np-ljll7c: failed waiting for nodepool="np-ljll7c" for cluster "cluster-candidate-5-1-ljll7c" in resourcegroup="rg-candidate-5-1-m96hr99msv6f" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/3472529a-993b-497a-a853-58314ee41fea -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/3472529a-993b-497a-a853-58314ee41fea", "name": "3472529a-993b-497a-a853-58314ee41fea", "status": "Failed", "startTime": "2026-10-02T01:39:28.8432233Z", "endTime": "2026-10-02T01:58:35.211672532Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift node pool not cached yet" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-ljll7c" for cluster "cluster-candidate-5-1-ljll7c" Unexpected error: <fmt.wrapError \| 0xc002d92dc0>: failed to create NodePool np-ljll7c: failed waiting for nodepool="np-ljll7c" for cluster "cluster-candidate-5-1-ljll7c" in resourcegroup="rg-candidate-5-1-m96hr99msv6f" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/3472529a-993b-497a-a853-58314ee41fea -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/3472529a-993b-497a-a853-58314ee41fea", "name": "3472529a-993b-497a-a853-58314ee41fea", "status": "Failed", "startTime": "2026-10-02T01:39:28.8432233Z", "endTime": "2026-10-02T01:58:35.211672532Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift node pool not cached yet" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 00:53:40 UTC · failed at e2e · test Customer should create a cluster with private KAS and verify API server is only reachable from VNet while ingress remains public · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:159]: failed to create node pool "np-1" for private KAS cluster "private-kas" Unexpected error: <fmt.wrapError \| 0xc000b95a80>: failed to create NodePool np-1: failed waiting for nodepool="np-1" for cluster "private-kas" in resourcegroup="private-kas-vp5rgzktgrqp" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/7950f938-55ee-4ce6-a5ca-480917c685bf -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/7950f938-55ee-4ce6-a5ca-480917c685bf", "name": "7950f938-55ee-4ce6-a5ca-480917c685bf", "status": "Failed", "startTime": "2026-10-02T01:15:07.199174306Z", "endTime": "2026-10-02T01:34:15.817819834Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 1; node pool condition AllNodesHealthy is False: 1 of 1 machines are not healthy\nMachine private-kas-np-1-vqbnm-zdcgw: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 1 of 1 machines are not ready\nMachine private-kas-np-1-vqbnm-zdcgw: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:159]: failed to create node pool "np-1" for private KAS cluster "private-kas" Unexpected error: <fmt.wrapError \| 0xc000b95a80>: failed to create NodePool np-1: failed waiting for nodepool="np-1" for cluster "private-kas" in resourcegroup="private-kas-vp5rgzktgrqp" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/7950f938-55ee-4ce6-a5ca-480917c685bf -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/7950f938-55ee-4ce6-a5ca-480917c685bf", "name": "7950f938-55ee-4ce6-a5ca-480917c685bf", "status": "Failed", "startTime": "2026-10-02T01:15:07.199174306Z", "endTime": "2026-10-02T01:34:15.817819834Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 1; node pool condition AllNodesHealthy is False: 1 of 1 machines are not healthy\nMachine private-kas-np-1-vqbnm-zdcgw: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 1 of 1 machines are not ready\nMachine private-kas-np-1-vqbnm-zdcgw: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 00:53:40 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.22 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-pdsktt" for cluster "cluster-candidate-4-22-pdsktt" Unexpected error: <fmt.wrapError \| 0xc0010d83a0>: failed to create NodePool np-pdsktt: failed waiting for nodepool="np-pdsktt" for cluster "cluster-candidate-4-22-pdsktt" in resourcegroup="rg-candidate-4-22-5hpdwnfcpnnf" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/987287c4-95cd-422d-be6d-282705a06618 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/987287c4-95cd-422d-be6d-282705a06618", "name": "987287c4-95cd-422d-be6d-282705a06618", "status": "Failed", "startTime": "2026-10-02T01:35:11.708229062Z", "endTime": "2026-10-02T01:54:15.19502959Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine m9p9y7p7y8q8y0t-np-pdsktt-tcq2h-b9m75: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine m9p9y7p7y8q8y0t-np-pdsktt-tcq2h-vsxkb: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine m9p9y7p7y8q8y0t-np-pdsktt-tcq2h-b9m75: Creating: virtualmachine creating or updating\nMachine m9p9y7p7y8q8y0t-np-pdsktt-tcq2h-vsxkb: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-pdsktt" for cluster "cluster-candidate-4-22-pdsktt" Unexpected error: <fmt.wrapError \| 0xc0010d83a0>: failed to create NodePool np-pdsktt: failed waiting for nodepool="np-pdsktt" for cluster "cluster-candidate-4-22-pdsktt" in resourcegroup="rg-candidate-4-22-5hpdwnfcpnnf" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/987287c4-95cd-422d-be6d-282705a06618 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/987287c4-95cd-422d-be6d-282705a06618", "name": "987287c4-95cd-422d-be6d-282705a06618", "status": "Failed", "startTime": "2026-10-02T01:35:11.708229062Z", "endTime": "2026-10-02T01:54:15.19502959Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine m9p9y7p7y8q8y0t-np-pdsktt-tcq2h-b9m75: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine m9p9y7p7y8q8y0t-np-pdsktt-tcq2h-vsxkb: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine m9p9y7p7y8q8y0t-np-pdsktt-tcq2h-b9m75: Creating: virtualmachine creating or updating\nMachine m9p9y7p7y8q8y0t-np-pdsktt-tcq2h-vsxkb: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want >= <count> (autoscaling min); node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 00:53:40 UTC · failed at e2e · test Customer should be able to create a cluster and node pools with aggregated advanced features · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:404]: failed to verify cilium is running and nodes are Ready for cluster "agg-cluster" Unexpected error: <errors.joinError \| 0xc001a9c060>: VerifyNodesReady failed: no nodes found in the cluster VerifyCiliumOperational failed: not all pods in kube-system namespace are running: failed to list pods in kube-system namespace: client rate limiter Wait returned an error: context deadline exceeded failed to create NodePool agg-np-a: failed waiting for nodepool="agg-np-a" for cluster "agg-cluster" in resourcegroup="feature-aggregation-v2ffkzvzwhp9" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/59c86be8-df72-4988-a51b-6b1b20b915f4 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/59c86be8-df72-4988-a51b-6b1b20b915f4", "name": "59c86be8-df72-4988-a51b-6b1b20b915f4", "status": "Failed", "startTime": "2026-10-02T01:13:42.494621442Z", "endTime": "2026-10-02T01:32:45.646902114Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want \u003e= 2 (autoscaling min); node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine agg-cluster-agg-np-a-69vhp-6nxhl: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine agg-cluster-agg-np-a-69vhp-rch98: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine agg-cluster-agg-np-a-69vhp-6nxhl: Creating: virtualmachine creating or updating\nMachine agg-cluster-agg-np-a-69vhp-rch98: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:404]: failed to verify cilium is running and nodes are Ready for cluster "agg-cluster" Unexpected error: <errors.joinError \| 0xc001a9c060>: VerifyNodesReady failed: no nodes found in the cluster VerifyCiliumOperational failed: not all pods in kube-system namespace are running: failed to list pods in kube-system namespace: client rate limiter Wait returned an error: context deadline exceeded failed to create NodePool agg-np-a: failed waiting for nodepool="agg-np-a" for cluster "agg-cluster" in resourcegroup="feature-aggregation-v2ffkzvzwhp9" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/59c86be8-df72-4988-a51b-6b1b20b915f4 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/59c86be8-df72-4988-a51b-6b1b20b915f4", "name": "59c86be8-df72-4988-a51b-6b1b20b915f4", "status": "Failed", "startTime": "2026-10-02T01:13:42.494621442Z", "endTime": "2026-10-02T01:32:45.646902114Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want \u003e= 2 (autoscaling min); node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine agg-cluster-agg-np-a-69vhp-6nxhl: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine agg-cluster-agg-np-a-69vhp-rch98: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine agg-cluster-agg-np-a-69vhp-6nxhl: Creating: virtualmachine creating or updating\nMachine agg-cluster-agg-np-a-69vhp-rch98: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 00:53:40 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.21 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-tnjk89" for cluster "cluster-candidate-4-21-tnjk89" Unexpected error: <fmt.wrapError \| 0xc000a24560>: failed to create NodePool np-tnjk89: failed waiting for nodepool="np-tnjk89" for cluster "cluster-candidate-4-21-tnjk89" in resourcegroup="rg-candidate-4-21-hbwb4qtqm5m5" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/c80392bf-1395-461b-9b36-b3ca3dfa3732 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/c80392bf-1395-461b-9b36-b3ca3dfa3732", "name": "c80392bf-1395-461b-9b36-b3ca3dfa3732", "status": "Failed", "startTime": "2026-10-02T01:34:52.67307417Z", "endTime": "2026-10-02T01:53:55.817268557Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine x5t3x3n0c0q1y5x-np-tnjk89-psj6d-mfsnf: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine x5t3x3n0c0q1y5x-np-tnjk89-psj6d-rmqgp: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine x5t3x3n0c0q1y5x-np-tnjk89-psj6d-mfsnf: Creating: virtualmachine creating or updating\nMachine x5t3x3n0c0q1y5x-np-tnjk89-psj6d-rmqgp: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-tnjk89" for cluster "cluster-candidate-4-21-tnjk89" Unexpected error: <fmt.wrapError \| 0xc000a24560>: failed to create NodePool np-tnjk89: failed waiting for nodepool="np-tnjk89" for cluster "cluster-candidate-4-21-tnjk89" in resourcegroup="rg-candidate-4-21-hbwb4qtqm5m5" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/c80392bf-1395-461b-9b36-b3ca3dfa3732 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/c80392bf-1395-461b-9b36-b3ca3dfa3732", "name": "c80392bf-1395-461b-9b36-b3ca3dfa3732", "status": "Failed", "startTime": "2026-10-02T01:34:52.67307417Z", "endTime": "2026-10-02T01:53:55.817268557Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine x5t3x3n0c0q1y5x-np-tnjk89-psj6d-mfsnf: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine x5t3x3n0c0q1y5x-np-tnjk89-psj6d-rmqgp: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine x5t3x3n0c0q1y5x-np-tnjk89-psj6d-mfsnf: Creating: virtualmachine creating or updating\nMachine x5t3x3n0c0q1y5x-np-tnjk89-psj6d-rmqgp: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred failed to create nodepool <nodepool>: failed waiting for nodepool="<nodepool>" for cluster "<cluster>" in resourcegroup="<resource-group>" to finish creating: GET <url> 00:53:40 UTC · failed at e2e · test Customer should be able to delete a nodepool · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_delete.go:128]: failed to create node pools Expected success, but got an error: <fmt.wrapError \| 0xc000eaa180>: failed to create NodePool np-deletable: failed waiting for nodepool="np-deletable" for cluster "np-delete-hcp-cluster" in resourcegroup="nodepool-delete-ks2bs7m442f9" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/3917dbfd-2f9f-4a35-8346-d9c0b133e313 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/3917dbfd-2f9f-4a35-8346-d9c0b133e313", "name": "3917dbfd-2f9f-4a35-8346-d9c0b133e313", "status": "Failed", "startTime": "2026-10-02T02:04:51.953995015Z", "endTime": "2026-10-02T02:23:58.43798257Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine w5t1z3a1d2p4k6h-np-deletable-l2j9j-85c4g: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine w5t1z3a1d2p4k6h-np-deletable-l2j9j-g4qwf: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine w5t1z3a1d2p4k6h-np-deletable-l2j9j-85c4g: Creating: virtualmachine creating or updating\nMachine w5t1z3a1d2p4k6h-np-deletable-l2j9j-g4qwf: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_delete.go:128]: failed to create node pools Expected success, but got an error: <fmt.wrapError \| 0xc000eaa180>: failed to create NodePool np-deletable: failed waiting for nodepool="np-deletable" for cluster "np-delete-hcp-cluster" in resourcegroup="nodepool-delete-ks2bs7m442f9" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/3917dbfd-2f9f-4a35-8346-d9c0b133e313 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/3917dbfd-2f9f-4a35-8346-d9c0b133e313", "name": "3917dbfd-2f9f-4a35-8346-d9c0b133e313", "status": "Failed", "startTime": "2026-10-02T02:04:51.953995015Z", "endTime": "2026-10-02T02:23:58.43798257Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine w5t1z3a1d2p4k6h-np-deletable-l2j9j-85c4g: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine w5t1z3a1d2p4k6h-np-deletable-l2j9j-g4qwf: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine w5t1z3a1d2p4k6h-np-deletable-l2j9j-85c4g: Creating: virtualmachine creating or updating\nMachine w5t1z3a1d2p4k6h-np-deletable-l2j9j-g4qwf: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ...