CIHealth Run Log

Environment: STG

Time (UTC)	Runtime	Job	Region	Failed at	Result	PR	Failed tests	Details
23:00:45	1h 24m 55s	periodic-ci-Azure-ARO-HCP-main-periodic-stage-e2e-parallel-ocp-nightly post-goodmerged PR No additional run metadata captured	uksouth	e2e	failed	n/a	14	e2e (14) ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 23:00:45 UTC · failed at e2e · test Customer should be able to create a no-CNI private cluster with a private key vault, a nodepool and install cilium CNI successfully · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:193]: failed to verify nodes are Ready with Cilium CNI for cluster "cilium-cluster" Unexpected error: <errors.joinError \| 0x3731739b0018>: VerifyNodesReady failed: no nodes found in the cluster VerifyCiliumOperational failed: not all pods in kube-system namespace are running: failed to list pods in kube-system namespace: client rate limiter Wait returned an error: context deadline exceeded failed to create NodePool cilium-np: failed waiting for nodepool="cilium-np" for cluster "cilium-cluster" in resourcegroup="complex-cilium-kv-4h58pvg4tw2z" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/1659164e-7252-4678-9399-950feff9f29d -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/1659164e-7252-4678-9399-950feff9f29d", "name": "1659164e-7252-4678-9399-950feff9f29d", "status": "Failed", "startTime": "2026-10-03T23:15:46.648351931Z", "endTime": "2026-10-03T23:34:50.305129886Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine cilium-cluster-cilium-np-hkqzd-6mfw6: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine cilium-cluster-cilium-np-hkqzd-7x9wt: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine cilium-cluster-cilium-np-hkqzd-6mfw6: Creating: virtualmachine creating or updating\nMachine cilium-cluster-cilium-np-hkqzd-7x9wt: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:193]: failed to verify nodes are Ready with Cilium CNI for cluster "cilium-cluster" Unexpected error: <errors.joinError \| 0x3731739b0018>: VerifyNodesReady failed: no nodes found in the cluster VerifyCiliumOperational failed: not all pods in kube-system namespace are running: failed to list pods in kube-system namespace: client rate limiter Wait returned an error: context deadline exceeded failed to create NodePool cilium-np: failed waiting for nodepool="cilium-np" for cluster "cilium-cluster" in resourcegroup="complex-cilium-kv-4h58pvg4tw2z" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/1659164e-7252-4678-9399-950feff9f29d -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/1659164e-7252-4678-9399-950feff9f29d", "name": "1659164e-7252-4678-9399-950feff9f29d", "status": "Failed", "startTime": "2026-10-03T23:15:46.648351931Z", "endTime": "2026-10-03T23:34:50.305129886Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine cilium-cluster-cilium-np-hkqzd-6mfw6: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine cilium-cluster-cilium-np-hkqzd-7x9wt: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine cilium-cluster-cilium-np-hkqzd-6mfw6: Creating: virtualmachine creating or updating\nMachine cilium-cluster-cilium-np-hkqzd-7x9wt: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InvalidRequestContent; detail message Invalid value: "<ocp-nightly>": must be at least 4.20.8; provider Microsoft.RedHatOpenShift 23:00:45 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP nightly channel for 4.20 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-zb9djq" for cluster "cluster-nightly-4-20-zb9djq" Unexpected error: <fmt.wrapError \| 0x242124721260>: failed to create NodePool np-zb9djq: failed starting nodepool creation "np-zb9djq" for cluster "cluster-nightly-4-20-zb9djq" in resourcegroup="rg-nightly-4-20-r55xk7zmjr9c": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-nightly-4-20-r55xk7zmjr9c/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-nightly-4-20-zb9djq/nodePools/np-zb9djq -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"4.20.0-0.nightly-multi-2026-10-01-125400\": must be at least 4.20.8", "target": "properties.version.id" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-zb9djq" for cluster "cluster-nightly-4-20-zb9djq" Unexpected error: <fmt.wrapError \| 0x242124721260>: failed to create NodePool np-zb9djq: failed starting nodepool creation "np-zb9djq" for cluster "cluster-nightly-4-20-zb9djq" in resourcegroup="rg-nightly-4-20-r55xk7zmjr9c": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-nightly-4-20-r55xk7zmjr9c/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-nightly-4-20-zb9djq/nodePools/np-zb9djq -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"4.20.0-0.nightly-multi-2026-10-01-125400\": must be at least 4.20.8", "target": "properties.version.id" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InvalidRequestContent; detail message Invalid value: "<ocp-nightly>": must be at least 4.20.8; provider Microsoft.RedHatOpenShift 23:00:45 UTC · failed at e2e · test Customer should upgrade and update a nodepool from 4.20.z to 4.21.zLatest · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:165]: failed to create node pool npupgrade-4-20 with version 4.20.0-0.nightly-multi-2026-10-01-125400 Unexpected error: <fmt.wrapError \| 0x184038744400>: failed to create NodePool npupgrade-4-20: failed starting nodepool creation "npupgrade-4-20" for cluster "np-version-upgrade-cluster-tzvzfh" in resourcegroup="rg-np-version-upgrade-tzvzfh-8sk9jqkdk6gz": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-version-upgrade-tzvzfh-8sk9jqkdk6gz/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-version-upgrade-cluster-tzvzfh/nodePools/npupgrade-4-20 -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"4.20.0-0.nightly-multi-2026-10-01-125400\": must be at least 4.20.8", "target": "properties.version.id" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:165]: failed to create node pool npupgrade-4-20 with version 4.20.0-0.nightly-multi-2026-10-01-125400 Unexpected error: <fmt.wrapError \| 0x184038744400>: failed to create NodePool npupgrade-4-20: failed starting nodepool creation "npupgrade-4-20" for cluster "np-version-upgrade-cluster-tzvzfh" in resourcegroup="rg-np-version-upgrade-tzvzfh-8sk9jqkdk6gz": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-version-upgrade-tzvzfh-8sk9jqkdk6gz/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-version-upgrade-cluster-tzvzfh/nodePools/npupgrade-4-20 -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"4.20.0-0.nightly-multi-2026-10-01-125400\": must be at least 4.20.8", "target": "properties.version.id" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 23:00:45 UTC · failed at e2e · test Create HCPOpenShiftCluster with Private KeyVault should create a cluster with private keyvault using v20251223preview API · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:138]: failed to create node pool "np-1" for private keyvault cluster "private-kv-cluster" Unexpected error: <fmt.wrapError \| 0x366855210620>: failed to create NodePool np-1: failed waiting for nodepool="np-1" for cluster "private-kv-cluster" in resourcegroup="private-keyvault-4c8kl482c2st" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/37548aa4-a4e7-458d-b686-fb1201996386 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/37548aa4-a4e7-458d-b686-fb1201996386", "name": "37548aa4-a4e7-458d-b686-fb1201996386", "status": "Failed", "startTime": "2026-10-03T23:15:28.253854505Z", "endTime": "2026-10-03T23:34:30.095088976Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine c4o5k1a3s4v8w4p-np-1-gjcb5-n2nhw: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine c4o5k1a3s4v8w4p-np-1-gjcb5-xsvvt: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine c4o5k1a3s4v8w4p-np-1-gjcb5-n2nhw: Creating: virtualmachine creating or updating\nMachine c4o5k1a3s4v8w4p-np-1-gjcb5-xsvvt: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:138]: failed to create node pool "np-1" for private keyvault cluster "private-kv-cluster" Unexpected error: <fmt.wrapError \| 0x366855210620>: failed to create NodePool np-1: failed waiting for nodepool="np-1" for cluster "private-kv-cluster" in resourcegroup="private-keyvault-4c8kl482c2st" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/37548aa4-a4e7-458d-b686-fb1201996386 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/37548aa4-a4e7-458d-b686-fb1201996386", "name": "37548aa4-a4e7-458d-b686-fb1201996386", "status": "Failed", "startTime": "2026-10-03T23:15:28.253854505Z", "endTime": "2026-10-03T23:34:30.095088976Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine c4o5k1a3s4v8w4p-np-1-gjcb5-n2nhw: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine c4o5k1a3s4v8w4p-np-1-gjcb5-xsvvt: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine c4o5k1a3s4v8w4p-np-1-gjcb5-n2nhw: Creating: virtualmachine creating or updating\nMachine c4o5k1a3s4v8w4p-np-1-gjcb5-xsvvt: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 23:00:45 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP nightly channel for 5.0 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-dz68ws" for cluster "cluster-nightly-5-0-dz68ws" Unexpected error: <fmt.wrapError \| 0x31a06c9a5d00>: failed to create NodePool np-dz68ws: failed waiting for nodepool="np-dz68ws" for cluster "cluster-nightly-5-0-dz68ws" in resourcegroup="rg-nightly-5-0-4t5m8n4czrvj" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/2e4aeefa-8e4c-446e-bbb2-afff95c12936 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/2e4aeefa-8e4c-446e-bbb2-afff95c12936", "name": "2e4aeefa-8e4c-446e-bbb2-afff95c12936", "status": "Failed", "startTime": "2026-10-03T23:35:55.566084896Z", "endTime": "2026-10-03T23:55:00.222357543Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine h3s7g6c6v2v9l8v-np-dz68ws-76lrt-95fl8: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine h3s7g6c6v2v9l8v-np-dz68ws-76lrt-kc94l: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine h3s7g6c6v2v9l8v-np-dz68ws-76lrt-95fl8: Creating: virtualmachine creating or updating\nMachine h3s7g6c6v2v9l8v-np-dz68ws-76lrt-kc94l: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-dz68ws" for cluster "cluster-nightly-5-0-dz68ws" Unexpected error: <fmt.wrapError \| 0x31a06c9a5d00>: failed to create NodePool np-dz68ws: failed waiting for nodepool="np-dz68ws" for cluster "cluster-nightly-5-0-dz68ws" in resourcegroup="rg-nightly-5-0-4t5m8n4czrvj" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/2e4aeefa-8e4c-446e-bbb2-afff95c12936 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/2e4aeefa-8e4c-446e-bbb2-afff95c12936", "name": "2e4aeefa-8e4c-446e-bbb2-afff95c12936", "status": "Failed", "startTime": "2026-10-03T23:35:55.566084896Z", "endTime": "2026-10-03T23:55:00.222357543Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine h3s7g6c6v2v9l8v-np-dz68ws-76lrt-95fl8: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine h3s7g6c6v2v9l8v-np-dz68ws-76lrt-kc94l: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine h3s7g6c6v2v9l8v-np-dz68ws-76lrt-95fl8: Creating: virtualmachine creating or updating\nMachine h3s7g6c6v2v9l8v-np-dz68ws-76lrt-kc94l: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InvalidRequestContent; detail message Invalid value: "4.21": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is "nightly"; provider Microsoft.RedHatOpenShift 23:00:45 UTC · failed at e2e · test Customer should be able to create an HCP cluster with custom autoscaling and update the autoscaling configuration · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:173]: failed to update cluster "autoscaling-hcp-cluster" autoscaling configuration Unexpected error: <exported.ResponseError \| 0x36252fbe4930>: PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/autoscaling-cluster-548xlmq4vp84/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/autoscaling-hcp-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"4.21\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"", "target": "properties.version.id" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:173]: failed to update cluster "autoscaling-hcp-cluster" autoscaling configuration Unexpected error: <exported.ResponseError \| 0x36252fbe4930>: PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/autoscaling-cluster-548xlmq4vp84/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/autoscaling-hcp-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"4.21\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"", "target": "properties.version.id" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InvalidRequestContent; detail message Invalid value: "4.21": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is "nightly"; provider Microsoft.RedHatOpenShift 23:00:45 UTC · failed at e2e · test Customer should be able to create a cluster and node pools with aggregated advanced features · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:275]: failed to begin cluster "agg-cluster" authorized CIDR update Unexpected error: <exported.ResponseError \| 0x3102b57c8300>: PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/feature-aggregation-5xj4fb5fc6dl/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/agg-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"4.21\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"", "target": "properties.version.id" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:275]: failed to begin cluster "agg-cluster" authorized CIDR update Unexpected error: <exported.ResponseError \| 0x3102b57c8300>: PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/feature-aggregation-5xj4fb5fc6dl/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/agg-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"4.21\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"", "target": "properties.version.id" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 23:00:45 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP nightly channel for 4.21 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-vgspf2" for cluster "cluster-nightly-4-21-vgspf2" Unexpected error: <fmt.wrapError \| 0x34ef387fcea0>: failed to create NodePool np-vgspf2: failed waiting for nodepool="np-vgspf2" for cluster "cluster-nightly-4-21-vgspf2" in resourcegroup="rg-nightly-4-21-dsfs76js9854" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/78a87535-b0bc-47d2-9812-340ab7a0fb8e -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/78a87535-b0bc-47d2-9812-340ab7a0fb8e", "name": "78a87535-b0bc-47d2-9812-340ab7a0fb8e", "status": "Failed", "startTime": "2026-10-03T23:34:31.068400483Z", "endTime": "2026-10-03T23:53:40.358231747Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine r3x3m3o0p2b4e4x-np-vgspf2-qj7bv-dh299: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine r3x3m3o0p2b4e4x-np-vgspf2-qj7bv-jlf82: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine r3x3m3o0p2b4e4x-np-vgspf2-qj7bv-dh299: Creating: virtualmachine creating or updating\nMachine r3x3m3o0p2b4e4x-np-vgspf2-qj7bv-jlf82: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-vgspf2" for cluster "cluster-nightly-4-21-vgspf2" Unexpected error: <fmt.wrapError \| 0x34ef387fcea0>: failed to create NodePool np-vgspf2: failed waiting for nodepool="np-vgspf2" for cluster "cluster-nightly-4-21-vgspf2" in resourcegroup="rg-nightly-4-21-dsfs76js9854" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/78a87535-b0bc-47d2-9812-340ab7a0fb8e -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/78a87535-b0bc-47d2-9812-340ab7a0fb8e", "name": "78a87535-b0bc-47d2-9812-340ab7a0fb8e", "status": "Failed", "startTime": "2026-10-03T23:34:31.068400483Z", "endTime": "2026-10-03T23:53:40.358231747Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine r3x3m3o0p2b4e4x-np-vgspf2-qj7bv-dh299: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine r3x3m3o0p2b4e4x-np-vgspf2-qj7bv-jlf82: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine r3x3m3o0p2b4e4x-np-vgspf2-qj7bv-dh299: Creating: virtualmachine creating or updating\nMachine r3x3m3o0p2b4e4x-np-vgspf2-qj7bv-jlf82: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 23:00:45 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP nightly channel for 5.1 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-4pbb46" for cluster "cluster-nightly-5-1-4pbb46" Unexpected error: <fmt.wrapError \| 0x1fb215239360>: failed to create NodePool np-4pbb46: failed waiting for nodepool="np-4pbb46" for cluster "cluster-nightly-5-1-4pbb46" in resourcegroup="rg-nightly-5-1-4rftrv2tqf7l" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/e74d3e2c-deeb-4d38-9a72-e8d6c34c576d -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/e74d3e2c-deeb-4d38-9a72-e8d6c34c576d", "name": "e74d3e2c-deeb-4d38-9a72-e8d6c34c576d", "status": "Failed", "startTime": "2026-10-03T23:37:12.955436587Z", "endTime": "2026-10-03T23:56:20.22170931Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine d6a2h5t1r0s8k0n-np-4pbb46-vntxw-4s8vt: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine d6a2h5t1r0s8k0n-np-4pbb46-vntxw-rvt6q: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine d6a2h5t1r0s8k0n-np-4pbb46-vntxw-4s8vt: Creating: virtualmachine creating or updating\nMachine d6a2h5t1r0s8k0n-np-4pbb46-vntxw-rvt6q: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-4pbb46" for cluster "cluster-nightly-5-1-4pbb46" Unexpected error: <fmt.wrapError \| 0x1fb215239360>: failed to create NodePool np-4pbb46: failed waiting for nodepool="np-4pbb46" for cluster "cluster-nightly-5-1-4pbb46" in resourcegroup="rg-nightly-5-1-4rftrv2tqf7l" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/e74d3e2c-deeb-4d38-9a72-e8d6c34c576d -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/e74d3e2c-deeb-4d38-9a72-e8d6c34c576d", "name": "e74d3e2c-deeb-4d38-9a72-e8d6c34c576d", "status": "Failed", "startTime": "2026-10-03T23:37:12.955436587Z", "endTime": "2026-10-03T23:56:20.22170931Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine d6a2h5t1r0s8k0n-np-4pbb46-vntxw-4s8vt: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine d6a2h5t1r0s8k0n-np-4pbb46-vntxw-rvt6q: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine d6a2h5t1r0s8k0n-np-4pbb46-vntxw-4s8vt: Creating: virtualmachine creating or updating\nMachine d6a2h5t1r0s8k0n-np-4pbb46-vntxw-rvt6q: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InvalidRequestContent; detail message Invalid value: "4.21": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is "nightly"; provider Microsoft.RedHatOpenShift 23:00:45 UTC · failed at e2e · test Customer should be able to create an HCP cluster and manage ImageDigestMirrors · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/idms_lifecycle.go:175]: failed to update cluster with second ImageDigestMirror set Unexpected error: <exported.ResponseError \| 0x2f86d9c91d0>: PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/idms-4kkvfds7xktp/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/idms-e2e-hcp-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"4.21\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"", "target": "properties.version.id" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/idms_lifecycle.go:175]: failed to update cluster with second ImageDigestMirror set Unexpected error: <exported.ResponseError \| 0x2f86d9c91d0>: PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/idms-4kkvfds7xktp/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/idms-e2e-hcp-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"4.21\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"", "target": "properties.version.id" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InvalidRequestContent; detail message Invalid value: "4.21": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is "nightly"; provider Microsoft.RedHatOpenShift 23:00:45 UTC · failed at e2e · test Update HCPOpenShiftCluster Positive creates a cluster and updates tags with a PATCH request · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:193]: failed to update HCP cluster tags via PATCH Unexpected error: <exported.ResponseError \| 0x145ed3760d50>: PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-tags-cxtlbfhqtdd7/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-tags-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"4.21\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"", "target": "properties.version.id" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:193]: failed to update HCP cluster tags via PATCH Unexpected error: <exported.ResponseError \| 0x145ed3760d50>: PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-tags-cxtlbfhqtdd7/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-tags-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"4.21\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"", "target": "properties.version.id" } } -------------------------------------------------------------------------------- ... occurred failed to create nodepool <nodepool>: failed waiting for nodepool="<nodepool>" for cluster "<cluster>" in resourcegroup="<resource-group>" to finish creating: GET <url> 23:00:45 UTC · failed at e2e · test Customer should be able to delete a nodepool · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_delete.go:128]: failed to create node pools Expected success, but got an error: <fmt.wrapError \| 0x104897805d20>: failed to create NodePool np-delete-main: failed waiting for nodepool="np-delete-main" for cluster "np-delete-hcp-cluster" in resourcegroup="nodepool-delete-r8db5d4ldfmh" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/12e347fe-1273-4e0c-9b2a-1c191d6d4055 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/12e347fe-1273-4e0c-9b2a-1c191d6d4055", "name": "12e347fe-1273-4e0c-9b2a-1c191d6d4055", "status": "Failed", "startTime": "2026-10-03T23:41:40.611562032Z", "endTime": "2026-10-04T00:00:46.799966407Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine o0s0h4b0m0k9t7l-np-delete-main-sj9cl-hlwvx: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine o0s0h4b0m0k9t7l-np-delete-main-sj9cl-xbp6q: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine o0s0h4b0m0k9t7l-np-delete-main-sj9cl-hlwvx: Creating: virtualmachine creating or updating\nMachine o0s0h4b0m0k9t7l-np-delete-main-sj9cl-xbp6q: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_delete.go:128]: failed to create node pools Expected success, but got an error: <fmt.wrapError \| 0x104897805d20>: failed to create NodePool np-delete-main: failed waiting for nodepool="np-delete-main" for cluster "np-delete-hcp-cluster" in resourcegroup="nodepool-delete-r8db5d4ldfmh" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/12e347fe-1273-4e0c-9b2a-1c191d6d4055 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/12e347fe-1273-4e0c-9b2a-1c191d6d4055", "name": "12e347fe-1273-4e0c-9b2a-1c191d6d4055", "status": "Failed", "startTime": "2026-10-03T23:41:40.611562032Z", "endTime": "2026-10-04T00:00:46.799966407Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine o0s0h4b0m0k9t7l-np-delete-main-sj9cl-hlwvx: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine o0s0h4b0m0k9t7l-np-delete-main-sj9cl-xbp6q: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine o0s0h4b0m0k9t7l-np-delete-main-sj9cl-hlwvx: Creating: virtualmachine creating or updating\nMachine o0s0h4b0m0k9t7l-np-delete-main-sj9cl-xbp6q: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... ERROR CODE: InvalidRequestContent; detail message Invalid value: "4.21": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is "nightly"; provider Microsoft.RedHatOpenShift 23:00:45 UTC · failed at e2e · test Authorized CIDRs Connectivity should allow API access only from authorized VM IP · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:408]: failed to begin updating cluster "cidr-connectivity-test" authorized CIDRs Unexpected error: <exported.ResponseError \| 0x336e50b8a360>: PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-njsbz992h4xx/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cidr-connectivity-test -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"4.21\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"", "target": "properties.version.id" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:408]: failed to begin updating cluster "cidr-connectivity-test" authorized CIDRs Unexpected error: <exported.ResponseError \| 0x336e50b8a360>: PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-njsbz992h4xx/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cidr-connectivity-test -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"4.21\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"", "target": "properties.version.id" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 23:00:45 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP nightly channel for 4.23 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-hz7c47" for cluster "cluster-nightly-4-23-hz7c47" Unexpected error: <fmt.wrapError \| 0x33bfacf01ac0>: failed to create NodePool np-hz7c47: failed waiting for nodepool="np-hz7c47" for cluster "cluster-nightly-4-23-hz7c47" in resourcegroup="rg-nightly-4-23-zskzh2wkzsf8" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/ea1a45b2-8ea6-4372-9874-0ab41e3da185 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/ea1a45b2-8ea6-4372-9874-0ab41e3da185", "name": "ea1a45b2-8ea6-4372-9874-0ab41e3da185", "status": "Failed", "startTime": "2026-10-03T23:36:57.511630158Z", "endTime": "2026-10-03T23:56:00.189645328Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine p8n6w8u1n2s0m9t-np-hz7c47-xs76h-hmv9g: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine p8n6w8u1n2s0m9t-np-hz7c47-xs76h-qxrgm: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine p8n6w8u1n2s0m9t-np-hz7c47-xs76h-hmv9g: Creating: virtualmachine creating or updating\nMachine p8n6w8u1n2s0m9t-np-hz7c47-xs76h-qxrgm: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-hz7c47" for cluster "cluster-nightly-4-23-hz7c47" Unexpected error: <fmt.wrapError \| 0x33bfacf01ac0>: failed to create NodePool np-hz7c47: failed waiting for nodepool="np-hz7c47" for cluster "cluster-nightly-4-23-hz7c47" in resourcegroup="rg-nightly-4-23-zskzh2wkzsf8" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/ea1a45b2-8ea6-4372-9874-0ab41e3da185 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/ea1a45b2-8ea6-4372-9874-0ab41e3da185", "name": "ea1a45b2-8ea6-4372-9874-0ab41e3da185", "status": "Failed", "startTime": "2026-10-03T23:36:57.511630158Z", "endTime": "2026-10-03T23:56:00.189645328Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine p8n6w8u1n2s0m9t-np-hz7c47-xs76h-hmv9g: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine p8n6w8u1n2s0m9t-np-hz7c47-xs76h-qxrgm: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine p8n6w8u1n2s0m9t-np-hz7c47-xs76h-hmv9g: Creating: virtualmachine creating or updating\nMachine p8n6w8u1n2s0m9t-np-hz7c47-xs76h-qxrgm: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred
02:51:08	1h 4m 0s	branch-ci-Azure-ARO-HCP-main-e2e-stage-e2e-parallel post-goodmerged PR No additional run metadata captured	n/a	unknown	failed	n/a	1	non-artifact-backed failure (no junit artifacts)
02:00:44	2h 44m 47s	periodic-ci-Azure-ARO-HCP-main-periodic-stage-e2e-parallel post-goodmerged PR No additional run metadata captured	uksouth	e2e	failed	n/a	11	e2e (11) ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 02:00:44 UTC · failed at e2e · test Customer should create a cluster with private KAS and verify API server is only reachable from VNet while ingress remains public · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:159]: failed to create node pool "np-1" for private KAS cluster "private-kas" Unexpected error: <fmt.wrapError \| 0x1634a92060e0>: failed to create NodePool np-1: failed waiting for nodepool="np-1" for cluster "private-kas" in resourcegroup="private-kas-5tmnm5zpfp6k" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/2872059a-968a-400a-8256-03392979308f -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/2872059a-968a-400a-8256-03392979308f", "name": "2872059a-968a-400a-8256-03392979308f", "status": "Failed", "startTime": "2026-10-03T02:59:17.985754909Z", "endTime": "2026-10-03T03:18:20.080261329Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 1; node pool condition AllNodesHealthy is False: 1 of 1 machines are not healthy\nMachine private-kas-np-1-x5mt5-q58ts: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 1 of 1 machines are not ready\nMachine private-kas-np-1-x5mt5-q58ts: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:159]: failed to create node pool "np-1" for private KAS cluster "private-kas" Unexpected error: <fmt.wrapError \| 0x1634a92060e0>: failed to create NodePool np-1: failed waiting for nodepool="np-1" for cluster "private-kas" in resourcegroup="private-kas-5tmnm5zpfp6k" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/2872059a-968a-400a-8256-03392979308f -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/2872059a-968a-400a-8256-03392979308f", "name": "2872059a-968a-400a-8256-03392979308f", "status": "Failed", "startTime": "2026-10-03T02:59:17.985754909Z", "endTime": "2026-10-03T03:18:20.080261329Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 1; node pool condition AllNodesHealthy is False: 1 of 1 machines are not healthy\nMachine private-kas-np-1-x5mt5-q58ts: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 1 of 1 machines are not ready\nMachine private-kas-np-1-x5mt5-q58ts: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 02:00:44 UTC · failed at e2e · test Customer should be able to create a no-CNI private cluster with a private key vault, a nodepool and install cilium CNI successfully · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:193]: failed to verify nodes are Ready with Cilium CNI for cluster "cilium-cluster" Unexpected error: <errors.joinError \| 0x1c6f23908030>: VerifyNodesReady failed: no nodes found in the cluster VerifyCiliumOperational failed: not all pods in kube-system namespace are running: failed to list pods in kube-system namespace: client rate limiter Wait returned an error: context deadline exceeded failed to create NodePool cilium-np: failed waiting for nodepool="cilium-np" for cluster "cilium-cluster" in resourcegroup="complex-cilium-kv-ddfqq694gxhd" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/bea67035-4bd4-49aa-97d8-eb0a71fb82dc -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/bea67035-4bd4-49aa-97d8-eb0a71fb82dc", "name": "bea67035-4bd4-49aa-97d8-eb0a71fb82dc", "status": "Failed", "startTime": "2026-10-03T02:59:14.55416176Z", "endTime": "2026-10-03T03:18:19.785989613Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine cilium-cluster-cilium-np-7d6m6-8kqhg: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine cilium-cluster-cilium-np-7d6m6-ptnv4: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine cilium-cluster-cilium-np-7d6m6-8kqhg: Creating: virtualmachine creating or updating\nMachine cilium-cluster-cilium-np-7d6m6-ptnv4: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:193]: failed to verify nodes are Ready with Cilium CNI for cluster "cilium-cluster" Unexpected error: <errors.joinError \| 0x1c6f23908030>: VerifyNodesReady failed: no nodes found in the cluster VerifyCiliumOperational failed: not all pods in kube-system namespace are running: failed to list pods in kube-system namespace: client rate limiter Wait returned an error: context deadline exceeded failed to create NodePool cilium-np: failed waiting for nodepool="cilium-np" for cluster "cilium-cluster" in resourcegroup="complex-cilium-kv-ddfqq694gxhd" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/bea67035-4bd4-49aa-97d8-eb0a71fb82dc -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/bea67035-4bd4-49aa-97d8-eb0a71fb82dc", "name": "bea67035-4bd4-49aa-97d8-eb0a71fb82dc", "status": "Failed", "startTime": "2026-10-03T02:59:14.55416176Z", "endTime": "2026-10-03T03:18:19.785989613Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine cilium-cluster-cilium-np-7d6m6-8kqhg: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine cilium-cluster-cilium-np-7d6m6-ptnv4: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine cilium-cluster-cilium-np-7d6m6-8kqhg: Creating: virtualmachine creating or updating\nMachine cilium-cluster-cilium-np-7d6m6-ptnv4: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 02:00:44 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 5.1 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-wgvn44" for cluster "cluster-candidate-5-1-wgvn44" Unexpected error: <fmt.wrapError \| 0x582bdd8e0c0>: failed to create NodePool np-wgvn44: failed waiting for nodepool="np-wgvn44" for cluster "cluster-candidate-5-1-wgvn44" in resourcegroup="rg-candidate-5-1-g5g8pnhdz6gx" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/7b96711e-b171-48d1-87d7-1f055025bc87 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/7b96711e-b171-48d1-87d7-1f055025bc87", "name": "7b96711e-b171-48d1-87d7-1f055025bc87", "status": "Failed", "startTime": "2026-10-03T03:24:01.084946017Z", "endTime": "2026-10-03T03:43:03.38976371Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine j6u5b3c2z9q1l8v-np-wgvn44-b9dp7-5mgvm: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine j6u5b3c2z9q1l8v-np-wgvn44-b9dp7-vkmcn: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine j6u5b3c2z9q1l8v-np-wgvn44-b9dp7-5mgvm: Creating: virtualmachine creating or updating\nMachine j6u5b3c2z9q1l8v-np-wgvn44-b9dp7-vkmcn: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-wgvn44" for cluster "cluster-candidate-5-1-wgvn44" Unexpected error: <fmt.wrapError \| 0x582bdd8e0c0>: failed to create NodePool np-wgvn44: failed waiting for nodepool="np-wgvn44" for cluster "cluster-candidate-5-1-wgvn44" in resourcegroup="rg-candidate-5-1-g5g8pnhdz6gx" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/7b96711e-b171-48d1-87d7-1f055025bc87 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/7b96711e-b171-48d1-87d7-1f055025bc87", "name": "7b96711e-b171-48d1-87d7-1f055025bc87", "status": "Failed", "startTime": "2026-10-03T03:24:01.084946017Z", "endTime": "2026-10-03T03:43:03.38976371Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine j6u5b3c2z9q1l8v-np-wgvn44-b9dp7-5mgvm: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine j6u5b3c2z9q1l8v-np-wgvn44-b9dp7-vkmcn: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine j6u5b3c2z9q1l8v-np-wgvn44-b9dp7-5mgvm: Creating: virtualmachine creating or updating\nMachine j6u5b3c2z9q1l8v-np-wgvn44-b9dp7-vkmcn: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 02:00:44 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 5.0 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-994gtb" for cluster "cluster-candidate-5-0-994gtb" Unexpected error: <fmt.wrapError \| 0x3b59f6a200a0>: failed to create NodePool np-994gtb: failed waiting for nodepool="np-994gtb" for cluster "cluster-candidate-5-0-994gtb" in resourcegroup="rg-candidate-5-0-dpqs94628k6g" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/f3ae09fa-a5f4-44d2-9f00-68063627145c -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/f3ae09fa-a5f4-44d2-9f00-68063627145c", "name": "f3ae09fa-a5f4-44d2-9f00-68063627145c", "status": "Failed", "startTime": "2026-10-03T03:22:48.672584866Z", "endTime": "2026-10-03T03:41:53.397092955Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine w0a5p3m7x2i2z6g-np-994gtb-827pv-9xspx: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine w0a5p3m7x2i2z6g-np-994gtb-827pv-v8vvg: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine w0a5p3m7x2i2z6g-np-994gtb-827pv-9xspx: Creating: virtualmachine creating or updating\nMachine w0a5p3m7x2i2z6g-np-994gtb-827pv-v8vvg: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-994gtb" for cluster "cluster-candidate-5-0-994gtb" Unexpected error: <fmt.wrapError \| 0x3b59f6a200a0>: failed to create NodePool np-994gtb: failed waiting for nodepool="np-994gtb" for cluster "cluster-candidate-5-0-994gtb" in resourcegroup="rg-candidate-5-0-dpqs94628k6g" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/f3ae09fa-a5f4-44d2-9f00-68063627145c -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/f3ae09fa-a5f4-44d2-9f00-68063627145c", "name": "f3ae09fa-a5f4-44d2-9f00-68063627145c", "status": "Failed", "startTime": "2026-10-03T03:22:48.672584866Z", "endTime": "2026-10-03T03:41:53.397092955Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine w0a5p3m7x2i2z6g-np-994gtb-827pv-9xspx: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine w0a5p3m7x2i2z6g-np-994gtb-827pv-v8vvg: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine w0a5p3m7x2i2z6g-np-994gtb-827pv-9xspx: Creating: virtualmachine creating or updating\nMachine w0a5p3m7x2i2z6g-np-994gtb-827pv-v8vvg: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 02:00:44 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.22 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-zsxxnl" for cluster "cluster-candidate-4-22-zsxxnl" Unexpected error: <fmt.wrapError \| 0x8479408b140>: failed to create NodePool np-zsxxnl: failed waiting for nodepool="np-zsxxnl" for cluster "cluster-candidate-4-22-zsxxnl" in resourcegroup="rg-candidate-4-22-7822rkz4qt8d" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/9fd0879a-ebbb-4fea-b214-ece8494edab0 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/9fd0879a-ebbb-4fea-b214-ece8494edab0", "name": "9fd0879a-ebbb-4fea-b214-ece8494edab0", "status": "Failed", "startTime": "2026-10-03T03:22:25.37619508Z", "endTime": "2026-10-03T03:41:30.020299253Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine r5g9o6x4t5g2e7e-np-zsxxnl-wkz99-6hmr2: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine r5g9o6x4t5g2e7e-np-zsxxnl-wkz99-b6wrl: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine r5g9o6x4t5g2e7e-np-zsxxnl-wkz99-6hmr2: Creating: virtualmachine creating or updating\nMachine r5g9o6x4t5g2e7e-np-zsxxnl-wkz99-b6wrl: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-zsxxnl" for cluster "cluster-candidate-4-22-zsxxnl" Unexpected error: <fmt.wrapError \| 0x8479408b140>: failed to create NodePool np-zsxxnl: failed waiting for nodepool="np-zsxxnl" for cluster "cluster-candidate-4-22-zsxxnl" in resourcegroup="rg-candidate-4-22-7822rkz4qt8d" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/9fd0879a-ebbb-4fea-b214-ece8494edab0 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/9fd0879a-ebbb-4fea-b214-ece8494edab0", "name": "9fd0879a-ebbb-4fea-b214-ece8494edab0", "status": "Failed", "startTime": "2026-10-03T03:22:25.37619508Z", "endTime": "2026-10-03T03:41:30.020299253Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine r5g9o6x4t5g2e7e-np-zsxxnl-wkz99-6hmr2: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine r5g9o6x4t5g2e7e-np-zsxxnl-wkz99-b6wrl: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine r5g9o6x4t5g2e7e-np-zsxxnl-wkz99-6hmr2: Creating: virtualmachine creating or updating\nMachine r5g9o6x4t5g2e7e-np-zsxxnl-wkz99-b6wrl: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred failed to create nodepool <nodepool>: failed waiting for nodepool="<nodepool>" for cluster "<cluster>" in resourcegroup="<resource-group>" to finish creating: GET <url> 02:00:44 UTC · failed at e2e · test Customer should be able to delete a nodepool · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_delete.go:128]: failed to create node pools Expected success, but got an error: <fmt.wrapError \| 0x933c5f12a60>: failed to create NodePool np-delete-main: failed waiting for nodepool="np-delete-main" for cluster "np-delete-hcp-cluster" in resourcegroup="nodepool-delete-lbkmsp45kpxz" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/efe85861-3de4-4231-94f4-c87043439307 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/efe85861-3de4-4231-94f4-c87043439307", "name": "efe85861-3de4-4231-94f4-c87043439307", "status": "Failed", "startTime": "2026-10-03T03:54:52.260057343Z", "endTime": "2026-10-03T04:13:53.608229594Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine k7l8y6q4q2l9i3h-np-delete-main-68dll-hcvpc: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine k7l8y6q4q2l9i3h-np-delete-main-68dll-mgh9q: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine k7l8y6q4q2l9i3h-np-delete-main-68dll-hcvpc: Creating: virtualmachine creating or updating\nMachine k7l8y6q4q2l9i3h-np-delete-main-68dll-mgh9q: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_delete.go:128]: failed to create node pools Expected success, but got an error: <fmt.wrapError \| 0x933c5f12a60>: failed to create NodePool np-delete-main: failed waiting for nodepool="np-delete-main" for cluster "np-delete-hcp-cluster" in resourcegroup="nodepool-delete-lbkmsp45kpxz" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/efe85861-3de4-4231-94f4-c87043439307 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/efe85861-3de4-4231-94f4-c87043439307", "name": "efe85861-3de4-4231-94f4-c87043439307", "status": "Failed", "startTime": "2026-10-03T03:54:52.260057343Z", "endTime": "2026-10-03T04:13:53.608229594Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine k7l8y6q4q2l9i3h-np-delete-main-68dll-hcvpc: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine k7l8y6q4q2l9i3h-np-delete-main-68dll-mgh9q: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine k7l8y6q4q2l9i3h-np-delete-main-68dll-hcvpc: Creating: virtualmachine creating or updating\nMachine k7l8y6q4q2l9i3h-np-delete-main-68dll-mgh9q: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift node pool not cached yet; provider Microsoft.RedHatOpenShift 02:00:44 UTC · failed at e2e · test Customer should be able to create a cluster and node pools with aggregated advanced features · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:404]: failed to verify cilium is running and nodes are Ready for cluster "agg-cluster" Unexpected error: <errors.joinError \| 0xe7412ec40a8>: VerifyNodesReady failed: no nodes found in the cluster VerifyCiliumOperational failed: not all pods in kube-system namespace are running: failed to list pods in kube-system namespace: client rate limiter Wait returned an error: context deadline exceeded failed to create NodePool agg-np-a: failed waiting for nodepool="agg-np-a" for cluster "agg-cluster" in resourcegroup="feature-aggregation-j2962r94fd6x" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4588d6e6-e963-44fd-8b18-89f83b2a899d -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4588d6e6-e963-44fd-8b18-89f83b2a899d", "name": "4588d6e6-e963-44fd-8b18-89f83b2a899d", "status": "Failed", "startTime": "2026-10-03T03:00:19.581988952Z", "endTime": "2026-10-03T03:19:19.770602537Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift node pool not cached yet" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:404]: failed to verify cilium is running and nodes are Ready for cluster "agg-cluster" Unexpected error: <errors.joinError \| 0xe7412ec40a8>: VerifyNodesReady failed: no nodes found in the cluster VerifyCiliumOperational failed: not all pods in kube-system namespace are running: failed to list pods in kube-system namespace: client rate limiter Wait returned an error: context deadline exceeded failed to create NodePool agg-np-a: failed waiting for nodepool="agg-np-a" for cluster "agg-cluster" in resourcegroup="feature-aggregation-j2962r94fd6x" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4588d6e6-e963-44fd-8b18-89f83b2a899d -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4588d6e6-e963-44fd-8b18-89f83b2a899d", "name": "4588d6e6-e963-44fd-8b18-89f83b2a899d", "status": "Failed", "startTime": "2026-10-03T03:00:19.581988952Z", "endTime": "2026-10-03T03:19:19.770602537Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift node pool not cached yet" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 02:00:44 UTC · failed at e2e · test Create HCPOpenShiftCluster with Private KeyVault should create a cluster with private keyvault using v20251223preview API · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:138]: failed to create node pool "np-1" for private keyvault cluster "private-kv-cluster" Unexpected error: <fmt.wrapError \| 0x21c259b32dc0>: failed to create NodePool np-1: failed waiting for nodepool="np-1" for cluster "private-kv-cluster" in resourcegroup="private-keyvault-2sgbstf7nlxj" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/3898db04-618a-4491-92e8-a44483e69093 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/3898db04-618a-4491-92e8-a44483e69093", "name": "3898db04-618a-4491-92e8-a44483e69093", "status": "Failed", "startTime": "2026-10-03T02:58:32.753298506Z", "endTime": "2026-10-03T03:17:39.788365749Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine m5r0r0n9t0l1p2d-np-1-dr752-hbt2g: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine m5r0r0n9t0l1p2d-np-1-dr752-zpxs2: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine m5r0r0n9t0l1p2d-np-1-dr752-hbt2g: Creating: virtualmachine creating or updating\nMachine m5r0r0n9t0l1p2d-np-1-dr752-zpxs2: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:138]: failed to create node pool "np-1" for private keyvault cluster "private-kv-cluster" Unexpected error: <fmt.wrapError \| 0x21c259b32dc0>: failed to create NodePool np-1: failed waiting for nodepool="np-1" for cluster "private-kv-cluster" in resourcegroup="private-keyvault-2sgbstf7nlxj" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/3898db04-618a-4491-92e8-a44483e69093 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/3898db04-618a-4491-92e8-a44483e69093", "name": "3898db04-618a-4491-92e8-a44483e69093", "status": "Failed", "startTime": "2026-10-03T02:58:32.753298506Z", "endTime": "2026-10-03T03:17:39.788365749Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine m5r0r0n9t0l1p2d-np-1-dr752-hbt2g: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine m5r0r0n9t0l1p2d-np-1-dr752-zpxs2: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine m5r0r0n9t0l1p2d-np-1-dr752-hbt2g: Creating: virtualmachine creating or updating\nMachine m5r0r0n9t0l1p2d-np-1-dr752-zpxs2: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; provider Microsoft.Compute 02:00:44 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.21 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-6tqvsp" for cluster "cluster-candidate-4-21-6tqvsp" Unexpected error: <fmt.wrapError \| 0x15f1e13c9ee0>: failed to create NodePool np-6tqvsp: failed waiting for nodepool="np-6tqvsp" for cluster "cluster-candidate-4-21-6tqvsp" in resourcegroup="rg-candidate-4-21-qxp54ksqdkvb" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/cd90e427-e90a-4449-9d9c-98919a3e2323 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/cd90e427-e90a-4449-9d9c-98919a3e2323", "name": "cd90e427-e90a-4449-9d9c-98919a3e2323", "status": "Failed", "startTime": "2026-10-03T03:23:37.870437581Z", "endTime": "2026-10-03T03:42:43.397028044Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine y1i0m6w3c6r9w2i-np-6tqvsp-mg8xh-88lr5: InspectionFailed: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-4-21-qxp54ksqdkvb-candidate-6tqvsp--managed/providers/Microsoft.Compute/virtualMachines/y1i0m6w3c6r9w2i-np-6tqvsp-mg8xh-88lr5 to exist\nMachine y1i0m6w3c6r9w2i-np-6tqvsp-mg8xh-cm8m4: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine y1i0m6w3c6r9w2i-np-6tqvsp-mg8xh-cm8m4: Creating: virtualmachine creating or updating\nMachine y1i0m6w3c6r9w2i-np-6tqvsp-mg8xh-88lr5: ReadyUnknown: NodeHealthy: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-4-21-qxp54ksqdkvb-candidate-6tqvsp--managed/providers/Microsoft.Compute/virtualMachines/y1i0m6w3c6r9w2i-np-6tqvsp-mg8xh-88lr5 to exist\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-6tqvsp" for cluster "cluster-candidate-4-21-6tqvsp" Unexpected error: <fmt.wrapError \| 0x15f1e13c9ee0>: failed to create NodePool np-6tqvsp: failed waiting for nodepool="np-6tqvsp" for cluster "cluster-candidate-4-21-6tqvsp" in resourcegroup="rg-candidate-4-21-qxp54ksqdkvb" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/cd90e427-e90a-4449-9d9c-98919a3e2323 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/cd90e427-e90a-4449-9d9c-98919a3e2323", "name": "cd90e427-e90a-4449-9d9c-98919a3e2323", "status": "Failed", "startTime": "2026-10-03T03:23:37.870437581Z", "endTime": "2026-10-03T03:42:43.397028044Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine y1i0m6w3c6r9w2i-np-6tqvsp-mg8xh-88lr5: InspectionFailed: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-4-21-qxp54ksqdkvb-candidate-6tqvsp--managed/providers/Microsoft.Compute/virtualMachines/y1i0m6w3c6r9w2i-np-6tqvsp-mg8xh-88lr5 to exist\nMachine y1i0m6w3c6r9w2i-np-6tqvsp-mg8xh-cm8m4: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine y1i0m6w3c6r9w2i-np-6tqvsp-mg8xh-cm8m4: Creating: virtualmachine creating or updating\nMachine y1i0m6w3c6r9w2i-np-6tqvsp-mg8xh-88lr5: ReadyUnknown: NodeHealthy: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-4-21-qxp54ksqdkvb-candidate-6tqvsp--managed/providers/Microsoft.Compute/virtualMachines/y1i0m6w3c6r9w2i-np-6tqvsp-mg8xh-88lr5 to exist\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 02:00:44 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.23 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-vlg9bq" for cluster "cluster-candidate-4-23-vlg9bq" Unexpected error: <fmt.wrapError \| 0x1e6e62f6cda0>: failed to create NodePool np-vlg9bq: failed waiting for nodepool="np-vlg9bq" for cluster "cluster-candidate-4-23-vlg9bq" in resourcegroup="rg-candidate-4-23-dfclsfklgxtx" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/98169a27-755e-48b1-800f-32bc5e94d96c -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/98169a27-755e-48b1-800f-32bc5e94d96c", "name": "98169a27-755e-48b1-800f-32bc5e94d96c", "status": "Failed", "startTime": "2026-10-03T03:22:35.181930151Z", "endTime": "2026-10-03T03:41:39.859849458Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine g4t5b9n2y6n5g3x-np-vlg9bq-nplgj-mqggg: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine g4t5b9n2y6n5g3x-np-vlg9bq-nplgj-zfwnx: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine g4t5b9n2y6n5g3x-np-vlg9bq-nplgj-mqggg: Creating: virtualmachine creating or updating\nMachine g4t5b9n2y6n5g3x-np-vlg9bq-nplgj-zfwnx: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-vlg9bq" for cluster "cluster-candidate-4-23-vlg9bq" Unexpected error: <fmt.wrapError \| 0x1e6e62f6cda0>: failed to create NodePool np-vlg9bq: failed waiting for nodepool="np-vlg9bq" for cluster "cluster-candidate-4-23-vlg9bq" in resourcegroup="rg-candidate-4-23-dfclsfklgxtx" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/98169a27-755e-48b1-800f-32bc5e94d96c -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/98169a27-755e-48b1-800f-32bc5e94d96c", "name": "98169a27-755e-48b1-800f-32bc5e94d96c", "status": "Failed", "startTime": "2026-10-03T03:22:35.181930151Z", "endTime": "2026-10-03T03:41:39.859849458Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine g4t5b9n2y6n5g3x-np-vlg9bq-nplgj-mqggg: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine g4t5b9n2y6n5g3x-np-vlg9bq-nplgj-zfwnx: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine g4t5b9n2y6n5g3x-np-vlg9bq-nplgj-mqggg: Creating: virtualmachine creating or updating\nMachine g4t5b9n2y6n5g3x-np-vlg9bq-nplgj-zfwnx: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 02:00:44 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.20 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-7k8q7d" for cluster "cluster-candidate-4-20-7k8q7d" Unexpected error: <fmt.wrapError \| 0x342eb43e6c40>: failed to create NodePool np-7k8q7d: failed waiting for nodepool="np-7k8q7d" for cluster "cluster-candidate-4-20-7k8q7d" in resourcegroup="rg-candidate-4-20-vn2gzb9bgx68" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/8f5da487-7370-42cc-8931-634a89f3149c -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/8f5da487-7370-42cc-8931-634a89f3149c", "name": "8f5da487-7370-42cc-8931-634a89f3149c", "status": "Failed", "startTime": "2026-10-03T03:18:20.263677419Z", "endTime": "2026-10-03T03:37:30.013101747Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine n2s3g3y6j4i7c5t-np-7k8q7d-vrbbm-rlpsx: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine n2s3g3y6j4i7c5t-np-7k8q7d-vrbbm-v5gzq: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine n2s3g3y6j4i7c5t-np-7k8q7d-vrbbm-rlpsx: Creating: virtualmachine creating or updating\nMachine n2s3g3y6j4i7c5t-np-7k8q7d-vrbbm-v5gzq: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-7k8q7d" for cluster "cluster-candidate-4-20-7k8q7d" Unexpected error: <fmt.wrapError \| 0x342eb43e6c40>: failed to create NodePool np-7k8q7d: failed waiting for nodepool="np-7k8q7d" for cluster "cluster-candidate-4-20-7k8q7d" in resourcegroup="rg-candidate-4-20-vn2gzb9bgx68" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/8f5da487-7370-42cc-8931-634a89f3149c -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/8f5da487-7370-42cc-8931-634a89f3149c", "name": "8f5da487-7370-42cc-8931-634a89f3149c", "status": "Failed", "startTime": "2026-10-03T03:18:20.263677419Z", "endTime": "2026-10-03T03:37:30.013101747Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine n2s3g3y6j4i7c5t-np-7k8q7d-vrbbm-rlpsx: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine n2s3g3y6j4i7c5t-np-7k8q7d-vrbbm-v5gzq: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine n2s3g3y6j4i7c5t-np-7k8q7d-vrbbm-rlpsx: Creating: virtualmachine creating or updating\nMachine n2s3g3y6j4i7c5t-np-7k8q7d-vrbbm-v5gzq: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred
00:25:08	2h 18m 49s	branch-ci-Azure-ARO-HCP-main-e2e-stage-e2e-parallel post-goodmerged PR No additional run metadata captured	uksouth	e2e	failed	n/a	12	e2e (12) failed to create nodepool <nodepool>: failed waiting for nodepool="<nodepool>" for cluster "<cluster>" in resourcegroup="<resource-group>" to finish creating: GET <url> 00:25:08 UTC · failed at e2e · test Customer should be able to delete a nodepool · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_delete.go:128]: failed to create node pools Expected success, but got an error: <fmt.wrapError \| 0xca0cd836f80>: failed to create NodePool np-deletable: failed waiting for nodepool="np-deletable" for cluster "np-delete-hcp-cluster" in resourcegroup="nodepool-delete-xrjvzm8fxrgt" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4e0749eb-04f8-4fdd-b9be-8393584f3697 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4e0749eb-04f8-4fdd-b9be-8393584f3697", "name": "4e0749eb-04f8-4fdd-b9be-8393584f3697", "status": "Failed", "startTime": "2026-10-03T01:51:49.926221366Z", "endTime": "2026-10-03T02:10:56.26372719Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine m9l6j7a1w6x8b0s-np-deletable-z98x4-45vn5: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine m9l6j7a1w6x8b0s-np-deletable-z98x4-bswmx: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine m9l6j7a1w6x8b0s-np-deletable-z98x4-45vn5: Creating: virtualmachine creating or updating\nMachine m9l6j7a1w6x8b0s-np-deletable-z98x4-bswmx: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_delete.go:128]: failed to create node pools Expected success, but got an error: <fmt.wrapError \| 0xca0cd836f80>: failed to create NodePool np-deletable: failed waiting for nodepool="np-deletable" for cluster "np-delete-hcp-cluster" in resourcegroup="nodepool-delete-xrjvzm8fxrgt" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4e0749eb-04f8-4fdd-b9be-8393584f3697 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4e0749eb-04f8-4fdd-b9be-8393584f3697", "name": "4e0749eb-04f8-4fdd-b9be-8393584f3697", "status": "Failed", "startTime": "2026-10-03T01:51:49.926221366Z", "endTime": "2026-10-03T02:10:56.26372719Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine m9l6j7a1w6x8b0s-np-deletable-z98x4-45vn5: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine m9l6j7a1w6x8b0s-np-deletable-z98x4-bswmx: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine m9l6j7a1w6x8b0s-np-deletable-z98x4-45vn5: Creating: virtualmachine creating or updating\nMachine m9l6j7a1w6x8b0s-np-deletable-z98x4-bswmx: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want >= <count> (autoscaling min); node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 00:25:08 UTC · failed at e2e · test Customer should be able to create a cluster and node pools with aggregated advanced features · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:404]: failed to verify cilium is running and nodes are Ready for cluster "agg-cluster" Unexpected error: <errors.joinError \| 0x1363e14cc018>: VerifyNodesReady failed: no nodes found in the cluster VerifyCiliumOperational failed: not all pods in kube-system namespace are running: failed to list pods in kube-system namespace: client rate limiter Wait returned an error: context deadline exceeded failed to create NodePool agg-np-a: failed waiting for nodepool="agg-np-a" for cluster "agg-cluster" in resourcegroup="feature-aggregation-jtmvgv9m6829" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/31894849-27e9-4053-a67e-9e3babf2ad8a -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/31894849-27e9-4053-a67e-9e3babf2ad8a", "name": "31894849-27e9-4053-a67e-9e3babf2ad8a", "status": "Failed", "startTime": "2026-10-03T01:01:05.764058109Z", "endTime": "2026-10-03T01:20:12.852653102Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want \u003e= 2 (autoscaling min); node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine agg-cluster-agg-np-a-wxgnc-9w4jt: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine agg-cluster-agg-np-a-wxgnc-clcm7: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine agg-cluster-agg-np-a-wxgnc-9w4jt: Creating: virtualmachine creating or updating\nMachine agg-cluster-agg-np-a-wxgnc-clcm7: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:404]: failed to verify cilium is running and nodes are Ready for cluster "agg-cluster" Unexpected error: <errors.joinError \| 0x1363e14cc018>: VerifyNodesReady failed: no nodes found in the cluster VerifyCiliumOperational failed: not all pods in kube-system namespace are running: failed to list pods in kube-system namespace: client rate limiter Wait returned an error: context deadline exceeded failed to create NodePool agg-np-a: failed waiting for nodepool="agg-np-a" for cluster "agg-cluster" in resourcegroup="feature-aggregation-jtmvgv9m6829" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/31894849-27e9-4053-a67e-9e3babf2ad8a -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/31894849-27e9-4053-a67e-9e3babf2ad8a", "name": "31894849-27e9-4053-a67e-9e3babf2ad8a", "status": "Failed", "startTime": "2026-10-03T01:01:05.764058109Z", "endTime": "2026-10-03T01:20:12.852653102Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want \u003e= 2 (autoscaling min); node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine agg-cluster-agg-np-a-wxgnc-9w4jt: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine agg-cluster-agg-np-a-wxgnc-clcm7: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine agg-cluster-agg-np-a-wxgnc-9w4jt: Creating: virtualmachine creating or updating\nMachine agg-cluster-agg-np-a-wxgnc-clcm7: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 00:25:08 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.21 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-7m65d4" for cluster "cluster-candidate-4-21-7m65d4" Unexpected error: <fmt.wrapError \| 0xdd74a91040>: failed to create NodePool np-7m65d4: failed waiting for nodepool="np-7m65d4" for cluster "cluster-candidate-4-21-7m65d4" in resourcegroup="rg-candidate-4-21-cdkldcpn7c4w" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/f45b9d58-6bb8-408f-943e-164ec6193300 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/f45b9d58-6bb8-408f-943e-164ec6193300", "name": "f45b9d58-6bb8-408f-943e-164ec6193300", "status": "Failed", "startTime": "2026-10-03T01:23:06.848197915Z", "endTime": "2026-10-03T01:42:11.810063991Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine z8t8u8s2s4m5t9g-np-7m65d4-6msht-7jtvb: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine z8t8u8s2s4m5t9g-np-7m65d4-6msht-8fd6d: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine z8t8u8s2s4m5t9g-np-7m65d4-6msht-7jtvb: Creating: virtualmachine creating or updating\nMachine z8t8u8s2s4m5t9g-np-7m65d4-6msht-8fd6d: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-7m65d4" for cluster "cluster-candidate-4-21-7m65d4" Unexpected error: <fmt.wrapError \| 0xdd74a91040>: failed to create NodePool np-7m65d4: failed waiting for nodepool="np-7m65d4" for cluster "cluster-candidate-4-21-7m65d4" in resourcegroup="rg-candidate-4-21-cdkldcpn7c4w" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/f45b9d58-6bb8-408f-943e-164ec6193300 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/f45b9d58-6bb8-408f-943e-164ec6193300", "name": "f45b9d58-6bb8-408f-943e-164ec6193300", "status": "Failed", "startTime": "2026-10-03T01:23:06.848197915Z", "endTime": "2026-10-03T01:42:11.810063991Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine z8t8u8s2s4m5t9g-np-7m65d4-6msht-7jtvb: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine z8t8u8s2s4m5t9g-np-7m65d4-6msht-8fd6d: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine z8t8u8s2s4m5t9g-np-7m65d4-6msht-7jtvb: Creating: virtualmachine creating or updating\nMachine z8t8u8s2s4m5t9g-np-7m65d4-6msht-8fd6d: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 00:25:08 UTC · failed at e2e · test Customer should create a cluster with private KAS and verify API server is only reachable from VNet while ingress remains public · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:159]: failed to create node pool "np-1" for private KAS cluster "private-kas" Unexpected error: <fmt.wrapError \| 0x356eae117080>: failed to create NodePool np-1: failed waiting for nodepool="np-1" for cluster "private-kas" in resourcegroup="private-kas-2xqzzh8h7jx6" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/c2692539-c900-4d9b-9952-7a76ad7fc6a0 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/c2692539-c900-4d9b-9952-7a76ad7fc6a0", "name": "c2692539-c900-4d9b-9952-7a76ad7fc6a0", "status": "Failed", "startTime": "2026-10-03T00:56:20.964200965Z", "endTime": "2026-10-03T01:15:22.786636188Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 1; node pool condition AllNodesHealthy is False: 1 of 1 machines are not healthy\nMachine private-kas-np-1-kbv8l-5qvk6: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 1 of 1 machines are not ready\nMachine private-kas-np-1-kbv8l-5qvk6: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:159]: failed to create node pool "np-1" for private KAS cluster "private-kas" Unexpected error: <fmt.wrapError \| 0x356eae117080>: failed to create NodePool np-1: failed waiting for nodepool="np-1" for cluster "private-kas" in resourcegroup="private-kas-2xqzzh8h7jx6" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/c2692539-c900-4d9b-9952-7a76ad7fc6a0 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/c2692539-c900-4d9b-9952-7a76ad7fc6a0", "name": "c2692539-c900-4d9b-9952-7a76ad7fc6a0", "status": "Failed", "startTime": "2026-10-03T00:56:20.964200965Z", "endTime": "2026-10-03T01:15:22.786636188Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 1; node pool condition AllNodesHealthy is False: 1 of 1 machines are not healthy\nMachine private-kas-np-1-kbv8l-5qvk6: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 1 of 1 machines are not ready\nMachine private-kas-np-1-kbv8l-5qvk6: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 00:25:08 UTC · failed at e2e · test Create HCPOpenShiftCluster with Private KeyVault should create a cluster with private keyvault using v20251223preview API · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:138]: failed to create node pool "np-1" for private keyvault cluster "private-kv-cluster" Unexpected error: <fmt.wrapError \| 0x36891cc75140>: failed to create NodePool np-1: failed waiting for nodepool="np-1" for cluster "private-kv-cluster" in resourcegroup="private-keyvault-rw4pgrx2qsqc" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/a719f074-9cbb-4f60-8f74-9acd323de195 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/a719f074-9cbb-4f60-8f74-9acd323de195", "name": "a719f074-9cbb-4f60-8f74-9acd323de195", "status": "Failed", "startTime": "2026-10-03T00:58:50.620270926Z", "endTime": "2026-10-03T01:17:53.07846241Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine y4c5l7p2o7h6i3y-np-1-7slbg-nmjht: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine y4c5l7p2o7h6i3y-np-1-7slbg-qnrnm: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine y4c5l7p2o7h6i3y-np-1-7slbg-nmjht: Creating: virtualmachine creating or updating\nMachine y4c5l7p2o7h6i3y-np-1-7slbg-qnrnm: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:138]: failed to create node pool "np-1" for private keyvault cluster "private-kv-cluster" Unexpected error: <fmt.wrapError \| 0x36891cc75140>: failed to create NodePool np-1: failed waiting for nodepool="np-1" for cluster "private-kv-cluster" in resourcegroup="private-keyvault-rw4pgrx2qsqc" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/a719f074-9cbb-4f60-8f74-9acd323de195 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/a719f074-9cbb-4f60-8f74-9acd323de195", "name": "a719f074-9cbb-4f60-8f74-9acd323de195", "status": "Failed", "startTime": "2026-10-03T00:58:50.620270926Z", "endTime": "2026-10-03T01:17:53.07846241Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine y4c5l7p2o7h6i3y-np-1-7slbg-nmjht: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine y4c5l7p2o7h6i3y-np-1-7slbg-qnrnm: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine y4c5l7p2o7h6i3y-np-1-7slbg-nmjht: Creating: virtualmachine creating or updating\nMachine y4c5l7p2o7h6i3y-np-1-7slbg-qnrnm: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 00:25:08 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 5.1 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-7bgsbj" for cluster "cluster-candidate-5-1-7bgsbj" Unexpected error: <fmt.wrapError \| 0x355ed6679960>: failed to create NodePool np-7bgsbj: failed waiting for nodepool="np-7bgsbj" for cluster "cluster-candidate-5-1-7bgsbj" in resourcegroup="rg-candidate-5-1-twzxzjn6gcp4" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/48ddacde-704d-413e-818f-fb753e5b6f29 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/48ddacde-704d-413e-818f-fb753e5b6f29", "name": "48ddacde-704d-413e-818f-fb753e5b6f29", "status": "Failed", "startTime": "2026-10-03T01:21:40.063888823Z", "endTime": "2026-10-03T01:40:41.803400223Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine h4l9i6c5a5m3n3t-np-7bgsbj-b5gdb-b2m7f: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine h4l9i6c5a5m3n3t-np-7bgsbj-b5gdb-ssxpc: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine h4l9i6c5a5m3n3t-np-7bgsbj-b5gdb-b2m7f: Creating: virtualmachine creating or updating\nMachine h4l9i6c5a5m3n3t-np-7bgsbj-b5gdb-ssxpc: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-7bgsbj" for cluster "cluster-candidate-5-1-7bgsbj" Unexpected error: <fmt.wrapError \| 0x355ed6679960>: failed to create NodePool np-7bgsbj: failed waiting for nodepool="np-7bgsbj" for cluster "cluster-candidate-5-1-7bgsbj" in resourcegroup="rg-candidate-5-1-twzxzjn6gcp4" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/48ddacde-704d-413e-818f-fb753e5b6f29 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/48ddacde-704d-413e-818f-fb753e5b6f29", "name": "48ddacde-704d-413e-818f-fb753e5b6f29", "status": "Failed", "startTime": "2026-10-03T01:21:40.063888823Z", "endTime": "2026-10-03T01:40:41.803400223Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine h4l9i6c5a5m3n3t-np-7bgsbj-b5gdb-b2m7f: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine h4l9i6c5a5m3n3t-np-7bgsbj-b5gdb-ssxpc: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine h4l9i6c5a5m3n3t-np-7bgsbj-b5gdb-b2m7f: Creating: virtualmachine creating or updating\nMachine h4l9i6c5a5m3n3t-np-7bgsbj-b5gdb-ssxpc: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 00:25:08 UTC · failed at e2e · test Customer should be able to create a no-CNI private cluster with a private key vault, a nodepool and install cilium CNI successfully · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:193]: failed to verify nodes are Ready with Cilium CNI for cluster "cilium-cluster" Unexpected error: <errors.joinError \| 0x382d24c28030>: VerifyNodesReady failed: no nodes found in the cluster VerifyCiliumOperational failed: not all pods in kube-system namespace are running: failed to list pods in kube-system namespace: client rate limiter Wait returned an error: context deadline exceeded failed to create NodePool cilium-np: failed waiting for nodepool="cilium-np" for cluster "cilium-cluster" in resourcegroup="complex-cilium-kv-zvljfb77qpnf" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/fd6903c4-bc0c-4634-b456-f0053f4e62ce -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/fd6903c4-bc0c-4634-b456-f0053f4e62ce", "name": "fd6903c4-bc0c-4634-b456-f0053f4e62ce", "status": "Failed", "startTime": "2026-10-03T00:59:54.046743948Z", "endTime": "2026-10-03T01:19:03.00985984Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine cilium-cluster-cilium-np-2hhfv-7r58z: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine cilium-cluster-cilium-np-2hhfv-fmwpb: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine cilium-cluster-cilium-np-2hhfv-7r58z: Creating: virtualmachine creating or updating\nMachine cilium-cluster-cilium-np-2hhfv-fmwpb: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:193]: failed to verify nodes are Ready with Cilium CNI for cluster "cilium-cluster" Unexpected error: <errors.joinError \| 0x382d24c28030>: VerifyNodesReady failed: no nodes found in the cluster VerifyCiliumOperational failed: not all pods in kube-system namespace are running: failed to list pods in kube-system namespace: client rate limiter Wait returned an error: context deadline exceeded failed to create NodePool cilium-np: failed waiting for nodepool="cilium-np" for cluster "cilium-cluster" in resourcegroup="complex-cilium-kv-zvljfb77qpnf" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/fd6903c4-bc0c-4634-b456-f0053f4e62ce -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/fd6903c4-bc0c-4634-b456-f0053f4e62ce", "name": "fd6903c4-bc0c-4634-b456-f0053f4e62ce", "status": "Failed", "startTime": "2026-10-03T00:59:54.046743948Z", "endTime": "2026-10-03T01:19:03.00985984Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine cilium-cluster-cilium-np-2hhfv-7r58z: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine cilium-cluster-cilium-np-2hhfv-fmwpb: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine cilium-cluster-cilium-np-2hhfv-7r58z: Creating: virtualmachine creating or updating\nMachine cilium-cluster-cilium-np-2hhfv-fmwpb: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 00:25:08 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 5.0 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-mpbw9s" for cluster "cluster-candidate-5-0-mpbw9s" Unexpected error: <fmt.wrapError \| 0xea20e6b7700>: failed to create NodePool np-mpbw9s: failed waiting for nodepool="np-mpbw9s" for cluster "cluster-candidate-5-0-mpbw9s" in resourcegroup="rg-candidate-5-0-6l9pw288lpdv" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/49f8bd06-343a-4b31-8a4d-21de9398ca35 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/49f8bd06-343a-4b31-8a4d-21de9398ca35", "name": "49f8bd06-343a-4b31-8a4d-21de9398ca35", "status": "Failed", "startTime": "2026-10-03T01:23:58.4098212Z", "endTime": "2026-10-03T01:43:01.804835963Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine k9g6t4i6h6q8k7w-np-mpbw9s-tmszd-sx8g2: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine k9g6t4i6h6q8k7w-np-mpbw9s-tmszd-xvdjz: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine k9g6t4i6h6q8k7w-np-mpbw9s-tmszd-sx8g2: Creating: virtualmachine creating or updating\nMachine k9g6t4i6h6q8k7w-np-mpbw9s-tmszd-xvdjz: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-mpbw9s" for cluster "cluster-candidate-5-0-mpbw9s" Unexpected error: <fmt.wrapError \| 0xea20e6b7700>: failed to create NodePool np-mpbw9s: failed waiting for nodepool="np-mpbw9s" for cluster "cluster-candidate-5-0-mpbw9s" in resourcegroup="rg-candidate-5-0-6l9pw288lpdv" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/49f8bd06-343a-4b31-8a4d-21de9398ca35 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/49f8bd06-343a-4b31-8a4d-21de9398ca35", "name": "49f8bd06-343a-4b31-8a4d-21de9398ca35", "status": "Failed", "startTime": "2026-10-03T01:23:58.4098212Z", "endTime": "2026-10-03T01:43:01.804835963Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine k9g6t4i6h6q8k7w-np-mpbw9s-tmszd-sx8g2: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine k9g6t4i6h6q8k7w-np-mpbw9s-tmszd-xvdjz: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine k9g6t4i6h6q8k7w-np-mpbw9s-tmszd-sx8g2: Creating: virtualmachine creating or updating\nMachine k9g6t4i6h6q8k7w-np-mpbw9s-tmszd-xvdjz: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 00:25:08 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.23 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-55sh69" for cluster "cluster-candidate-4-23-55sh69" Unexpected error: <fmt.wrapError \| 0x2d83ca910b00>: failed to create NodePool np-55sh69: failed waiting for nodepool="np-55sh69" for cluster "cluster-candidate-4-23-55sh69" in resourcegroup="rg-candidate-4-23-b2jh9rr7848q" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/857aaaff-55f2-4507-bff3-f5f99ad82196 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/857aaaff-55f2-4507-bff3-f5f99ad82196", "name": "857aaaff-55f2-4507-bff3-f5f99ad82196", "status": "Failed", "startTime": "2026-10-03T01:21:00.037991541Z", "endTime": "2026-10-03T01:40:02.219519171Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine z7q7z3g1g0w5l4w-np-55sh69-kmbcq-qt7pp: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine z7q7z3g1g0w5l4w-np-55sh69-kmbcq-xbwxp: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine z7q7z3g1g0w5l4w-np-55sh69-kmbcq-qt7pp: Creating: virtualmachine creating or updating\nMachine z7q7z3g1g0w5l4w-np-55sh69-kmbcq-xbwxp: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-55sh69" for cluster "cluster-candidate-4-23-55sh69" Unexpected error: <fmt.wrapError \| 0x2d83ca910b00>: failed to create NodePool np-55sh69: failed waiting for nodepool="np-55sh69" for cluster "cluster-candidate-4-23-55sh69" in resourcegroup="rg-candidate-4-23-b2jh9rr7848q" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/857aaaff-55f2-4507-bff3-f5f99ad82196 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/857aaaff-55f2-4507-bff3-f5f99ad82196", "name": "857aaaff-55f2-4507-bff3-f5f99ad82196", "status": "Failed", "startTime": "2026-10-03T01:21:00.037991541Z", "endTime": "2026-10-03T01:40:02.219519171Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine z7q7z3g1g0w5l4w-np-55sh69-kmbcq-qt7pp: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine z7q7z3g1g0w5l4w-np-55sh69-kmbcq-xbwxp: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine z7q7z3g1g0w5l4w-np-55sh69-kmbcq-qt7pp: Creating: virtualmachine creating or updating\nMachine z7q7z3g1g0w5l4w-np-55sh69-kmbcq-xbwxp: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 00:25:08 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.20 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-xcs2zx" for cluster "cluster-candidate-4-20-xcs2zx" Unexpected error: <fmt.wrapError \| 0x3bb65f4dc3e0>: failed to create NodePool np-xcs2zx: failed waiting for nodepool="np-xcs2zx" for cluster "cluster-candidate-4-20-xcs2zx" in resourcegroup="rg-candidate-4-20-9tm8m64jwh6q" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b5e3d740-6442-4b8e-9013-9867a8b30bc0 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b5e3d740-6442-4b8e-9013-9867a8b30bc0", "name": "b5e3d740-6442-4b8e-9013-9867a8b30bc0", "status": "Failed", "startTime": "2026-10-03T01:20:28.820955232Z", "endTime": "2026-10-03T01:39:31.794768159Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine b5o5a6z5q4p5e0b-np-xcs2zx-hlrdg-42gm5: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine b5o5a6z5q4p5e0b-np-xcs2zx-hlrdg-zmvl5: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine b5o5a6z5q4p5e0b-np-xcs2zx-hlrdg-42gm5: Creating: virtualmachine creating or updating\nMachine b5o5a6z5q4p5e0b-np-xcs2zx-hlrdg-zmvl5: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-xcs2zx" for cluster "cluster-candidate-4-20-xcs2zx" Unexpected error: <fmt.wrapError \| 0x3bb65f4dc3e0>: failed to create NodePool np-xcs2zx: failed waiting for nodepool="np-xcs2zx" for cluster "cluster-candidate-4-20-xcs2zx" in resourcegroup="rg-candidate-4-20-9tm8m64jwh6q" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b5e3d740-6442-4b8e-9013-9867a8b30bc0 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b5e3d740-6442-4b8e-9013-9867a8b30bc0", "name": "b5e3d740-6442-4b8e-9013-9867a8b30bc0", "status": "Failed", "startTime": "2026-10-03T01:20:28.820955232Z", "endTime": "2026-10-03T01:39:31.794768159Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine b5o5a6z5q4p5e0b-np-xcs2zx-hlrdg-42gm5: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine b5o5a6z5q4p5e0b-np-xcs2zx-hlrdg-zmvl5: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine b5o5a6z5q4p5e0b-np-xcs2zx-hlrdg-42gm5: Creating: virtualmachine creating or updating\nMachine b5o5a6z5q4p5e0b-np-xcs2zx-hlrdg-zmvl5: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; detail message node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is <count>, want <count>; node pool condition AllNodesHealthy is False: <count> of <count> machines are not healthy Machine <node>: InspectionFailed: Waiting for AzureMachine to report spec.providerID; node pool condition AllMachinesReady is False: <count> of <count> machines are not ready Machine <node>: Creating: virtualmachine creating or updating; provider Microsoft.RedHatOpenShift 00:25:08 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.22 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-md77w4" for cluster "cluster-candidate-4-22-md77w4" Unexpected error: <fmt.wrapError \| 0x34d9aee24b60>: failed to create NodePool np-md77w4: failed waiting for nodepool="np-md77w4" for cluster "cluster-candidate-4-22-md77w4" in resourcegroup="rg-candidate-4-22-qt94z5jzjgsz" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/c7b70034-c32d-4689-8c16-d6b19ed89cc8 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/c7b70034-c32d-4689-8c16-d6b19ed89cc8", "name": "c7b70034-c32d-4689-8c16-d6b19ed89cc8", "status": "Failed", "startTime": "2026-10-03T01:25:32.783100892Z", "endTime": "2026-10-03T01:44:41.785015844Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine m6p3c2b3z7i0g2t-np-md77w4-k4nbk-488zq: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine m6p3c2b3z7i0g2t-np-md77w4-k4nbk-f2khf: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine m6p3c2b3z7i0g2t-np-md77w4-k4nbk-488zq: Creating: virtualmachine creating or updating\nMachine m6p3c2b3z7i0g2t-np-md77w4-k4nbk-f2khf: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-md77w4" for cluster "cluster-candidate-4-22-md77w4" Unexpected error: <fmt.wrapError \| 0x34d9aee24b60>: failed to create NodePool np-md77w4: failed waiting for nodepool="np-md77w4" for cluster "cluster-candidate-4-22-md77w4" in resourcegroup="rg-candidate-4-22-qt94z5jzjgsz" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/c7b70034-c32d-4689-8c16-d6b19ed89cc8 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/c7b70034-c32d-4689-8c16-d6b19ed89cc8", "name": "c7b70034-c32d-4689-8c16-d6b19ed89cc8", "status": "Failed", "startTime": "2026-10-03T01:25:32.783100892Z", "endTime": "2026-10-03T01:44:41.785015844Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 0, want 2; node pool condition AllNodesHealthy is False: 2 of 2 machines are not healthy\nMachine m6p3c2b3z7i0g2t-np-md77w4-k4nbk-488zq: InspectionFailed: Waiting for AzureMachine to report spec.providerID\nMachine m6p3c2b3z7i0g2t-np-md77w4-k4nbk-f2khf: InspectionFailed: Waiting for AzureMachine to report spec.providerID\n; node pool condition AllMachinesReady is False: 2 of 2 machines are not ready\nMachine m6p3c2b3z7i0g2t-np-md77w4-k4nbk-488zq: Creating: virtualmachine creating or updating\nMachine m6p3c2b3z7i0g2t-np-md77w4-k4nbk-f2khf: Creating: virtualmachine creating or updating\n" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeadlineExceeded; provider Microsoft.Compute 00:25:08 UTC · failed at e2e · test Customer should upgrade and update a nodepool from 4.21.z to 4.21.zLatest · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:165]: failed to create node pool npupgrade-4-21 with version 4.21.35 Unexpected error: <fmt.wrapError \| 0xbbf668ab320>: failed to create NodePool npupgrade-4-21: failed waiting for nodepool="npupgrade-4-21" for cluster "np-version-upgrade-cluster-lqwbkg" in resourcegroup="rg-np-version-upgrade-lqwbkg-jl7ktvcrp5z8" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/a4143c12-1ac1-4d68-ba18-1bb777cf81b4 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/a4143c12-1ac1-4d68-ba18-1bb777cf81b4", "name": "a4143c12-1ac1-4d68-ba18-1bb777cf81b4", "status": "Failed", "startTime": "2026-10-03T01:58:23.770970582Z", "endTime": "2026-10-03T02:17:26.319658092Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 1, want 2; node pool condition AllNodesHealthy is False: 1 of 2 machines are not healthy\nMachine q0g7j2k6j1j2c5g-npupgrade-4-21-z7zbc-cgvzh: InspectionFailed: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-version-upgrade-lqwbkg-jl7ktvcrp5z8-np-upgrade-lq-e094d963/providers/Microsoft.Compute/virtualMachines/q0g7j2k6j1j2c5g-npupgrade-4-21-z7zbc-cgvzh to exist\n; node pool condition AllMachinesReady is False: 1 of 2 machines are not ready\nMachine q0g7j2k6j1j2c5g-npupgrade-4-21-z7zbc-cgvzh: ReadyUnknown: NodeHealthy: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-version-upgrade-lqwbkg-jl7ktvcrp5z8-np-upgrade-lq-e094d963/providers/Microsoft.Compute/virtualMachines/q0g7j2k6j1j2c5g-npupgrade-4-21-z7zbc-cgvzh to exist\n" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:165]: failed to create node pool npupgrade-4-21 with version 4.21.35 Unexpected error: <fmt.wrapError \| 0xbbf668ab320>: failed to create NodePool npupgrade-4-21: failed waiting for nodepool="npupgrade-4-21" for cluster "np-version-upgrade-cluster-lqwbkg" in resourcegroup="rg-np-version-upgrade-lqwbkg-jl7ktvcrp5z8" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/a4143c12-1ac1-4d68-ba18-1bb777cf81b4 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeadlineExceeded -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/a4143c12-1ac1-4d68-ba18-1bb777cf81b4", "name": "a4143c12-1ac1-4d68-ba18-1bb777cf81b4", "status": "Failed", "startTime": "2026-10-03T01:58:23.770970582Z", "endTime": "2026-10-03T02:17:26.319658092Z", "error": { "code": "DeadlineExceeded", "message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 1, want 2; node pool condition AllNodesHealthy is False: 1 of 2 machines are not healthy\nMachine q0g7j2k6j1j2c5g-npupgrade-4-21-z7zbc-cgvzh: InspectionFailed: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-version-upgrade-lqwbkg-jl7ktvcrp5z8-np-upgrade-lq-e094d963/providers/Microsoft.Compute/virtualMachines/q0g7j2k6j1j2c5g-npupgrade-4-21-z7zbc-cgvzh to exist\n; node pool condition AllMachinesReady is False: 1 of 2 machines are not ready\nMachine q0g7j2k6j1j2c5g-npupgrade-4-21-z7zbc-cgvzh: ReadyUnknown: NodeHealthy: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-version-upgrade-lqwbkg-jl7ktvcrp5z8-np-upgrade-lq-e094d963/providers/Microsoft.Compute/virtualMachines/q0g7j2k6j1j2c5g-npupgrade-4-21-z7zbc-cgvzh to exist\n" } } -------------------------------------------------------------------------------- ... occurred