CIHealth
<
Single Day: Sep 25▾
Pick a day
>
View JSON API
↗ Open Failure patterns for this day

Environment: INT

Time (UTC)RuntimeJobRegionFailed atResultPRFailed testsDetails
2h 20m 53s branch-ci-Azure-ARO-HCP-main-e2e-integration-e2e-parallel
post-goodmerged PR
No additional run metadata captured
 westus3 n/a passed n/a 0 n/a
2h 18m 42s branch-ci-Azure-ARO-HCP-main-e2e-integration-e2e-parallel
post-goodmerged PR
No additional run metadata captured
 westus3 n/a passed n/a 0 n/a
2h 16m 39s branch-ci-Azure-ARO-HCP-main-e2e-integration-e2e-parallel
post-goodmerged PR
No additional run metadata captured
 westus3 e2e failed n/a 2
e2e (2)
ERROR CODE: InternalServerError; detail message [hypershiftNodePool] hypershift NodePool status replicas is 1, want 2; node pool condition AllNodesHealthy is False: 1 of 2 machines are not healthy Machine <node>: InspectionFailed: Waiting for a Node with spec.providerID azure:///subscriptions/<subscription>/resourcegroups/<resource-group>/providers/Microsoft.Compute/virtualMachines/<node> to exist ; node pool condition AllMachinesReady is False: 1 of 2 machines are not ready Machine <node>: ReadyUnknown: * NodeHealthy: Waiting for a Node with spec.providerID azure:///subscriptions/<subscription>/resourcegroups/<resource-group>/providers/Microsoft.Compute/virtualMachines/<node> to exist; provider Microsoft.Compute
15:01:22 UTC · failed at e2e · test Customer should upgrade a nodepool skipping one minor version (+2) from 4.20.z to 4.22.zLatest · suite integration/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:459]: failed to create nodepool nps-420-422 at version 4.20.39
Unexpected error:
    <*fmt.wrapError | 0xc000599de0>: 
    failed to create NodePool nps-420-422: failed waiting for nodepool="nps-420-422" for cluster "np-skip-minor-kcj4l5" in resourcegroup="rg-np-skip-minor-kcj4l5-45s8nhm4k78b" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/da564fd3-aea3-4a41-b171-3969372d023d
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/da564fd3-aea3-4a41-b171-3969372d023d",
      "name": "da564fd3-aea3-4a41-b171-3969372d023d",
      "status": "Failed",
      "startTime": "2026-09-25T16:39:04.653915252Z",
      "endTime": "2026-09-25T16:58:09.716908828Z",
      "error": {
        "code": "InternalServerError",
        "message": "[hypershiftNodePool] hypershift NodePool status replicas is 1, want 2; node pool condition AllNodesHealthy is False: 1 of 2 machines are not healthy\nMachine k7t8o1t6u7r7f0q-nps-420-422-lf9bj-qzzxg: InspectionFailed: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-skip-minor-kcj4l5-45s8nhm4k78b-np-sm-kcj4l5--managed/providers/Microsoft.Compute/virtualMachines/k7t8o1t6u7r7f0q-nps-420-422-lf9bj-qzzxg to exist\n; node pool condition AllMachinesReady is False: 1 of 2 machines are not ready\nMachine k7t8o1t6u7r7f0q-nps-420-422-lf9bj-qzzxg: ReadyUnknown: * NodeHealthy: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-skip-minor-kcj4l5-45s8nhm4k78b-np-sm-kcj4l5--managed/providers/Microsoft.Compute/virtualMachines/k7t8o1t6u7r7f0q-nps-420-422-lf9bj-qzzxg to exist\n"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:459]: failed to create nodepool nps-420-422 at version 4.20.39
Unexpected error:
    <*fmt.wrapError | 0xc000599de0>: 
    failed to create NodePool nps-420-422: failed waiting for nodepool="nps-420-422" for cluster "np-skip-minor-kcj4l5" in resourcegroup="rg-np-skip-minor-kcj4l5-45s8nhm4k78b" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/da564fd3-aea3-4a41-b171-3969372d023d
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/da564fd3-aea3-4a41-b171-3969372d023d",
      "name": "da564fd3-aea3-4a41-b171-3969372d023d",
      "status": "Failed",
      "startTime": "2026-09-25T16:39:04.653915252Z",
      "endTime": "2026-09-25T16:58:09.716908828Z",
      "error": {
        "code": "InternalServerError",
        "message": "[hypershiftNodePool] hypershift NodePool status replicas is 1, want 2; node pool condition AllNodesHealthy is False: 1 of 2 machines are not healthy\nMachine k7t8o1t6u7r7f0q-nps-420-422-lf9bj-qzzxg: InspectionFailed: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-skip-minor-kcj4l5-45s8nhm4k78b-np-sm-kcj4l5--managed/providers/Microsoft.Compute/virtualMachines/k7t8o1t6u7r7f0q-nps-420-422-lf9bj-qzzxg to exist\n; node pool condition AllMachinesReady is False: 1 of 2 machines are not ready\nMachine k7t8o1t6u7r7f0q-nps-420-422-lf9bj-qzzxg: ReadyUnknown: * NodeHealthy: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-skip-minor-kcj4l5-45s8nhm4k78b-np-sm-kcj4l5--managed/providers/Microsoft.Compute/virtualMachines/k7t8o1t6u7r7f0q-nps-420-422-lf9bj-qzzxg to exist\n"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InternalServerError; detail message [hypershiftHostedCluster] hosted cluster is not available: KubeconfigWaitingForCreate: Waiting for hosted control plane kubeconfig to be created; hosted cluster degraded: UnavailableReplicas: [capi-provider]; provider Microsoft.RedHatOpenShift
15:01:22 UTC · failed at e2e · test Update HCPOpenShiftCluster Negative creates a cluster and fails to update its name with a PATCH request · suite integration/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:84]: failed to create HCP cluster for patch-name test
Unexpected error:
    <*fmt.wrapError | 0xc000f18e80>: 
    failed to create HCP cluster patch-name-cluster: failed waiting for cluster="patch-name-cluster" in resourcegroup="patch-name-p9kv2wz66ng2" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/82d3308e-222c-40d8-a1be-8c057c26bab1
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/82d3308e-222c-40d8-a1be-8c057c26bab1",
      "name": "82d3308e-222c-40d8-a1be-8c057c26bab1",
      "status": "Failed",
      "startTime": "2026-09-25T15:45:43.486812846Z",
      "endTime": "2026-09-25T16:14:49.560161271Z",
      "error": {
        "code": "InternalServerError",
        "message": "[hypershiftHostedCluster] hosted cluster is not available: KubeconfigWaitingForCreate: Waiting for hosted control plane kubeconfig to be created; hosted cluster degraded: UnavailableReplicas: capi-provider deployment has 1 unavailable replicas"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:84]: failed to create HCP cluster for patch-name test
Unexpected error:
    <*fmt.wrapError | 0xc000f18e80>: 
    failed to create HCP cluster patch-name-cluster: failed waiting for cluster="patch-name-cluster" in resourcegroup="patch-name-p9kv2wz66ng2" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/82d3308e-222c-40d8-a1be-8c057c26bab1
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/82d3308e-222c-40d8-a1be-8c057c26bab1",
      "name": "82d3308e-222c-40d8-a1be-8c057c26bab1",
      "status": "Failed",
      "startTime": "2026-09-25T15:45:43.486812846Z",
      "endTime": "2026-09-25T16:14:49.560161271Z",
      "error": {
        "code": "InternalServerError",
        "message": "[hypershiftHostedCluster] hosted cluster is not available: KubeconfigWaitingForCreate: Waiting for hosted control plane kubeconfig to be created; hosted cluster degraded: UnavailableReplicas: capi-provider deployment has 1 unavailable replicas"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
2h 21m 35s branch-ci-Azure-ARO-HCP-main-e2e-integration-e2e-parallel
post-goodmerged PR
No additional run metadata captured
 westus3 n/a passed n/a 0 n/a
2h 4m 55s branch-ci-Azure-ARO-HCP-main-e2e-integration-e2e-parallel
post-goodmerged PR
No additional run metadata captured
 westus3 e2e failed n/a 2
e2e (2)
ERROR CODE: InternalServerError; detail message [hypershiftNodePool] hypershift NodePool status replicas is 1, want 2; node pool condition AllNodesHealthy is False: 1 of 2 machines are not healthy Machine <node>: InspectionFailed: Waiting for a Node with spec.providerID azure:///subscriptions/<subscription>/resourcegroups/<resource-group>/providers/Microsoft.Compute/virtualMachines/<node> to exist ; node pool condition AllMachinesReady is False: 1 of 2 machines are not ready Machine <node>: ReadyUnknown: * NodeHealthy: Waiting for a Node with spec.providerID azure:///subscriptions/<subscription>/resourcegroups/<resource-group>/providers/Microsoft.Compute/virtualMachines/<node> to exist; provider Microsoft.Compute
08:13:43 UTC · failed at e2e · test Create HCPOpenShiftCluster with Private KeyVault should create a cluster with private keyvault using v20251223preview API · suite integration/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:138]: failed to create node pool "np-1" for private keyvault cluster "private-kv-cluster"
Unexpected error:
    <*fmt.wrapError | 0xc00075fc80>: 
    failed to create NodePool np-1: failed waiting for nodepool="np-1" for cluster "private-kv-cluster" in resourcegroup="private-keyvault-dnhw57q2zhsr" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/9cb82bcd-1a84-400b-ad2e-8cf3dfa50902
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/9cb82bcd-1a84-400b-ad2e-8cf3dfa50902",
      "name": "9cb82bcd-1a84-400b-ad2e-8cf3dfa50902",
      "status": "Failed",
      "startTime": "2026-09-25T08:30:53.138668141Z",
      "endTime": "2026-09-25T08:49:53.966422643Z",
      "error": {
        "code": "InternalServerError",
        "message": "[hypershiftNodePool] hypershift NodePool status replicas is 1, want 2; node pool condition AllNodesHealthy is False: 1 of 2 machines are not healthy\nMachine q8f0o9t2z5z1x2r-np-1-9lssw-jg4gh: InspectionFailed: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/private-keyvault-dnhw57q2zhsr--managed/providers/Microsoft.Compute/virtualMachines/q8f0o9t2z5z1x2r-np-1-9lssw-jg4gh to exist\n; node pool condition AllMachinesReady is False: 1 of 2 machines are not ready\nMachine q8f0o9t2z5z1x2r-np-1-9lssw-jg4gh: ReadyUnknown: * NodeHealthy: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/private-keyvault-dnhw57q2zhsr--managed/providers/Microsoft.Compute/virtualMachines/q8f0o9t2z5z1x2r-np-1-9lssw-jg4gh to exist\n"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:138]: failed to create node pool "np-1" for private keyvault cluster "private-kv-cluster"
Unexpected error:
    <*fmt.wrapError | 0xc00075fc80>: 
    failed to create NodePool np-1: failed waiting for nodepool="np-1" for cluster "private-kv-cluster" in resourcegroup="private-keyvault-dnhw57q2zhsr" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/9cb82bcd-1a84-400b-ad2e-8cf3dfa50902
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/9cb82bcd-1a84-400b-ad2e-8cf3dfa50902",
      "name": "9cb82bcd-1a84-400b-ad2e-8cf3dfa50902",
      "status": "Failed",
      "startTime": "2026-09-25T08:30:53.138668141Z",
      "endTime": "2026-09-25T08:49:53.966422643Z",
      "error": {
        "code": "InternalServerError",
        "message": "[hypershiftNodePool] hypershift NodePool status replicas is 1, want 2; node pool condition AllNodesHealthy is False: 1 of 2 machines are not healthy\nMachine q8f0o9t2z5z1x2r-np-1-9lssw-jg4gh: InspectionFailed: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/private-keyvault-dnhw57q2zhsr--managed/providers/Microsoft.Compute/virtualMachines/q8f0o9t2z5z1x2r-np-1-9lssw-jg4gh to exist\n; node pool condition AllMachinesReady is False: 1 of 2 machines are not ready\nMachine q8f0o9t2z5z1x2r-np-1-9lssw-jg4gh: ReadyUnknown: * NodeHealthy: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/private-keyvault-dnhw57q2zhsr--managed/providers/Microsoft.Compute/virtualMachines/q8f0o9t2z5z1x2r-np-1-9lssw-jg4gh to exist\n"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InternalServerError; detail message [hypershiftNodePool] hypershift NodePool status replicas is 1, want 2; node pool condition AllNodesHealthy is False: 1 of 2 machines are not healthy Machine <node>: InspectionFailed: Waiting for a Node with spec.providerID azure:///subscriptions/<subscription>/resourcegroups/<resource-group>/providers/Microsoft.Compute/virtualMachines/<node> to exist ; node pool condition AllMachinesReady is False: 1 of 2 machines are not ready Machine <node>: ReadyUnknown: * NodeHealthy: Waiting for a Node with spec.providerID azure:///subscriptions/<subscription>/resourcegroups/<resource-group>/providers/Microsoft.Compute/virtualMachines/<node> to exist; provider Microsoft.Compute
08:13:43 UTC · failed at e2e · test Customer should be able to create an HCP cluster and custom node pool osDisk size · suite integration/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_nodepool_osdisk.go:96]: failed to create node pool "nodepool-128GiB" with 128GiB osDisk
Unexpected error:
    <*fmt.wrapError | 0xc0010c02c0>: 
    failed to create NodePool nodepool-128GiB: failed waiting for nodepool="nodepool-128GiB" for cluster "hcp-cluster-np-128" in resourcegroup="clusternp128-2mw49n77wnb2" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/e57de6e4-350f-44ca-8989-0fa815b3fd2f
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/e57de6e4-350f-44ca-8989-0fa815b3fd2f",
      "name": "e57de6e4-350f-44ca-8989-0fa815b3fd2f",
      "status": "Failed",
      "startTime": "2026-09-25T08:27:10.94071903Z",
      "endTime": "2026-09-25T08:46:13.959695236Z",
      "error": {
        "code": "InternalServerError",
        "message": "[hypershiftNodePool] hypershift NodePool status replicas is 1, want 2; node pool condition AllNodesHealthy is False: 1 of 2 machines are not healthy\nMachine y5d4i4z6u4m1b4g-nodepool-128gib-j5q6z-kjlzl: InspectionFailed: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/clusternp128-2mw49n77wnb2--managed/providers/Microsoft.Compute/virtualMachines/y5d4i4z6u4m1b4g-nodepool-128gib-j5q6z-kjlzl to exist\n; node pool condition AllMachinesReady is False: 1 of 2 machines are not ready\nMachine y5d4i4z6u4m1b4g-nodepool-128gib-j5q6z-kjlzl: ReadyUnknown: * NodeHealthy: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/clusternp128-2mw49n77wnb2--managed/providers/Microsoft.Compute/virtualMachines/y5d4i4z6u4m1b4g-nodepool-128gib-j5q6z-kjlzl to exist\n"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_nodepool_osdisk.go:96]: failed to create node pool "nodepool-128GiB" with 128GiB osDisk
Unexpected error:
    <*fmt.wrapError | 0xc0010c02c0>: 
    failed to create NodePool nodepool-128GiB: failed waiting for nodepool="nodepool-128GiB" for cluster "hcp-cluster-np-128" in resourcegroup="clusternp128-2mw49n77wnb2" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/e57de6e4-350f-44ca-8989-0fa815b3fd2f
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/e57de6e4-350f-44ca-8989-0fa815b3fd2f",
      "name": "e57de6e4-350f-44ca-8989-0fa815b3fd2f",
      "status": "Failed",
      "startTime": "2026-09-25T08:27:10.94071903Z",
      "endTime": "2026-09-25T08:46:13.959695236Z",
      "error": {
        "code": "InternalServerError",
        "message": "[hypershiftNodePool] hypershift NodePool status replicas is 1, want 2; node pool condition AllNodesHealthy is False: 1 of 2 machines are not healthy\nMachine y5d4i4z6u4m1b4g-nodepool-128gib-j5q6z-kjlzl: InspectionFailed: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/clusternp128-2mw49n77wnb2--managed/providers/Microsoft.Compute/virtualMachines/y5d4i4z6u4m1b4g-nodepool-128gib-j5q6z-kjlzl to exist\n; node pool condition AllMachinesReady is False: 1 of 2 machines are not ready\nMachine y5d4i4z6u4m1b4g-nodepool-128gib-j5q6z-kjlzl: ReadyUnknown: * NodeHealthy: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/clusternp128-2mw49n77wnb2--managed/providers/Microsoft.Compute/virtualMachines/y5d4i4z6u4m1b4g-nodepool-128gib-j5q6z-kjlzl to exist\n"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
2h 18m 30s branch-ci-Azure-ARO-HCP-main-e2e-integration-e2e-parallel
post-goodmerged PR
No additional run metadata captured
 westus3 e2e failed n/a 1
e2e (1)
ERROR CODE: Conflict; detail message Cannot update resource while resource is "provisioning"; provider Microsoft.RedHatOpenShift
05:49:44 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 5.0 · suite integration/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-pt5qmx" for cluster "cluster-candidate-5-0-pt5qmx"
Unexpected error:
    <*fmt.wrapError | 0xc000c10900>: 
    failed to create NodePool np-pt5qmx: failed starting nodepool creation "np-pt5qmx" for cluster "cluster-candidate-5-0-pt5qmx" in resourcegroup="rg-candidate-5-0-9clwz8rf6599": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-5-0-9clwz8rf6599/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-candidate-5-0-pt5qmx/nodePools/np-pt5qmx
    --------------------------------------------------------------------------------
    RESPONSE 409: 409 Conflict
    ERROR CODE: Conflict
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "Conflict",
        "message": "Cannot update resource while resource is \"provisioning\"",
        "target": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-5-0-9clwz8rf6599/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-candidate-5-0-pt5qmx/nodePools/np-pt5qmx"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-pt5qmx" for cluster "cluster-candidate-5-0-pt5qmx"
Unexpected error:
    <*fmt.wrapError | 0xc000c10900>: 
    failed to create NodePool np-pt5qmx: failed starting nodepool creation "np-pt5qmx" for cluster "cluster-candidate-5-0-pt5qmx" in resourcegroup="rg-candidate-5-0-9clwz8rf6599": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-5-0-9clwz8rf6599/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-candidate-5-0-pt5qmx/nodePools/np-pt5qmx
    --------------------------------------------------------------------------------
    RESPONSE 409: 409 Conflict
    ERROR CODE: Conflict
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "Conflict",
        "message": "Cannot update resource while resource is \"provisioning\"",
        "target": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-5-0-9clwz8rf6599/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-candidate-5-0-pt5qmx/nodePools/np-pt5qmx"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
2h 22m 33s branch-ci-Azure-ARO-HCP-main-e2e-integration-e2e-parallel
post-goodmerged PR
No additional run metadata captured
 westus3 e2e failed n/a 1
e2e (1)
ERROR CODE: InternalServerError; detail message [hypershiftNodePool] hypershift NodePool status replicas is 1, want 2; node pool condition AllNodesHealthy is False: 1 of 2 machines are not healthy Machine <node>: InspectionFailed: Waiting for a Node with spec.providerID azure:///subscriptions/<subscription>/resourcegroups/<resource-group>/providers/Microsoft.Compute/virtualMachines/<node> to exist ; node pool condition AllMachinesReady is False: 1 of 2 machines are not ready Machine <node>: ReadyUnknown: * NodeHealthy: Waiting for a Node with spec.providerID azure:///subscriptions/<subscription>/resourcegroups/<resource-group>/providers/Microsoft.Compute/virtualMachines/<node> to exist; provider Microsoft.Compute
02:01:43 UTC · failed at e2e · test FIPS Mode Support with v2026-09-01 API should create an HCP cluster with FIPS mode enabled via cryptoRestrictions property · suite integration/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_fips_mode.go:103]: failed to create node pool "np-1" for fips-enabled cluster "fips-enabled-cluster"
Unexpected error:
    <*fmt.wrapError | 0xc0007453e0>: 
    failed to create NodePool np-1: failed waiting for nodepool="np-1" for cluster "fips-enabled-cluster" in resourcegroup="fips-enabled-58b5tglh77bz" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/28fe561e-2b93-4934-b274-0e50cb5390d6
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/28fe561e-2b93-4934-b274-0e50cb5390d6",
      "name": "28fe561e-2b93-4934-b274-0e50cb5390d6",
      "status": "Failed",
      "startTime": "2026-09-25T02:36:40.600765804Z",
      "endTime": "2026-09-25T02:55:41.450889323Z",
      "error": {
        "code": "InternalServerError",
        "message": "[hypershiftNodePool] hypershift NodePool status replicas is 1, want 2; node pool condition AllNodesHealthy is False: 1 of 2 machines are not healthy\nMachine x1c7o2k3x9l0n1u-np-1-kxng9-xrnnb: InspectionFailed: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/fips-enabled-58b5tglh77bz--managed/providers/Microsoft.Compute/virtualMachines/x1c7o2k3x9l0n1u-np-1-kxng9-xrnnb to exist\n; node pool condition AllMachinesReady is False: 1 of 2 machines are not ready\nMachine x1c7o2k3x9l0n1u-np-1-kxng9-xrnnb: ReadyUnknown: * NodeHealthy: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/fips-enabled-58b5tglh77bz--managed/providers/Microsoft.Compute/virtualMachines/x1c7o2k3x9l0n1u-np-1-kxng9-xrnnb to exist\n"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_fips_mode.go:103]: failed to create node pool "np-1" for fips-enabled cluster "fips-enabled-cluster"
Unexpected error:
    <*fmt.wrapError | 0xc0007453e0>: 
    failed to create NodePool np-1: failed waiting for nodepool="np-1" for cluster "fips-enabled-cluster" in resourcegroup="fips-enabled-58b5tglh77bz" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/28fe561e-2b93-4934-b274-0e50cb5390d6
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOperationStatuses/28fe561e-2b93-4934-b274-0e50cb5390d6",
      "name": "28fe561e-2b93-4934-b274-0e50cb5390d6",
      "status": "Failed",
      "startTime": "2026-09-25T02:36:40.600765804Z",
      "endTime": "2026-09-25T02:55:41.450889323Z",
      "error": {
        "code": "InternalServerError",
        "message": "[hypershiftNodePool] hypershift NodePool status replicas is 1, want 2; node pool condition AllNodesHealthy is False: 1 of 2 machines are not healthy\nMachine x1c7o2k3x9l0n1u-np-1-kxng9-xrnnb: InspectionFailed: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/fips-enabled-58b5tglh77bz--managed/providers/Microsoft.Compute/virtualMachines/x1c7o2k3x9l0n1u-np-1-kxng9-xrnnb to exist\n; node pool condition AllMachinesReady is False: 1 of 2 machines are not ready\nMachine x1c7o2k3x9l0n1u-np-1-kxng9-xrnnb: ReadyUnknown: * NodeHealthy: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/fips-enabled-58b5tglh77bz--managed/providers/Microsoft.Compute/virtualMachines/x1c7o2k3x9l0n1u-np-1-kxng9-xrnnb to exist\n"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred