CIHealth
<
Single Day: Jun 9▾
Pick a day
>
View JSON API
↗ Open Failure patterns for this day

Environment: STG

Time (UTC)JobFailed atResultPRFailed testsDetails
periodic-ci-Azure-ARO-HCP-main-periodic-stage-e2e-parallel-ocp-nightly
post-goodmerged PR
No additional run metadata captured
 e2e failed n/a 3
e2e (3)
ERROR CODE: InvalidRequestContent; detail message Version 'openshift-v<version>' doesn't exist; provider Microsoft.RedHatOpenShift
23:00:04 UTC · failed at e2e · test Customer should downgrade a nodepool to a lower minor version from 4.21.zLatest to 4.19.zLatest · suite stage/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:704]: failed to create HCP cluster
Unexpected error:
    <*fmt.wrapError | 0xc000a44740>: 
    failed to create HCP cluster np-dg-minor-fnlrn4: failed starting cluster creation "np-dg-minor-fnlrn4" in resourcegroup="rg-np-dg-minor-fnlrn4-vhqtrs": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-dg-minor-fnlrn4-vhqtrs/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-dg-minor-fnlrn4
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Version 'openshift-v4.21.0-nightly' doesn't exist"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:704]: failed to create HCP cluster
Unexpected error:
    <*fmt.wrapError | 0xc000a44740>: 
    failed to create HCP cluster np-dg-minor-fnlrn4: failed starting cluster creation "np-dg-minor-fnlrn4" in resourcegroup="rg-np-dg-minor-fnlrn4-vhqtrs": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-dg-minor-fnlrn4-vhqtrs/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-dg-minor-fnlrn4
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Version 'openshift-v4.21.0-nightly' doesn't exist"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
CreateNodePoolFromParam timed out after <minutes> minutes; context deadline exceeded
23:00:04 UTC · failed at e2e · test Customer should be able to create a no-CNI private cluster with a private key vault, a nodepool and install cilium CNI successfully · suite stage/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:190]: failed to verify nodes are Ready with Cilium CNI for cluster "cilium-cluster"
Unexpected error:
    <*errors.joinError | 0xc001090570>: 
    VerifyNodesReady failed: no nodes found in the cluster
    VerifyCiliumOperational failed: not all pods in kube-system namespace are running: failed to list pods in kube-system namespace: client rate limiter Wait returned an error: rate: Wait(n=1) would exceed context deadline
    failed to create NodePool cilium-np, caused by: timeout '20.000000' minutes exceeded during CreateNodePoolFromParam for node pool cilium-np in resource group complex-cilium-kv-92tkr6, error: failed waiting for nodepool="cilium-np" for cluster "cilium-cluster" in resourcegroup="complex-cilium-kv-92tkr6" to finish creating: context deadline exceeded
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:190]: failed to verify nodes are Ready with Cilium CNI for cluster "cilium-cluster"
Unexpected error:
    <*errors.joinError | 0xc001090570>: 
    VerifyNodesReady failed: no nodes found in the cluster
    VerifyCiliumOperational failed: not all pods in kube-system namespace are running: failed to list pods in kube-system namespace: client rate limiter Wait returned an error: rate: Wait(n=1) would exceed context deadline
    failed to create NodePool cilium-np, caused by: timeout '20.000000' minutes exceeded during CreateNodePoolFromParam for node pool cilium-np in resource group complex-cilium-kv-92tkr6, error: failed waiting for nodepool="cilium-np" for cluster "cilium-cluster" in resourcegroup="complex-cilium-kv-92tkr6" to finish creating: context deadline exceeded
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Version 'openshift-v<version>' doesn't exist; provider Microsoft.RedHatOpenShift
23:00:04 UTC · failed at e2e · test Customer should upgrade a nodepool skipping one minor version (+2) from 4.20.z to 4.22.zLatest · suite stage/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:443]: failed to create HCP cluster np-skip-minor-sgczmg at version 4.22.0
Unexpected error:
    <*fmt.wrapError | 0xc000988f40>: 
    failed to create HCP cluster np-skip-minor-sgczmg: failed starting cluster creation "np-skip-minor-sgczmg" in resourcegroup="rg-np-skip-minor-sgczmg-8fp7x5": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-skip-minor-sgczmg-8fp7x5/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-skip-minor-sgczmg
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Version 'openshift-v4.22.0-nightly' doesn't exist"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:443]: failed to create HCP cluster np-skip-minor-sgczmg at version 4.22.0
Unexpected error:
    <*fmt.wrapError | 0xc000988f40>: 
    failed to create HCP cluster np-skip-minor-sgczmg: failed starting cluster creation "np-skip-minor-sgczmg" in resourcegroup="rg-np-skip-minor-sgczmg-8fp7x5": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-skip-minor-sgczmg-8fp7x5/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-skip-minor-sgczmg
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Version 'openshift-v4.22.0-nightly' doesn't exist"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
periodic-ci-Azure-ARO-HCP-main-periodic-stage-e2e-parallel
post-goodmerged PR
No additional run metadata captured
 e2e failed n/a 8
e2e (8)
timeout during CreateHCPClusterAndWait; context deadline exceeded
02:00:03 UTC · failed at e2e · test Customer should be able to create a cluster with an external auth config and get the external auth config · suite stage/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:93]: failed to create HCP cluster for external auth test
Unexpected error:
    <*fmt.wrapErrors | 0xc0012900c0>: 
    failed to create HCP cluster ea-cluster, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster ea-cluster in resource group external-auth-cluster-pb9d9w, error: failed waiting for cluster="ea-cluster" in resourcegroup="external-auth-cluster-pb9d9w" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster ea-cluster in resource group external-auth-cluster-pb9d9w, error: context deadline exceeded
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:93]: failed to create HCP cluster for external auth test
Unexpected error:
    <*fmt.wrapErrors | 0xc0012900c0>: 
    failed to create HCP cluster ea-cluster, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster ea-cluster in resource group external-auth-cluster-pb9d9w, error: failed waiting for cluster="ea-cluster" in resourcegroup="external-auth-cluster-pb9d9w" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster ea-cluster in resource group external-auth-cluster-pb9d9w, error: context deadline exceeded
    ...
occurred
timeout during CreateHCPClusterAndWait; context deadline exceeded
02:00:03 UTC · failed at e2e · test Nodepool Ephemeral OS Disk should create a nodepool with ephemeral OS disk when autoRepair is enabled · suite stage/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_ephemeral_osdisk.go:111]: failed to create HCP cluster ephemeral-disk
Unexpected error:
    <*fmt.wrapErrors | 0xc0009a7350>: 
    failed to create HCP cluster ephemeral-disk, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster ephemeral-disk in resource group ephemeral-osdisk-8vkvbj, error: failed waiting for cluster="ephemeral-disk" in resourcegroup="ephemeral-osdisk-8vkvbj" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster ephemeral-disk in resource group ephemeral-osdisk-8vkvbj, error: context deadline exceeded
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_ephemeral_osdisk.go:111]: failed to create HCP cluster ephemeral-disk
Unexpected error:
    <*fmt.wrapErrors | 0xc0009a7350>: 
    failed to create HCP cluster ephemeral-disk, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster ephemeral-disk in resource group ephemeral-osdisk-8vkvbj, error: failed waiting for cluster="ephemeral-disk" in resourcegroup="ephemeral-osdisk-8vkvbj" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster ephemeral-disk in resource group ephemeral-osdisk-8vkvbj, error: context deadline exceeded
    ...
occurred
timeout during CreateHCPClusterAndWait; context deadline exceeded
02:00:03 UTC · failed at e2e · test Customer should be able to lifecycle and confirm external auth on a cluster · suite stage/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_list_and_verify.go:83]: failed to create HCP cluster for external auth list test
Unexpected error:
    <*fmt.wrapErrors | 0xc0012e8300>: 
    failed to create HCP cluster ea-listjfj47p, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster ea-listjfj47p in resource group ea-list-l9l62g, error: failed waiting for cluster="ea-listjfj47p" in resourcegroup="ea-list-l9l62g" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster ea-listjfj47p in resource group ea-list-l9l62g, error: context deadline exceeded
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_list_and_verify.go:83]: failed to create HCP cluster for external auth list test
Unexpected error:
    <*fmt.wrapErrors | 0xc0012e8300>: 
    failed to create HCP cluster ea-listjfj47p, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster ea-listjfj47p in resource group ea-list-l9l62g, error: failed waiting for cluster="ea-listjfj47p" in resourcegroup="ea-list-l9l62g" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster ea-listjfj47p in resource group ea-list-l9l62g, error: context deadline exceeded
    ...
occurred
timeout during CreateHCPClusterAndWait; context deadline exceeded
02:00:03 UTC · failed at e2e · test Customer should respect cluster-wide node limits with nodepool autoscaling · suite stage/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:282]: failed to create HCP cluster node-limit-cluster with MaxNodesTotal limit
Unexpected error:
    <*fmt.wrapErrors | 0xc0013504b0>: 
    failed to create HCP cluster node-limit-cluster, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster node-limit-cluster in resource group node-limit-test-qsxzzc, error: failed waiting for cluster="node-limit-cluster" in resourcegroup="node-limit-test-qsxzzc" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster node-limit-cluster in resource group node-limit-test-qsxzzc, error: context deadline exceeded
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:282]: failed to create HCP cluster node-limit-cluster with MaxNodesTotal limit
Unexpected error:
    <*fmt.wrapErrors | 0xc0013504b0>: 
    failed to create HCP cluster node-limit-cluster, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster node-limit-cluster in resource group node-limit-test-qsxzzc, error: failed waiting for cluster="node-limit-cluster" in resourcegroup="node-limit-test-qsxzzc" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster node-limit-cluster in resource group node-limit-test-qsxzzc, error: context deadline exceeded
    ...
occurred
timeout during CreateHCPClusterAndWait; context deadline exceeded
02:00:03 UTC · failed at e2e · test Customer should be able to create a cluster with default autoscaling and a nodepool with autoscaling enabled up to replica limits · suite stage/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:95]: failed to create HCP cluster np-autoscale-cluster
Unexpected error:
    <*fmt.wrapErrors | 0xc0011120f0>: 
    failed to create HCP cluster np-autoscale-cluster, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster np-autoscale-cluster in resource group np-autoscaling-tskqgd, error: failed waiting for cluster="np-autoscale-cluster" in resourcegroup="np-autoscaling-tskqgd" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster np-autoscale-cluster in resource group np-autoscaling-tskqgd, error: context deadline exceeded
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:95]: failed to create HCP cluster np-autoscale-cluster
Unexpected error:
    <*fmt.wrapErrors | 0xc0011120f0>: 
    failed to create HCP cluster np-autoscale-cluster, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster np-autoscale-cluster in resource group np-autoscaling-tskqgd, error: failed waiting for cluster="np-autoscale-cluster" in resourcegroup="np-autoscaling-tskqgd" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster np-autoscale-cluster in resource group np-autoscaling-tskqgd, error: context deadline exceeded
    ...
occurred
timeout during CreateHCPClusterAndWait; context deadline exceeded
02:00:03 UTC · failed at e2e · test Customer should update NodeDrainTimeoutMinutes and upgrade a nodepool · suite stage/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_drain_timeout_update.go:102]: failed to create HCP cluster with version 4.20.24
Unexpected error:
    <*fmt.wrapErrors | 0xc0016aa750>: 
    failed to create HCP cluster np-drain-timeout-and-upgrade-nx9rt2, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster np-drain-timeout-and-upgrade-nx9rt2 in resource group rg-np-drain-timeout-and-upgrade-nx9rt2-bbgt48, error: failed waiting for cluster="np-drain-timeout-and-upgrade-nx9rt2" in resourcegroup="rg-np-drain-timeout-and-upgrade-nx9rt2-bbgt48" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster np-drain-timeout-and-upgrade-nx9rt2 in resource group rg-np-drain-timeout-and-upgrade-nx9rt2-bbgt48, error: context deadline exceeded
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_drain_timeout_update.go:102]: failed to create HCP cluster with version 4.20.24
Unexpected error:
    <*fmt.wrapErrors | 0xc0016aa750>: 
    failed to create HCP cluster np-drain-timeout-and-upgrade-nx9rt2, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster np-drain-timeout-and-upgrade-nx9rt2 in resource group rg-np-drain-timeout-and-upgrade-nx9rt2-bbgt48, error: failed waiting for cluster="np-drain-timeout-and-upgrade-nx9rt2" in resourcegroup="rg-np-drain-timeout-and-upgrade-nx9rt2-bbgt48" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster np-drain-timeout-and-upgrade-nx9rt2 in resource group rg-np-drain-timeout-and-upgrade-nx9rt2-bbgt48, error: context deadline exceeded
    ...
occurred
timeout during CreateHCPClusterAndWait; context deadline exceeded
02:00:03 UTC · failed at e2e · test Customer should be able to successfully upgrade control plane minor version from 4.21 minor to 4.22 minor · suite stage/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:158]: failed to create HCP cluster "cp-ystream-upgrade-4-22-jfsbmm" with install version 4.21.17
Unexpected error:
    <*fmt.wrapErrors | 0xc0009f6300>: 
    failed to create HCP cluster cp-ystream-upgrade-4-22-jfsbmm, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster cp-ystream-upgrade-4-22-jfsbmm in resource group rg-cp-ystream-upgrade-4-22-xv288j, error: failed waiting for cluster="cp-ystream-upgrade-4-22-jfsbmm" in resourcegroup="rg-cp-ystream-upgrade-4-22-xv288j" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster cp-ystream-upgrade-4-22-jfsbmm in resource group rg-cp-ystream-upgrade-4-22-xv288j, error: context deadline exceeded
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:158]: failed to create HCP cluster "cp-ystream-upgrade-4-22-jfsbmm" with install version 4.21.17
Unexpected error:
    <*fmt.wrapErrors | 0xc0009f6300>: 
    failed to create HCP cluster cp-ystream-upgrade-4-22-jfsbmm, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster cp-ystream-upgrade-4-22-jfsbmm in resource group rg-cp-ystream-upgrade-4-22-xv288j, error: failed waiting for cluster="cp-ystream-upgrade-4-22-jfsbmm" in resourcegroup="rg-cp-ystream-upgrade-4-22-xv288j" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster cp-ystream-upgrade-4-22-jfsbmm in resource group rg-cp-ystream-upgrade-4-22-xv288j, error: context deadline exceeded
    ...
occurred
CreateNodePoolFromParam timed out after <minutes> minutes; context deadline exceeded
02:00:03 UTC · failed at e2e · test Create HCPOpenShiftCluster with Private KeyVault should create a cluster with private keyvault using v20251223preview API · suite stage/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:136]: failed to create node pool "np-1" for private keyvault cluster "private-kv-cluster"
Unexpected error:
    <*fmt.wrapErrors | 0xc000a3e570>: 
    failed to create NodePool np-1, caused by: timeout '20.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group private-keyvault-gsnpnk, error: failed waiting for nodepool="np-1" for cluster "private-kv-cluster" in resourcegroup="private-keyvault-gsnpnk" to finish creating: context deadline exceeded
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:136]: failed to create node pool "np-1" for private keyvault cluster "private-kv-cluster"
Unexpected error:
    <*fmt.wrapErrors | 0xc000a3e570>: 
    failed to create NodePool np-1, caused by: timeout '20.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group private-keyvault-gsnpnk, error: failed waiting for nodepool="np-1" for cluster "private-kv-cluster" in resourcegroup="private-keyvault-gsnpnk" to finish creating: context deadline exceeded
    ...
occurred