<
Single Day: Jun 18
Pick a day
>
View JSON API
↗ Open Failure patterns for this day

Environment: PROD

Time (UTC)JobFailed atResultPRFailed testsDetails
periodic-ci-Azure-ARO-HCP-main-periodic-prod-e2e-parallel-ocp-nightly
post-goodmerged PR
No additional run metadata captured
e2e failed n/a 2
e2e (2)
Should fail with EOF error indicating blocked connection
23:00:22 UTC · failed at e2e · test Authorized CIDRs Connectivity should allow API access only from authorized VM IP · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:190]: Should fail with EOF error indicating blocked connection
Expected
    <string>: ...
to contain substring
    <string>: ...
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:190]: Should fail with EOF error indicating blocked connection
Expected
    <string>: ...
to contain substring
    <string>: ...
CreateNodePoolFromParam timed out after <minutes> minutes; context deadline exceeded
23:00:22 UTC · failed at e2e · test Create HCPOpenShiftCluster with Private KeyVault should create a cluster with private keyvault using v20251223preview API · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:136]: failed to create node pool "np-1" for private keyvault cluster "private-kv-cluster"
Unexpected error:
    <*fmt.wrapErrors | 0xc0009e83c0>: 
    failed to create NodePool np-1, caused by: timeout '20.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group private-keyvault-6m9cck, error: failed waiting for nodepool="np-1" for cluster "private-kv-cluster" in resourcegroup="private-keyvault-6m9cck" to finish creating: context deadline exceeded
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:136]: failed to create node pool "np-1" for private keyvault cluster "private-kv-cluster"
Unexpected error:
    <*fmt.wrapErrors | 0xc0009e83c0>: 
    failed to create NodePool np-1, caused by: timeout '20.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group private-keyvault-6m9cck, error: failed waiting for nodepool="np-1" for cluster "private-kv-cluster" in resourcegroup="private-keyvault-6m9cck" to finish creating: context deadline exceeded
    ...
occurred
branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel
post-goodmerged PR
No additional run metadata captured
n/a passed n/a 0 n/a
branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel
post-goodmerged PR
No additional run metadata captured
e2e failed n/a 2
e2e (2)
timeout during CreateHCPClusterAndWait; context deadline exceeded
16:20:38 UTC · failed at e2e · test Customer should be able to lifecycle and confirm external auth on a cluster · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_list_and_verify.go:83]: failed to create HCP cluster for external auth list test
Unexpected error:
    <*fmt.wrapErrors | 0xc0012cf560>: 
    failed to create HCP cluster ea-listdw56tc, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster ea-listdw56tc in resource group ea-list-pfrc2w, error: failed waiting for cluster="ea-listdw56tc" in resourcegroup="ea-list-pfrc2w" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster ea-listdw56tc in resource group ea-list-pfrc2w, error: context deadline exceeded
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_list_and_verify.go:83]: failed to create HCP cluster for external auth list test
Unexpected error:
    <*fmt.wrapErrors | 0xc0012cf560>: 
    failed to create HCP cluster ea-listdw56tc, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster ea-listdw56tc in resource group ea-list-pfrc2w, error: failed waiting for cluster="ea-listdw56tc" in resourcegroup="ea-list-pfrc2w" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster ea-listdw56tc in resource group ea-list-pfrc2w, error: context deadline exceeded
    ...
occurred
CreateNodePoolFromParam timed out after <minutes> minutes; context deadline exceeded
16:20:38 UTC · failed at e2e · test Nodepool Ephemeral OS Disk should create a nodepool with ephemeral OS disk when autoRepair is enabled · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_ephemeral_osdisk.go:135]: failed to create nodepool ephemeral-np with ephemeral OS disk
Unexpected error:
    <*fmt.wrapErrors | 0xc0015c0990>: 
    failed to create NodePool ephemeral-np, caused by: timeout '20.000000' minutes exceeded during CreateNodePoolFromParam for node pool ephemeral-np in resource group ephemeral-osdisk-fkxsj5, error: failed waiting for nodepool="ephemeral-np" for cluster "ephemeral-disk" in resourcegroup="ephemeral-osdisk-fkxsj5" to finish creating: context deadline exceeded
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_ephemeral_osdisk.go:135]: failed to create nodepool ephemeral-np with ephemeral OS disk
Unexpected error:
    <*fmt.wrapErrors | 0xc0015c0990>: 
    failed to create NodePool ephemeral-np, caused by: timeout '20.000000' minutes exceeded during CreateNodePoolFromParam for node pool ephemeral-np in resource group ephemeral-osdisk-fkxsj5, error: failed waiting for nodepool="ephemeral-np" for cluster "ephemeral-disk" in resourcegroup="ephemeral-osdisk-fkxsj5" to finish creating: context deadline exceeded
    ...
occurred
branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel
post-goodmerged PR
No additional run metadata captured
e2e failed n/a 1
e2e (1)
client rate limiter Wait returned an error: context deadline exceeded
16:11:15 UTC · failed at e2e · test Customer should be able to create a no-CNI private cluster with a private key vault, a nodepool and install cilium CNI successfully · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:190]: failed to verify nodes are Ready with Cilium CNI for cluster "cilium-cluster"
Unexpected error:
    <*errors.joinError | 0xc000b508d0>: 
    VerifyNodesReady failed: no nodes found in the cluster
    VerifyCiliumOperational failed: not all pods in kube-system namespace are running: failed to list pods in kube-system namespace: client rate limiter Wait returned an error: context deadline exceeded
    failed to create NodePool cilium-np, caused by: timeout '20.000000' minutes exceeded during CreateNodePoolFromParam for node pool cilium-np in resource group complex-cilium-kv-mgjdp6, error: failed waiting for nodepool="cilium-np" for cluster "cilium-cluster" in resourcegroup="complex-cilium-kv-mgjdp6" to finish creating: context deadline exceeded
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:190]: failed to verify nodes are Ready with Cilium CNI for cluster "cilium-cluster"
Unexpected error:
    <*errors.joinError | 0xc000b508d0>: 
    VerifyNodesReady failed: no nodes found in the cluster
    VerifyCiliumOperational failed: not all pods in kube-system namespace are running: failed to list pods in kube-system namespace: client rate limiter Wait returned an error: context deadline exceeded
    failed to create NodePool cilium-np, caused by: timeout '20.000000' minutes exceeded during CreateNodePoolFromParam for node pool cilium-np in resource group complex-cilium-kv-mgjdp6, error: failed waiting for nodepool="cilium-np" for cluster "cilium-cluster" in resourcegroup="complex-cilium-kv-mgjdp6" to finish creating: context deadline exceeded
    ...
occurred
branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel
post-goodmerged PR
No additional run metadata captured
e2e failed n/a 2
e2e (2)
ERROR CODE: DeploymentNotFound
13:21:02 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 4.20 · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:92]: failed to create customer resources for z-stream cluster "cluster-zstream-4-20-fw7f2d"
Unexpected error:
    <*fmt.wrapError | 0xc000f9b3c0>: 
    failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cluster-zstream-4-20-fw7f2d-k89rj5" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-zstream-4-20-fw7f2d-k89rj5/operationStatuses/08584198113386810784
    --------------------------------------------------------------------------------
    RESPONSE 404: 404 Not Found
    ERROR CODE: DeploymentNotFound
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "DeploymentNotFound",
        "message": "Deployment 'mi-cluster-zstream-4-20-fw7f2d-k89rj5' could not be found."
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:92]: failed to create customer resources for z-stream cluster "cluster-zstream-4-20-fw7f2d"
Unexpected error:
    <*fmt.wrapError | 0xc000f9b3c0>: 
    failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cluster-zstream-4-20-fw7f2d-k89rj5" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-zstream-4-20-fw7f2d-k89rj5/operationStatuses/08584198113386810784
    --------------------------------------------------------------------------------
    RESPONSE 404: 404 Not Found
    ERROR CODE: DeploymentNotFound
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "DeploymentNotFound",
        "message": "Deployment 'mi-cluster-zstream-4-20-fw7f2d-k89rj5' could not be found."
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
CreateNodePoolFromParam timed out after <minutes> minutes; context deadline exceeded
13:21:02 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.21 · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:159]: failed to create node pool "np-c5q44t" for cluster "cluster-candidate-4-21-c5q44t"
Unexpected error:
    <*fmt.wrapErrors | 0xc0009f06c0>: 
    failed to create NodePool np-c5q44t, caused by: timeout '20.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-c5q44t in resource group rg-candidate-4-21-kztrgv, error: failed waiting for nodepool="np-c5q44t" for cluster "cluster-candidate-4-21-c5q44t" in resourcegroup="rg-candidate-4-21-kztrgv" to finish creating: context deadline exceeded
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:159]: failed to create node pool "np-c5q44t" for cluster "cluster-candidate-4-21-c5q44t"
Unexpected error:
    <*fmt.wrapErrors | 0xc0009f06c0>: 
    failed to create NodePool np-c5q44t, caused by: timeout '20.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-c5q44t in resource group rg-candidate-4-21-kztrgv, error: failed waiting for nodepool="np-c5q44t" for cluster "cluster-candidate-4-21-c5q44t" in resourcegroup="rg-candidate-4-21-kztrgv" to finish creating: context deadline exceeded
    ...
occurred
branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel
post-goodmerged PR
No additional run metadata captured
e2e failed n/a 1
e2e (1)
timeout during CreateHCPClusterAndWait; context deadline exceeded
08:28:32 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 5.0 · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:106]: HCP cluster rg-candidate-5-0-swwhjt/cluster-candidate-5-0-frc9tc should provision
Unexpected error:
    <*fmt.wrapErrors | 0xc0006eade0>: 
    failed to create HCP cluster cluster-candidate-5-0-frc9tc, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster cluster-candidate-5-0-frc9tc in resource group rg-candidate-5-0-swwhjt, error: failed waiting for cluster="cluster-candidate-5-0-frc9tc" in resourcegroup="rg-candidate-5-0-swwhjt" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster cluster-candidate-5-0-frc9tc in resource group rg-candidate-5-0-swwhjt, error: context deadline exceeded
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:106]: HCP cluster rg-candidate-5-0-swwhjt/cluster-candidate-5-0-frc9tc should provision
Unexpected error:
    <*fmt.wrapErrors | 0xc0006eade0>: 
    failed to create HCP cluster cluster-candidate-5-0-frc9tc, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster cluster-candidate-5-0-frc9tc in resource group rg-candidate-5-0-swwhjt, error: failed waiting for cluster="cluster-candidate-5-0-frc9tc" in resourcegroup="rg-candidate-5-0-swwhjt" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster cluster-candidate-5-0-frc9tc in resource group rg-candidate-5-0-swwhjt, error: context deadline exceeded
    ...
occurred
branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel
post-goodmerged PR
No additional run metadata captured
n/a passed n/a 0 n/a
branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel
post-goodmerged PR
No additional run metadata captured
e2e failed n/a 1
e2e (1)
timeout during CreateHCPClusterAndWait; context deadline exceeded
05:01:00 UTC · failed at e2e · test Customer should be able to create an HCP cluster with Image Registry not present · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/image_registry_cluster_create.go:88]: failed to create HCP cluster with image registry disabled
Unexpected error:
    <*fmt.wrapErrors | 0xc00144c750>: 
    failed to create HCP cluster disabled-image-registry-hcp-cluster, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster disabled-image-registry-hcp-cluster in resource group disabled-image-registry-5b4j95, error: failed waiting for cluster="disabled-image-registry-hcp-cluster" in resourcegroup="disabled-image-registry-5b4j95" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster disabled-image-registry-hcp-cluster in resource group disabled-image-registry-5b4j95, error: context deadline exceeded
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/image_registry_cluster_create.go:88]: failed to create HCP cluster with image registry disabled
Unexpected error:
    <*fmt.wrapErrors | 0xc00144c750>: 
    failed to create HCP cluster disabled-image-registry-hcp-cluster, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster disabled-image-registry-hcp-cluster in resource group disabled-image-registry-5b4j95, error: failed waiting for cluster="disabled-image-registry-hcp-cluster" in resourcegroup="disabled-image-registry-5b4j95" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster disabled-image-registry-hcp-cluster in resource group disabled-image-registry-5b4j95, error: context deadline exceeded
    ...
occurred
branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel
post-goodmerged PR
No additional run metadata captured
n/a passed n/a 0 n/a
branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel
post-goodmerged PR
No additional run metadata captured
unknown failed n/a 1 non-artifact-backed failure (no junit artifacts)
periodic-ci-Azure-ARO-HCP-main-periodic-prod-e2e-parallel
post-goodmerged PR
No additional run metadata captured
n/a passed n/a 0 n/a
branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel
post-goodmerged PR
No additional run metadata captured
n/a passed n/a 0 n/a