CIHealth
<
Single Day: Jul 9▾
Pick a day
>
View JSON API
↗ Open Failure patterns for this day

Environment: PROD

Time (UTC)JobFailed atResultPRFailed testsDetails
periodic-ci-Azure-ARO-HCP-main-periodic-prod-e2e-parallel-ocp-nightly
post-goodmerged PR
No additional run metadata captured
 e2e failed n/a 2
e2e (2)
tls: failed to verify certificate: x509: certificate signed by unknown authority
23:00:24 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP nightly channel for 4.21 · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:172]: failed to verify simple web app runs on cluster "cluster-nightly-4-21-pgpqxt"
Unexpected error:
    <*fmt.wrapError | 0xc00102e060>: 
    route was never reachable: Get "https://agnhost-e2e-serving-app-njf6z.apps.aro.d8f0y3s7c3c7p2d.ek0b.uksouth.aroapp-hcp.io": tls: failed to verify certificate: x509: certificate signed by unknown authority
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:172]: failed to verify simple web app runs on cluster "cluster-nightly-4-21-pgpqxt"
Unexpected error:
    <*fmt.wrapError | 0xc00102e060>: 
    route was never reachable: Get "https://agnhost-e2e-serving-app-njf6z.apps.aro.d8f0y3s7c3c7p2d.ek0b.uksouth.aroapp-hcp.io": tls: failed to verify certificate: x509: certificate signed by unknown authority
    ...
occurred
x509: certificate signed by unknown authority
23:00:24 UTC · failed at e2e · test Customer should create an HCP cluster and validate TLS certificates · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:167]: Timed out after 600.001s.
expect ingress certificate to be signed by a trusted Azure CA
Expected success, but got an error:
    <x509.UnknownAuthorityError>: 
    x509: certificate signed by unknown authority
    ...
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:167]: Timed out after 600.001s.
expect ingress certificate to be signed by a trusted Azure CA
Expected success, but got an error:
    <x509.UnknownAuthorityError>: 
    x509: certificate signed by unknown authority
    ...
branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel
post-goodmerged PR
No additional run metadata captured
 unknown failed n/a 1 non-artifact-backed failure (no junit artifacts)
branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel
post-goodmerged PR
No additional run metadata captured
 n/a passed n/a 0 n/a
branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel
post-goodmerged PR
No additional run metadata captured
 e2e failed n/a 1
e2e (1)
UpdateNodePoolAndWait timed out after <minutes> minutes; context deadline exceeded
15:23:08 UTC · failed at e2e · test Customer should be able to update node pool labels and taints · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_labels_taints.go:185]: failed to update node pool np-1 with new taints and scale up
Unexpected error:
    <*fmt.wrapErrors | 0xc0027caba0>: 
    failed waiting for nodepool="np-1" in cluster="cluster-np-labels-taints" resourcegroup="rg-np-labels-taints-9qjlw2" to finish updating, caused by: timeout '20.000000' minutes exceeded during UpdateNodePoolAndWait for nodepool np-1 in cluster cluster-np-labels-taints in resource group rg-np-labels-taints-9qjlw2, error: context deadline exceeded
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_labels_taints.go:185]: failed to update node pool np-1 with new taints and scale up
Unexpected error:
    <*fmt.wrapErrors | 0xc0027caba0>: 
    failed waiting for nodepool="np-1" in cluster="cluster-np-labels-taints" resourcegroup="rg-np-labels-taints-9qjlw2" to finish updating, caused by: timeout '20.000000' minutes exceeded during UpdateNodePoolAndWait for nodepool np-1 in cluster cluster-np-labels-taints in resource group rg-np-labels-taints-9qjlw2, error: context deadline exceeded
    ...
occurred
branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel
post-goodmerged PR
No additional run metadata captured
 e2e failed n/a 1
e2e (1)
x509: certificate signed by unknown authority
15:05:53 UTC · failed at e2e · test Customer should create an HCP cluster and validate TLS certificates · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:167]: Timed out after 600.003s.
expect ingress certificate to be signed by a trusted Azure CA
Expected success, but got an error:
    <x509.UnknownAuthorityError>: 
    x509: certificate signed by unknown authority
    ...
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:167]: Timed out after 600.003s.
expect ingress certificate to be signed by a trusted Azure CA
Expected success, but got an error:
    <x509.UnknownAuthorityError>: 
    x509: certificate signed by unknown authority
    ...
branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel
post-goodmerged PR
No additional run metadata captured
 e2e failed n/a 2
e2e (2)
client rate limiter Wait returned an error: context deadline exceeded
11:27:24 UTC · failed at e2e · test Customer should be able to create a no-CNI private cluster with a private key vault, a nodepool and install cilium CNI successfully · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:190]: failed to verify nodes are Ready with Cilium CNI for cluster "cilium-cluster"
Unexpected error:
    <*errors.joinError | 0xc001a6a3a8>: 
    VerifyNodesReady failed: no nodes found in the cluster
    VerifyCiliumOperational failed: not all pods in kube-system namespace are running: failed to list pods in kube-system namespace: client rate limiter Wait returned an error: context deadline exceeded
    failed to create NodePool cilium-np, caused by: timeout '20.000000' minutes exceeded during CreateNodePoolFromParam for node pool cilium-np in resource group complex-cilium-kv-rbkvwl, error: failed waiting for nodepool="cilium-np" for cluster "cilium-cluster" in resourcegroup="complex-cilium-kv-rbkvwl" to finish creating: context deadline exceeded
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:190]: failed to verify nodes are Ready with Cilium CNI for cluster "cilium-cluster"
Unexpected error:
    <*errors.joinError | 0xc001a6a3a8>: 
    VerifyNodesReady failed: no nodes found in the cluster
    VerifyCiliumOperational failed: not all pods in kube-system namespace are running: failed to list pods in kube-system namespace: client rate limiter Wait returned an error: context deadline exceeded
    failed to create NodePool cilium-np, caused by: timeout '20.000000' minutes exceeded during CreateNodePoolFromParam for node pool cilium-np in resource group complex-cilium-kv-rbkvwl, error: failed waiting for nodepool="cilium-np" for cluster "cilium-cluster" in resourcegroup="complex-cilium-kv-rbkvwl" to finish creating: context deadline exceeded
    ...
occurred
timeout during CreateHCPClusterAndWait; context deadline exceeded
11:27:24 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 5.0 · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:106]: HCP cluster rg-candidate-5-0-fmc67g/cluster-candidate-5-0-78f45c should provision
Unexpected error:
    <*fmt.wrapErrors | 0xc000752420>: 
    failed to create HCP cluster cluster-candidate-5-0-78f45c, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster cluster-candidate-5-0-78f45c in resource group rg-candidate-5-0-fmc67g, error: failed waiting for cluster="cluster-candidate-5-0-78f45c" in resourcegroup="rg-candidate-5-0-fmc67g" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster cluster-candidate-5-0-78f45c in resource group rg-candidate-5-0-fmc67g, error: context deadline exceeded
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:106]: HCP cluster rg-candidate-5-0-fmc67g/cluster-candidate-5-0-78f45c should provision
Unexpected error:
    <*fmt.wrapErrors | 0xc000752420>: 
    failed to create HCP cluster cluster-candidate-5-0-78f45c, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster cluster-candidate-5-0-78f45c in resource group rg-candidate-5-0-fmc67g, error: failed waiting for cluster="cluster-candidate-5-0-78f45c" in resourcegroup="rg-candidate-5-0-fmc67g" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster cluster-candidate-5-0-78f45c in resource group rg-candidate-5-0-fmc67g, error: context deadline exceeded
    ...
occurred
branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel
post-goodmerged PR
No additional run metadata captured
 e2e failed n/a 1
e2e (1)
timeout during CreateHCPClusterAndWait; context deadline exceeded
08:27:45 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 5.0 · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:106]: HCP cluster rg-candidate-5-0-4kczhg/cluster-candidate-5-0-g7fggr should provision
Unexpected error:
    <*fmt.wrapErrors | 0xc0015af680>: 
    failed to create HCP cluster cluster-candidate-5-0-g7fggr, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster cluster-candidate-5-0-g7fggr in resource group rg-candidate-5-0-4kczhg, error: failed waiting for cluster="cluster-candidate-5-0-g7fggr" in resourcegroup="rg-candidate-5-0-4kczhg" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster cluster-candidate-5-0-g7fggr in resource group rg-candidate-5-0-4kczhg, error: context deadline exceeded
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:106]: HCP cluster rg-candidate-5-0-4kczhg/cluster-candidate-5-0-g7fggr should provision
Unexpected error:
    <*fmt.wrapErrors | 0xc0015af680>: 
    failed to create HCP cluster cluster-candidate-5-0-g7fggr, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster cluster-candidate-5-0-g7fggr in resource group rg-candidate-5-0-4kczhg, error: failed waiting for cluster="cluster-candidate-5-0-g7fggr" in resourcegroup="rg-candidate-5-0-4kczhg" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster cluster-candidate-5-0-g7fggr in resource group rg-candidate-5-0-4kczhg, error: context deadline exceeded
    ...
occurred
branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel
post-goodmerged PR
No additional run metadata captured
 n/a passed n/a 0 n/a
branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel
post-goodmerged PR
No additional run metadata captured
 n/a passed n/a 0 n/a
branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel
post-goodmerged PR
No additional run metadata captured
 n/a passed n/a 0 n/a
branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel
post-goodmerged PR
No additional run metadata captured
 e2e failed n/a 1
e2e (1)
tls: failed to verify certificate: x509: certificate signed by unknown authority
04:26:09 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.20 · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:172]: failed to verify simple web app runs on cluster "cluster-candidate-4-20-grqfrx"
Unexpected error:
    <*fmt.wrapError | 0xc000498440>: 
    route was never reachable: Get "https://agnhost-e2e-serving-app-rvc2j.apps.aro.b6f7d0t5b9c7m1v.opfq.uksouth.aroapp-hcp.io": tls: failed to verify certificate: x509: certificate signed by unknown authority
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:172]: failed to verify simple web app runs on cluster "cluster-candidate-4-20-grqfrx"
Unexpected error:
    <*fmt.wrapError | 0xc000498440>: 
    route was never reachable: Get "https://agnhost-e2e-serving-app-rvc2j.apps.aro.b6f7d0t5b9c7m1v.opfq.uksouth.aroapp-hcp.io": tls: failed to verify certificate: x509: certificate signed by unknown authority
    ...
occurred
periodic-ci-Azure-ARO-HCP-main-periodic-prod-e2e-parallel
post-goodmerged PR
No additional run metadata captured
 n/a passed n/a 0 n/a
branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel
post-goodmerged PR
No additional run metadata captured
 e2e failed n/a 1
e2e (1)
tls: failed to verify certificate: x509: certificate signed by unknown authority
00:23:39 UTC · failed at e2e · test Customer should be able to create an HCP cluster with back-level version 4.19 · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:197]: failed to verify simple web app on back-level cluster version 4.19
Unexpected error:
    <*fmt.wrapError | 0xc000f18c00>: 
    route was never reachable: Get "https://agnhost-e2e-serving-app-wtf7m.apps.aro.e8g5g8f5f1b8d7k.ykxw.uksouth.aroapp-hcp.io": tls: failed to verify certificate: x509: certificate signed by unknown authority
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:197]: failed to verify simple web app on back-level cluster version 4.19
Unexpected error:
    <*fmt.wrapError | 0xc000f18c00>: 
    route was never reachable: Get "https://agnhost-e2e-serving-app-wtf7m.apps.aro.e8g5g8f5f1b8d7k.ykxw.uksouth.aroapp-hcp.io": tls: failed to verify certificate: x509: certificate signed by unknown authority
    ...
occurred