CIHealth
<
Single Day: Aug 18▾
Pick a day
>
View JSON API
↗ Open Failure patterns for this day

Environment: INT

Time (UTC)JobFailed atResultPRFailed testsDetails
branch-ci-Azure-ARO-HCP-main-e2e-integration-e2e-parallel
post-goodmerged PR
No additional run metadata captured
 e2e failed n/a 1
e2e (1)
tls: failed to verify certificate: x509: certificate signed by unknown authority
21:33:39 UTC · failed at e2e · test Customer should be able to create a no-CNI private cluster with a private key vault, a nodepool and install cilium CNI successfully · suite integration/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:196]: failed to run simple web app and connectivity check with cilium CNI
Unexpected error:
    <*errors.joinError | 0xc0018ea198>: 
    VerifySimpleWebApp failed: [strict TLS verification] route was never reachable: Get "https://agnhost-e2e-sample-app-jf5mq.apps.aro.cilium-cluster.exnb.uksouth.aroapp-hcp.azure-test.net": tls: failed to verify certificate: x509: certificate signed by unknown authority
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:196]: failed to run simple web app and connectivity check with cilium CNI
Unexpected error:
    <*errors.joinError | 0xc0018ea198>: 
    VerifySimpleWebApp failed: [strict TLS verification] route was never reachable: Get "https://agnhost-e2e-sample-app-jf5mq.apps.aro.cilium-cluster.exnb.uksouth.aroapp-hcp.azure-test.net": tls: failed to verify certificate: x509: certificate signed by unknown authority
    ...
occurred
branch-ci-Azure-ARO-HCP-main-e2e-integration-e2e-parallel
post-goodmerged PR
No additional run metadata captured
 e2e failed n/a 1
e2e (1)
ERROR CODE: InternalServerError; detail message [clusterServiceNodePoolStatus] <no_message>; provider Microsoft.RedHatOpenShift
18:37:47 UTC · failed at e2e · test Customer should be able to create an HCP cluster and custom node pool osDisk size · suite integration/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_nodepool_osdisk.go:96]: failed to create node pool "nodepool-128GiB" with 128GiB osDisk
Unexpected error:
    <*fmt.wrapError | 0xc00098d800>: 
    failed to create NodePool nodepool-128GiB: failed waiting for nodepool="nodepool-128GiB" for cluster "hcp-cluster-np-128" in resourcegroup="clusternp128-txwg4qgs4hsw" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/002755ed-c845-4398-a37b-8daa3b7cdc45
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/002755ed-c845-4398-a37b-8daa3b7cdc45",
      "name": "002755ed-c845-4398-a37b-8daa3b7cdc45",
      "status": "Failed",
      "startTime": "2026-08-18T18:50:47.72810349Z",
      "endTime": "2026-08-18T19:09:54.253810369Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceNodePoolStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_nodepool_osdisk.go:96]: failed to create node pool "nodepool-128GiB" with 128GiB osDisk
Unexpected error:
    <*fmt.wrapError | 0xc00098d800>: 
    failed to create NodePool nodepool-128GiB: failed waiting for nodepool="nodepool-128GiB" for cluster "hcp-cluster-np-128" in resourcegroup="clusternp128-txwg4qgs4hsw" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/002755ed-c845-4398-a37b-8daa3b7cdc45
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/002755ed-c845-4398-a37b-8daa3b7cdc45",
      "name": "002755ed-c845-4398-a37b-8daa3b7cdc45",
      "status": "Failed",
      "startTime": "2026-08-18T18:50:47.72810349Z",
      "endTime": "2026-08-18T19:09:54.253810369Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceNodePoolStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
branch-ci-Azure-ARO-HCP-main-e2e-integration-e2e-parallel
post-goodmerged PR
No additional run metadata captured
 e2e failed n/a 2
e2e (2)
x509: certificate signed by unknown authority
15:11:43 UTC · failed at e2e · test Customer should create an HCP cluster and validate TLS certificates · suite integration/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:236]: Timed out after 600.000s.
expect ingress certificate to be signed by a trusted Azure CA
Expected success, but got an error:
    <x509.UnknownAuthorityError>: 
    x509: certificate signed by unknown authority
    ...
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:236]: Timed out after 600.000s.
expect ingress certificate to be signed by a trusted Azure CA
Expected success, but got an error:
    <x509.UnknownAuthorityError>: 
    x509: certificate signed by unknown authority
    ...
ERROR CODE: InternalServerError; detail message cluster deletion did not complete before the deadline; [clusterServiceDeletion] ClusterService cluster still exists; [clusterServiceStatus] state is "uninstalling"; [descendantResources] remaining resources: serviceProviderClusters; provider Microsoft.RedHatOpenShift
15:11:43 UTC · failed at e2e · test Customer should be able to create a no-CNI private cluster with a private key vault, a nodepool and install cilium CNI successfully · suite integration/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:293]: Unexpected error:
    <*errors.joinError | 0xc00101a6a8>: 
    failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster="cilium-cluster" in resourcegroup="complex-cilium-kv-rkhdl8xqhdmz" to finish deleting: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0d86f430-60d6-4c0f-b439-c3ac3403f901
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0d86f430-60d6-4c0f-b439-c3ac3403f901",
      "name": "0d86f430-60d6-4c0f-b439-c3ac3403f901",
      "status": "Failed",
      "startTime": "2026-08-18T16:03:01.445081159Z",
      "endTime": "2026-08-18T16:27:09.852869865Z",
      "error": {
        "code": "InternalServerError",
        "message": "cluster deletion did not complete before the deadline; [clusterServiceDeletion] ClusterService cluster /api/aro_hcp/v1alpha1/clusters/2s92v1tvs13qsc1h7oqpk49j9bta62qs still exists (deletion dispatched at 2026-08-18T16:03:02Z); [clusterServiceStatus] ClusterService state is \"uninstalling\"; [descendantResources] remaining resources: 1 microsoft.redhatopenshift/hcpopenshiftclusters/serviceProviderClusters"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:293]: Unexpected error:
    <*errors.joinError | 0xc00101a6a8>: 
    failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster="cilium-cluster" in resourcegroup="complex-cilium-kv-rkhdl8xqhdmz" to finish deleting: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0d86f430-60d6-4c0f-b439-c3ac3403f901
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0d86f430-60d6-4c0f-b439-c3ac3403f901",
      "name": "0d86f430-60d6-4c0f-b439-c3ac3403f901",
      "status": "Failed",
      "startTime": "2026-08-18T16:03:01.445081159Z",
      "endTime": "2026-08-18T16:27:09.852869865Z",
      "error": {
        "code": "InternalServerError",
        "message": "cluster deletion did not complete before the deadline; [clusterServiceDeletion] ClusterService cluster /api/aro_hcp/v1alpha1/clusters/2s92v1tvs13qsc1h7oqpk49j9bta62qs still exists (deletion dispatched at 2026-08-18T16:03:02Z); [clusterServiceStatus] ClusterService state is \"uninstalling\"; [descendantResources] remaining resources: 1 microsoft.redhatopenshift/hcpopenshiftclusters/serviceProviderClusters"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
branch-ci-Azure-ARO-HCP-main-e2e-integration-e2e-parallel
post-goodmerged PR
No additional run metadata captured
 e2e failed n/a 1
e2e (1)
ERROR CODE: InternalServerError; detail message [clusterServiceNodePoolStatus] <no_message>; provider Microsoft.RedHatOpenShift
11:56:33 UTC · failed at e2e · test HCP Nodepools GPU instances creates and deletes a GPU nodepool in a single cluster · suite integration/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/gpu_nodepools_create_delete.go:131]: failed to create GPU nodepool gpu-np-1 with VM size Standard_NC4as_T4_v3
Unexpected error:
    <*fmt.wrapError | 0xc000a9e9c0>: 
    failed to create NodePool gpu-np-1: failed waiting for nodepool="gpu-np-1" for cluster "cluster-gpu-np" in resourcegroup="rg-gpu-nodepool-6rqw64hz9r29" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/df04d9fe-d188-4567-a852-79007382c2f0
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/df04d9fe-d188-4567-a852-79007382c2f0",
      "name": "df04d9fe-d188-4567-a852-79007382c2f0",
      "status": "Failed",
      "startTime": "2026-08-18T13:10:13.902866674Z",
      "endTime": "2026-08-18T13:29:15.631640019Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceNodePoolStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/gpu_nodepools_create_delete.go:131]: failed to create GPU nodepool gpu-np-1 with VM size Standard_NC4as_T4_v3
Unexpected error:
    <*fmt.wrapError | 0xc000a9e9c0>: 
    failed to create NodePool gpu-np-1: failed waiting for nodepool="gpu-np-1" for cluster "cluster-gpu-np" in resourcegroup="rg-gpu-nodepool-6rqw64hz9r29" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/df04d9fe-d188-4567-a852-79007382c2f0
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/df04d9fe-d188-4567-a852-79007382c2f0",
      "name": "df04d9fe-d188-4567-a852-79007382c2f0",
      "status": "Failed",
      "startTime": "2026-08-18T13:10:13.902866674Z",
      "endTime": "2026-08-18T13:29:15.631640019Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceNodePoolStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
branch-ci-Azure-ARO-HCP-main-e2e-integration-e2e-parallel
post-goodmerged PR
No additional run metadata captured
 n/a passed n/a 0 n/a
branch-ci-Azure-ARO-HCP-main-e2e-integration-e2e-parallel
post-goodmerged PR
No additional run metadata captured
 n/a passed n/a 0 n/a