CIHealth Run Log

Environment: PROD

Time (UTC)	Job	Failed at	Result	PR	Failed tests	Details
14:42:26	periodic-ci-Azure-ARO-HCP-main-periodic-prod-e2e-parallel post-goodmerged PR No additional run metadata captured	e2e	failed	n/a	1	e2e (1) expected 4 nodes, found 5 14:42:26 UTC · failed at e2e · test Customer should be able to update nodepool replicas and autoscaling · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_update_nodes.go:262]: Expected success, but got an error: <errors.errorString \| 0xc0006b7ab0>: expected 4 nodes, found 5 ... fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_update_nodes.go:262]: Expected success, but got an error: <errors.errorString \| 0xc0006b7ab0>: expected 4 nodes, found 5 ...
09:57:46	periodic-ci-Azure-ARO-HCP-main-periodic-prod-e2e-parallel post-goodmerged PR No additional run metadata captured	e2e	failed	n/a	2	e2e (2) expected 4 nodes, found 5 09:57:46 UTC · failed at e2e · test Customer should be able to update nodepool replicas and autoscaling · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_update_nodes.go:262]: Expected success, but got an error: <errors.errorString \| 0xc0008f1050>: expected 4 nodes, found 5 ... fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_update_nodes.go:262]: Expected success, but got an error: <errors.errorString \| 0xc0008f1050>: expected 4 nodes, found 5 ... CreateNodePoolFromParam timed out after <minutes> minutes; context deadline exceeded 09:57:46 UTC · failed at e2e · test Create HCPOpenShiftCluster with Private KeyVault should create a cluster with private keyvault using v20251223preview API · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:151]: Unexpected error: <fmt.wrapErrors \| 0xc001012960>: failed to create NodePool np-1, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group private-keyvault-965gz6, error: failed waiting for nodepool="np-1" for cluster "private-kv-cluster" in resourcegroup="private-keyvault-965gz6" to finish creating: context deadline exceeded ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:151]: Unexpected error: <fmt.wrapErrors \| 0xc001012960>: failed to create NodePool np-1, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group private-keyvault-965gz6, error: failed waiting for nodepool="np-1" for cluster "private-kv-cluster" in resourcegroup="private-keyvault-965gz6" to finish creating: context deadline exceeded ... occurred
07:46:41	periodic-ci-Azure-ARO-HCP-main-periodic-prod-e2e-parallel post-goodmerged PR No additional run metadata captured	e2e	failed	n/a	1	e2e (1) CreateNodePoolFromParam timed out after <minutes> minutes; context deadline exceeded 07:46:41 UTC · failed at e2e · test Create HCPOpenShiftCluster with Private KeyVault should create a cluster with private keyvault using v20251223preview API · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:151]: Unexpected error: <fmt.wrapErrors \| 0xc000e00090>: failed to create NodePool np-1, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group private-keyvault-8qlspn, error: failed waiting for nodepool="np-1" for cluster "private-kv-cluster" in resourcegroup="private-keyvault-8qlspn" to finish creating: context deadline exceeded ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:151]: Unexpected error: <fmt.wrapErrors \| 0xc000e00090>: failed to create NodePool np-1, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group private-keyvault-8qlspn, error: failed waiting for nodepool="np-1" for cluster "private-kv-cluster" in resourcegroup="private-keyvault-8qlspn" to finish creating: context deadline exceeded ... occurred
07:27:21	periodic-ci-Azure-ARO-HCP-main-periodic-prod-e2e-parallel post-goodmerged PR No additional run metadata captured	n/a	passed	n/a	0	n/a
03:36:54	periodic-ci-Azure-ARO-HCP-main-periodic-prod-e2e-parallel post-goodmerged PR No additional run metadata captured	e2e	failed	n/a	11	e2e (11) timeout during CreateHCPClusterAndWait; context deadline exceeded 03:36:54 UTC · failed at e2e · test Customer should be able to update node pool labels and taints · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_labels_taints.go:91]: Unexpected error: <fmt.wrapErrors \| 0xc00095ee70>: failed to create HCP cluster cluster-np-labels-taints, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster cluster-np-labels-taints in resource group rg-np-labels-taints-98sfgl, error: failed waiting for cluster="cluster-np-labels-taints" in resourcegroup="rg-np-labels-taints-98sfgl" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster cluster-np-labels-taints in resource group rg-np-labels-taints-98sfgl, error: context deadline exceeded ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_labels_taints.go:91]: Unexpected error: <fmt.wrapErrors \| 0xc00095ee70>: failed to create HCP cluster cluster-np-labels-taints, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster cluster-np-labels-taints in resource group rg-np-labels-taints-98sfgl, error: failed waiting for cluster="cluster-np-labels-taints" in resourcegroup="rg-np-labels-taints-98sfgl" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster cluster-np-labels-taints in resource group rg-np-labels-taints-98sfgl, error: context deadline exceeded ... occurred timeout during CreateHCPClusterAndWait; context deadline exceeded 03:36:54 UTC · failed at e2e · test Customer should be able to create an HCP cluster with Image Registry not present · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/image_registry_cluster_create.go:89]: Unexpected error: <fmt.wrapErrors \| 0xc001709110>: failed to create HCP cluster disabled-image-registry-hcp-cluster, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster disabled-image-registry-hcp-cluster in resource group disabled-image-registry-plxw8q, error: failed waiting for cluster="disabled-image-registry-hcp-cluster" in resourcegroup="disabled-image-registry-plxw8q" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster disabled-image-registry-hcp-cluster in resource group disabled-image-registry-plxw8q, error: context deadline exceeded ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/image_registry_cluster_create.go:89]: Unexpected error: <fmt.wrapErrors \| 0xc001709110>: failed to create HCP cluster disabled-image-registry-hcp-cluster, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster disabled-image-registry-hcp-cluster in resource group disabled-image-registry-plxw8q, error: failed waiting for cluster="disabled-image-registry-hcp-cluster" in resourcegroup="disabled-image-registry-plxw8q" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster disabled-image-registry-hcp-cluster in resource group disabled-image-registry-plxw8q, error: context deadline exceeded ... occurred timeout during CreateHCPClusterAndWait; context deadline exceeded 03:36:54 UTC · failed at e2e · test Nodepool Ephemeral OS Disk should create a nodepool with ephemeral OS disk when autoRepair is enabled · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_ephemeral_osdisk.go:112]: Unexpected error: <fmt.wrapErrors \| 0xc00092d200>: failed to create HCP cluster ephemeral-disk, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster ephemeral-disk in resource group ephemeral-osdisk-5fppd9, error: failed waiting for cluster="ephemeral-disk" in resourcegroup="ephemeral-osdisk-5fppd9" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster ephemeral-disk in resource group ephemeral-osdisk-5fppd9, error: context deadline exceeded ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_ephemeral_osdisk.go:112]: Unexpected error: <fmt.wrapErrors \| 0xc00092d200>: failed to create HCP cluster ephemeral-disk, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster ephemeral-disk in resource group ephemeral-osdisk-5fppd9, error: failed waiting for cluster="ephemeral-disk" in resourcegroup="ephemeral-osdisk-5fppd9" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster ephemeral-disk in resource group ephemeral-osdisk-5fppd9, error: context deadline exceeded ... occurred timeout during CreateHCPClusterAndWait; context deadline exceeded 03:36:54 UTC · failed at e2e · test Customer should be able to use workload identity via the cluster OIDC issuer URL · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/oidc_issuer_workload_identity.go:216]: Unexpected error: <fmt.wrapErrors \| 0xc000953290>: failed to create HCP cluster oidc-wi-cluster, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster oidc-wi-cluster in resource group oidc-wi-rgk47s, error: failed waiting for cluster="oidc-wi-cluster" in resourcegroup="oidc-wi-rgk47s" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster oidc-wi-cluster in resource group oidc-wi-rgk47s, error: context deadline exceeded ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/oidc_issuer_workload_identity.go:216]: Unexpected error: <fmt.wrapErrors \| 0xc000953290>: failed to create HCP cluster oidc-wi-cluster, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster oidc-wi-cluster in resource group oidc-wi-rgk47s, error: failed waiting for cluster="oidc-wi-cluster" in resourcegroup="oidc-wi-rgk47s" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster oidc-wi-cluster in resource group oidc-wi-rgk47s, error: context deadline exceeded ... occurred timeout during CreateHCPClusterAndWait; context deadline exceeded 03:36:54 UTC · failed at e2e · test Update HCPOpenShiftCluster Negative creates a cluster and fails to update its name with a PATCH request · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:84]: Unexpected error: <fmt.wrapErrors \| 0xc00111ea80>: failed to create HCP cluster patch-name-cluster, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster patch-name-cluster in resource group patch-name-n6pvrg, error: failed waiting for cluster="patch-name-cluster" in resourcegroup="patch-name-n6pvrg" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster patch-name-cluster in resource group patch-name-n6pvrg, error: context deadline exceeded ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:84]: Unexpected error: <fmt.wrapErrors \| 0xc00111ea80>: failed to create HCP cluster patch-name-cluster, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster patch-name-cluster in resource group patch-name-n6pvrg, error: failed waiting for cluster="patch-name-cluster" in resourcegroup="patch-name-n6pvrg" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster patch-name-cluster in resource group patch-name-n6pvrg, error: context deadline exceeded ... occurred timeout during CreateHCPClusterAndWait; context deadline exceeded 03:36:54 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 4.19 · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:141]: Unexpected error: <fmt.wrapErrors \| 0xc001788a80>: failed to create HCP cluster cluster-zstream-4-19-6n59wk, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster cluster-zstream-4-19-6n59wk in resource group rg-zstream-upgrade-4-19-4p2wmj, error: failed waiting for cluster="cluster-zstream-4-19-6n59wk" in resourcegroup="rg-zstream-upgrade-4-19-4p2wmj" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster cluster-zstream-4-19-6n59wk in resource group rg-zstream-upgrade-4-19-4p2wmj, error: context deadline exceeded ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:141]: Unexpected error: <fmt.wrapErrors \| 0xc001788a80>: failed to create HCP cluster cluster-zstream-4-19-6n59wk, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster cluster-zstream-4-19-6n59wk in resource group rg-zstream-upgrade-4-19-4p2wmj, error: failed waiting for cluster="cluster-zstream-4-19-6n59wk" in resourcegroup="rg-zstream-upgrade-4-19-4p2wmj" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster cluster-zstream-4-19-6n59wk in resource group rg-zstream-upgrade-4-19-4p2wmj, error: context deadline exceeded ... occurred timeout during CreateHCPClusterAndWait; context deadline exceeded 03:36:54 UTC · failed at e2e · test Customer should be able to forward control plane logs to a storage account via shoebox diagnostic settings · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/shoebox_logs.go:103]: Unexpected error: <fmt.wrapErrors \| 0xc0009a0ba0>: failed to create HCP cluster shoebox-hcp-cluster, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster shoebox-hcp-cluster in resource group shoebox-cluster-nv6nv7, error: failed waiting for cluster="shoebox-hcp-cluster" in resourcegroup="shoebox-cluster-nv6nv7" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster shoebox-hcp-cluster in resource group shoebox-cluster-nv6nv7, error: context deadline exceeded ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/shoebox_logs.go:103]: Unexpected error: <fmt.wrapErrors \| 0xc0009a0ba0>: failed to create HCP cluster shoebox-hcp-cluster, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster shoebox-hcp-cluster in resource group shoebox-cluster-nv6nv7, error: failed waiting for cluster="shoebox-hcp-cluster" in resourcegroup="shoebox-cluster-nv6nv7" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster shoebox-hcp-cluster in resource group shoebox-cluster-nv6nv7, error: context deadline exceeded ... occurred timeout during CreateHCPClusterAndWait; context deadline exceeded 03:36:54 UTC · failed at e2e · test Customer should be able to create node pools with ARM64-based VMs · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/arm64_nodepool.go:90]: Unexpected error: <fmt.wrapErrors \| 0xc0014fcde0>: failed to create HCP cluster arm64-vm-hcp-cluster, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster arm64-vm-hcp-cluster in resource group arm64-vm-cluster-bc85j4, error: failed waiting for cluster="arm64-vm-hcp-cluster" in resourcegroup="arm64-vm-cluster-bc85j4" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster arm64-vm-hcp-cluster in resource group arm64-vm-cluster-bc85j4, error: context deadline exceeded ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/arm64_nodepool.go:90]: Unexpected error: <fmt.wrapErrors \| 0xc0014fcde0>: failed to create HCP cluster arm64-vm-hcp-cluster, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster arm64-vm-hcp-cluster in resource group arm64-vm-cluster-bc85j4, error: failed waiting for cluster="arm64-vm-hcp-cluster" in resourcegroup="arm64-vm-cluster-bc85j4" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster arm64-vm-hcp-cluster in resource group arm64-vm-cluster-bc85j4, error: context deadline exceeded ... occurred timeout during CreateHCPClusterAndWait; context deadline exceeded 03:36:54 UTC · failed at e2e · test Customer should not be able to perform various invalid operations on cluster resources · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/simple_negative_cases.go:117]: Unexpected error: <fmt.wrapErrors \| 0xc0010b65d0>: failed to create HCP cluster negative-tests-cluster, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster negative-tests-cluster in resource group rg-negative-tests-wr7qgw, error: failed waiting for cluster="negative-tests-cluster" in resourcegroup="rg-negative-tests-wr7qgw" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster negative-tests-cluster in resource group rg-negative-tests-wr7qgw, error: context deadline exceeded ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/simple_negative_cases.go:117]: Unexpected error: <fmt.wrapErrors \| 0xc0010b65d0>: failed to create HCP cluster negative-tests-cluster, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster negative-tests-cluster in resource group rg-negative-tests-wr7qgw, error: failed waiting for cluster="negative-tests-cluster" in resourcegroup="rg-negative-tests-wr7qgw" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster negative-tests-cluster in resource group rg-negative-tests-wr7qgw, error: context deadline exceeded ... occurred timeout during CreateHCPClusterAndWait; context deadline exceeded 03:36:54 UTC · failed at e2e · test Customer should be able to lifecycle and confirm external auth on a cluster · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_list_and_verify.go:84]: Unexpected error: <fmt.wrapErrors \| 0xc0017a2ea0>: failed to create HCP cluster ea-listpcn8ws, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster ea-listpcn8ws in resource group ea-list-gtmnmh, error: failed waiting for cluster="ea-listpcn8ws" in resourcegroup="ea-list-gtmnmh" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster ea-listpcn8ws in resource group ea-list-gtmnmh, error: context deadline exceeded ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_list_and_verify.go:84]: Unexpected error: <fmt.wrapErrors \| 0xc0017a2ea0>: failed to create HCP cluster ea-listpcn8ws, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster ea-listpcn8ws in resource group ea-list-gtmnmh, error: failed waiting for cluster="ea-listpcn8ws" in resourcegroup="ea-list-gtmnmh" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster ea-listpcn8ws in resource group ea-list-gtmnmh, error: context deadline exceeded ... occurred timeout during CreateHCPClusterAndWait; context deadline exceeded 03:36:54 UTC · failed at e2e · test Customer should be able to update nodepool replicas and autoscaling · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_update_nodes.go:99]: Unexpected error: <fmt.wrapErrors \| 0xc0011f4ae0>: failed to create HCP cluster hcp-cluster-update-nodes, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster hcp-cluster-update-nodes in resource group rg-update-nodes-pfkhlf, error: failed waiting for cluster="hcp-cluster-update-nodes" in resourcegroup="rg-update-nodes-pfkhlf" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster hcp-cluster-update-nodes in resource group rg-update-nodes-pfkhlf, error: context deadline exceeded ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_update_nodes.go:99]: Unexpected error: <fmt.wrapErrors \| 0xc0011f4ae0>: failed to create HCP cluster hcp-cluster-update-nodes, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster hcp-cluster-update-nodes in resource group rg-update-nodes-pfkhlf, error: failed waiting for cluster="hcp-cluster-update-nodes" in resourcegroup="rg-update-nodes-pfkhlf" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster hcp-cluster-update-nodes in resource group rg-update-nodes-pfkhlf, error: context deadline exceeded ... occurred
02:00:39	periodic-ci-Azure-ARO-HCP-main-periodic-prod-e2e-parallel post-goodmerged PR No additional run metadata captured	n/a	passed	n/a	0	n/a