{"meta":{"date":"2026-06-19","timezone":"UTC","generated_at":"2026-08-29T14:57:36Z","environments":["stg"]},"environments":[{"environment":"stg","summary":{"total_runs":6,"failed_runs":5,"runs_with_occurrences":5,"runs_with_failure_pattern_matches":5,"runs_with_unmatched_occurrences":0,"runs_without_occurrence_rows":1,"failed_runs_without_occurrence_rows":0},"runs":[{"run":{"environment":"stg","run_url":"https://prow.ci.openshift.org/view/gs/test-platform-results/logs/periodic-ci-Azure-ARO-HCP-main-periodic-stage-e2e-parallel-ocp-nightly/2068106661565829120","job_name":"periodic-ci-Azure-ARO-HCP-main-periodic-stage-e2e-parallel-ocp-nightly","pr_number":0,"pr_state":"","pr_sha":"","final_merged_sha":"","merged_pr":true,"post_good_commit":true,"failed":true,"occurred_at":"2026-06-19T23:00:26Z"},"failed_at":["e2e"],"failed_test_count":2,"occurrences":[{"row_id":"d7d0c721763442a6deef598f5a267e70a7d04ab7fbd4976d17f56bf04c9d7f66","run_url":"https://prow.ci.openshift.org/view/gs/test-platform-results/logs/periodic-ci-Azure-ARO-HCP-main-periodic-stage-e2e-parallel-ocp-nightly/2068106661565829120","occurred_at":"2026-06-19T23:00:26Z","failed_at":"e2e","signature_id":"c14c8503e5d55013bbe09b887861f1b01dcba3f92ad3f1e969cd5670e5686999","test_name":"Customer should create an HCP cluster and validate TLS certificates","test_suite":"stage/parallel","failure_text":"fail [github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:285] A node timeout occurred and then the following failure was recorded in the timedout node before it exited:\nUnexpected error:\n    \u003c*errors.joinError | 0xc00017acd8\u003e: \n    failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster=\"tls-endpoint-hcp-cluster\" in resourcegroup=\"tls-endpoint-cluster-846tff\" to finish deleting: context canceled\n    ...\noccurred  \nfail [:0]: A node timeout occurred\nfail [github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:285] A node timeout occurred and then the following failure was recorded in the timedout node before it exited:\nUnexpected error:\n    \u003c*errors.joinError | 0xc00017acd8\u003e: \n    failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster=\"tls-endpoint-hcp-cluster\" in resourcegroup=\"tls-endpoint-cluster-846tff\" to finish deleting: context canceled\n    ...\noccurred  \nfail [:0]: A node timeout occurred","failure_pattern_match":{"status":"clustered","failure_pattern_id":"7f0e44476112ce4e675f687843407c03482d80d265239de5b2ddbd45c441a04e","failure_pattern":"failed to cleanup resource group: at least one HCP cluster \u003ccluster\u003e to delete: failed waiting for hcpcluster=\"\u003ccluster\u003e\" in resourcegroup=\"\u003cresource-group\u003e\" to finish deleting: context canceled","search_query":"failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster=\"arm64-vm-hcp-cluster\" in resourcegroup=\"arm64-vm-cluster-6hj75l\" to finish deleting: context canceled"}},{"row_id":"e68ea458bf6a63ce2ced10eea26a6ecf1ba6e89c9eea79f4b817874779a48bbd","run_url":"https://prow.ci.openshift.org/view/gs/test-platform-results/logs/periodic-ci-Azure-ARO-HCP-main-periodic-stage-e2e-parallel-ocp-nightly/2068106661565829120","occurred_at":"2026-06-19T23:00:26Z","failed_at":"e2e","signature_id":"d419ae1c2918cca767ef59cb3bf977dd2fe1f991cc8b16534ec0d3ff760b5600","test_name":"Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 4.22","test_suite":"stage/parallel","failure_text":"fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:102]: failed to create HCP cluster \"cluster-zstream-4-22-s2sv5c\" with version 4.22.0 on candidate channel\nUnexpected error:\n    \u003c*fmt.wrapErrors | 0xc001520390\u003e: \n    failed to create HCP cluster cluster-zstream-4-22-s2sv5c, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster cluster-zstream-4-22-s2sv5c in resource group rg-zstream-upgrade-4-22-9pz7hs, error: failed waiting for cluster=\"cluster-zstream-4-22-s2sv5c\" in resourcegroup=\"rg-zstream-upgrade-4-22-9pz7hs\" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster cluster-zstream-4-22-s2sv5c in resource group rg-zstream-upgrade-4-22-9pz7hs, error: context deadline exceeded\n    ...\noccurred\nfail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:102]: failed to create HCP cluster \"cluster-zstream-4-22-s2sv5c\" with version 4.22.0 on candidate channel\nUnexpected error:\n    \u003c*fmt.wrapErrors | 0xc001520390\u003e: \n    failed to create HCP cluster cluster-zstream-4-22-s2sv5c, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster cluster-zstream-4-22-s2sv5c in resource group rg-zstream-upgrade-4-22-9pz7hs, error: failed waiting for cluster=\"cluster-zstream-4-22-s2sv5c\" in resourcegroup=\"rg-zstream-upgrade-4-22-9pz7hs\" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster cluster-zstream-4-22-s2sv5c in resource group rg-zstream-upgrade-4-22-9pz7hs, error: context deadline exceeded\n    ...\noccurred","failure_pattern_match":{"status":"clustered","failure_pattern_id":"6ed54686dd2f6d32054cbf144906e399a60d9c8b1a6d87d91850a99759063298","failure_pattern":"timeout during CreateHCPClusterAndWait; context deadline exceeded","search_query":"failed to create HCP cluster idms-e2e-hcp-cluster, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster idms-e2e-hcp-cluster in resource group idms-c2ddz2, error: failed waiting for cluster=\"idms-e2e-hcp-cluster\" in resourcegroup=\"idms-c2ddz2\" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster idms-e2e-hcp-cluster in resource group idms-c2ddz2, error: context deadline exceeded"}}],"failure_pattern_summary":{"signature_count":2,"failure_pattern_ids":["6ed54686dd2f6d32054cbf144906e399a60d9c8b1a6d87d91850a99759063298","7f0e44476112ce4e675f687843407c03482d80d265239de5b2ddbd45c441a04e"],"matched_occurrences":2,"unmatched_occurrences":0,"match_summary":"multiple_clustered"}},{"run":{"environment":"stg","run_url":"https://prow.ci.openshift.org/view/gs/test-platform-results/logs/branch-ci-Azure-ARO-HCP-main-e2e-stage-e2e-parallel/2068055531859218432","job_name":"branch-ci-Azure-ARO-HCP-main-e2e-stage-e2e-parallel","pr_number":0,"pr_state":"","pr_sha":"","final_merged_sha":"","merged_pr":true,"post_good_commit":true,"failed":true,"occurred_at":"2026-06-19T19:37:16Z"},"failed_at":["e2e"],"failed_test_count":8,"occurrences":[{"row_id":"12e4d500f2b550ea3446cc5a78db80d526556856406ef628d8ef097ccac280af","run_url":"https://prow.ci.openshift.org/view/gs/test-platform-results/logs/branch-ci-Azure-ARO-HCP-main-e2e-stage-e2e-parallel/2068055531859218432","occurred_at":"2026-06-19T19:37:16Z","failed_at":"e2e","signature_id":"c8de6b1c03953f819d3340b8f1759cfb3830829273aa563001b5c0d29df7bdc5","test_name":"Customer should upgrade a nodepool skipping one minor version (+2) from 4.20.z to 4.22.zLatest","test_suite":"stage/parallel","failure_text":"fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:466]: failed to create HCP cluster np-skip-minor-nsh2xn at version 4.22.2\nUnexpected error:\n    \u003c*fmt.wrapErrors | 0xc0015dc090\u003e: \n    failed to create HCP cluster np-skip-minor-nsh2xn, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster np-skip-minor-nsh2xn in resource group rg-np-skip-minor-nsh2xn-bkjhkx, error: failed waiting for cluster=\"np-skip-minor-nsh2xn\" in resourcegroup=\"rg-np-skip-minor-nsh2xn-bkjhkx\" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster np-skip-minor-nsh2xn in resource group rg-np-skip-minor-nsh2xn-bkjhkx, error: context deadline exceeded\n    ...\noccurred\nfail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:466]: failed to create HCP cluster np-skip-minor-nsh2xn at version 4.22.2\nUnexpected error:\n    \u003c*fmt.wrapErrors | 0xc0015dc090\u003e: \n    failed to create HCP cluster np-skip-minor-nsh2xn, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster np-skip-minor-nsh2xn in resource group rg-np-skip-minor-nsh2xn-bkjhkx, error: failed waiting for cluster=\"np-skip-minor-nsh2xn\" in resourcegroup=\"rg-np-skip-minor-nsh2xn-bkjhkx\" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster np-skip-minor-nsh2xn in resource group rg-np-skip-minor-nsh2xn-bkjhkx, error: context deadline exceeded\n    ...\noccurred","failure_pattern_match":{"status":"clustered","failure_pattern_id":"6ed54686dd2f6d32054cbf144906e399a60d9c8b1a6d87d91850a99759063298","failure_pattern":"timeout during CreateHCPClusterAndWait; context deadline exceeded","search_query":"failed to create HCP cluster idms-e2e-hcp-cluster, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster idms-e2e-hcp-cluster in resource group idms-c2ddz2, error: failed waiting for cluster=\"idms-e2e-hcp-cluster\" in resourcegroup=\"idms-c2ddz2\" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster idms-e2e-hcp-cluster in resource group idms-c2ddz2, error: context deadline exceeded"}},{"row_id":"12e776d32a720c802eba97604545894938ace8a72ef8d8722817aa83198246b8","run_url":"https://prow.ci.openshift.org/view/gs/test-platform-results/logs/branch-ci-Azure-ARO-HCP-main-e2e-stage-e2e-parallel/2068055531859218432","occurred_at":"2026-06-19T19:37:16Z","failed_at":"e2e","signature_id":"5e1a783f06ceb8848828692cfaee71d285ad3492cee9f8675ce17e7f1a2c7fc5","test_name":"MISE Routing routes to the correct frontend based on version header default route returns no version header","test_suite":"stage/parallel","failure_text":"fail [github.com/Azure/ARO-HCP/test/e2e/mise_routing.go:132]: failed to create HCP cluster\nUnexpected error:\n    \u003c*fmt.wrapErrors | 0xc0014f2480\u003e: \n    failed waiting for cluster=\"mise-routing\" in resourcegroup=\"mise-default-smoke-m5x8rs\" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223AndWait for cluster mise-routing in resource group mise-default-smoke-m5x8rs, error: context deadline exceeded\n    ...\noccurred\nfail [github.com/Azure/ARO-HCP/test/e2e/mise_routing.go:132]: failed to create HCP cluster\nUnexpected error:\n    \u003c*fmt.wrapErrors | 0xc0014f2480\u003e: \n    failed waiting for cluster=\"mise-routing\" in resourcegroup=\"mise-default-smoke-m5x8rs\" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223AndWait for cluster mise-routing in resource group mise-default-smoke-m5x8rs, error: context deadline exceeded\n    ...\noccurred","failure_pattern_match":{"status":"clustered","failure_pattern_id":"6ed54686dd2f6d32054cbf144906e399a60d9c8b1a6d87d91850a99759063298","failure_pattern":"timeout during CreateHCPClusterAndWait; context deadline exceeded","search_query":"failed to create HCP cluster idms-e2e-hcp-cluster, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster idms-e2e-hcp-cluster in resource group idms-c2ddz2, error: failed waiting for cluster=\"idms-e2e-hcp-cluster\" in resourcegroup=\"idms-c2ddz2\" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster idms-e2e-hcp-cluster in resource group idms-c2ddz2, error: context deadline exceeded"}},{"row_id":"20ab35cf9290dc9f673a2717319e0b3bc3b5e11a551bf3e0230678afacfa5fe3","run_url":"https://prow.ci.openshift.org/view/gs/test-platform-results/logs/branch-ci-Azure-ARO-HCP-main-e2e-stage-e2e-parallel/2068055531859218432","occurred_at":"2026-06-19T19:37:16Z","failed_at":"e2e","signature_id":"5bf9b94184623367c7e7fb5bd3d812a02e381fb7f91799e4381867600e845b30","test_name":"Customer should downgrade a nodepool version z-stream downgrade from 4.21.zLatest to 4.21.zPrevious","test_suite":"stage/parallel","failure_text":"fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:596]: failed to create HCP cluster\nUnexpected error:\n    \u003c*fmt.wrapErrors | 0xc0012d4510\u003e: \n    failed to create HCP cluster np-downgrade-cpvw84, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster np-downgrade-cpvw84 in resource group rg-np-downgrade-cpvw84-sv54gs, error: failed waiting for cluster=\"np-downgrade-cpvw84\" in resourcegroup=\"rg-np-downgrade-cpvw84-sv54gs\" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster np-downgrade-cpvw84 in resource group rg-np-downgrade-cpvw84-sv54gs, error: context deadline exceeded\n    ...\noccurred\nfail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:596]: failed to create HCP cluster\nUnexpected error:\n    \u003c*fmt.wrapErrors | 0xc0012d4510\u003e: \n    failed to create HCP cluster np-downgrade-cpvw84, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster np-downgrade-cpvw84 in resource group rg-np-downgrade-cpvw84-sv54gs, error: failed waiting for cluster=\"np-downgrade-cpvw84\" in resourcegroup=\"rg-np-downgrade-cpvw84-sv54gs\" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster np-downgrade-cpvw84 in resource group rg-np-downgrade-cpvw84-sv54gs, error: context deadline exceeded\n    ...\noccurred","failure_pattern_match":{"status":"clustered","failure_pattern_id":"6ed54686dd2f6d32054cbf144906e399a60d9c8b1a6d87d91850a99759063298","failure_pattern":"timeout during CreateHCPClusterAndWait; context deadline exceeded","search_query":"failed to create HCP cluster idms-e2e-hcp-cluster, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster idms-e2e-hcp-cluster in resource group idms-c2ddz2, error: failed waiting for cluster=\"idms-e2e-hcp-cluster\" in resourcegroup=\"idms-c2ddz2\" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster idms-e2e-hcp-cluster in resource group idms-c2ddz2, error: context deadline exceeded"}},{"row_id":"81d988f938b3aaed35c82cee4352a44c3008c377fdf4a65eecc60925bc7442c7","run_url":"https://prow.ci.openshift.org/view/gs/test-platform-results/logs/branch-ci-Azure-ARO-HCP-main-e2e-stage-e2e-parallel/2068055531859218432","occurred_at":"2026-06-19T19:37:16Z","failed_at":"e2e","signature_id":"e6533d5ba7028317553795691d709a4e0fafb0b67862f2cce43e5bed15cd9612","test_name":"Customer should upgrade a nodepool to a version without Cincinnati upgrade edge z-stream upgrade without Cincinnati edge in 4.20","test_suite":"stage/parallel","failure_text":"fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:312]: failed to create HCP cluster np-noedge-c9txpb\nUnexpected error:\n    \u003c*fmt.wrapErrors | 0xc0007df0e0\u003e: \n    failed to create HCP cluster np-noedge-c9txpb, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster np-noedge-c9txpb in resource group rg-np-noedge-c9txpb-lmgnlk, error: failed waiting for cluster=\"np-noedge-c9txpb\" in resourcegroup=\"rg-np-noedge-c9txpb-lmgnlk\" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster np-noedge-c9txpb in resource group rg-np-noedge-c9txpb-lmgnlk, error: context deadline exceeded\n    ...\noccurred\nfail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:312]: failed to create HCP cluster np-noedge-c9txpb\nUnexpected error:\n    \u003c*fmt.wrapErrors | 0xc0007df0e0\u003e: \n    failed to create HCP cluster np-noedge-c9txpb, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster np-noedge-c9txpb in resource group rg-np-noedge-c9txpb-lmgnlk, error: failed waiting for cluster=\"np-noedge-c9txpb\" in resourcegroup=\"rg-np-noedge-c9txpb-lmgnlk\" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster np-noedge-c9txpb in resource group rg-np-noedge-c9txpb-lmgnlk, error: context deadline exceeded\n    ...\noccurred","failure_pattern_match":{"status":"clustered","failure_pattern_id":"6ed54686dd2f6d32054cbf144906e399a60d9c8b1a6d87d91850a99759063298","failure_pattern":"timeout during CreateHCPClusterAndWait; context deadline exceeded","search_query":"failed to create HCP cluster idms-e2e-hcp-cluster, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster idms-e2e-hcp-cluster in resource group idms-c2ddz2, error: failed waiting for cluster=\"idms-e2e-hcp-cluster\" in resourcegroup=\"idms-c2ddz2\" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster idms-e2e-hcp-cluster in resource group idms-c2ddz2, error: context deadline exceeded"}},{"row_id":"9446aca5ff81d2d8425bf4d5f3081f068f27b7bac7d591d8e7842c16c3eb94c4","run_url":"https://prow.ci.openshift.org/view/gs/test-platform-results/logs/branch-ci-Azure-ARO-HCP-main-e2e-stage-e2e-parallel/2068055531859218432","occurred_at":"2026-06-19T19:37:16Z","failed_at":"e2e","signature_id":"d9d853067effda96b006962f9dceb6b3a88511af192b07cd787e4d749191f0b0","test_name":"ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.22","test_suite":"stage/parallel","failure_text":"fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:159]: failed to create node pool \"np-r7rvfr\" for cluster \"cluster-candidate-4-22-r7rvfr\"\nUnexpected error:\n    \u003c*fmt.wrapErrors | 0xc0007a2c30\u003e: \n    failed to create NodePool np-r7rvfr, caused by: timeout '20.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-r7rvfr in resource group rg-candidate-4-22-gtwnxg, error: failed waiting for nodepool=\"np-r7rvfr\" for cluster \"cluster-candidate-4-22-r7rvfr\" in resourcegroup=\"rg-candidate-4-22-gtwnxg\" to finish creating: context deadline exceeded\n    ...\noccurred\nfail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:159]: failed to create node pool \"np-r7rvfr\" for cluster \"cluster-candidate-4-22-r7rvfr\"\nUnexpected error:\n    \u003c*fmt.wrapErrors | 0xc0007a2c30\u003e: \n    failed to create NodePool np-r7rvfr, caused by: timeout '20.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-r7rvfr in resource group rg-candidate-4-22-gtwnxg, error: failed waiting for nodepool=\"np-r7rvfr\" for cluster \"cluster-candidate-4-22-r7rvfr\" in resourcegroup=\"rg-candidate-4-22-gtwnxg\" to finish creating: context deadline exceeded\n    ...\noccurred","failure_pattern_match":{"status":"clustered","failure_pattern_id":"941cbd5c6e3f3d8ed572893a689b2dfded75f418444a1a0d01baa9fc9e30d9c0","failure_pattern":"CreateNodePoolFromParam timed out after \u003cminutes\u003e minutes; context deadline exceeded","search_query":"timeout '20.000000' minutes exceeded during CreateNodePoolFromParam for node pool cilium-np in resource group complex-cilium-kv-kdtw5w, error: failed waiting for nodepool=\"cilium-np\" for cluster \"cilium-cluster\" in resourcegroup=\"complex-cilium-kv-kdtw5w\" to finish creating: context deadline exceeded"}},{"row_id":"c416f7270eb14ac105fa2ffcedd9fee4e0f890e790c6730fdbb9d1d9080996ff","run_url":"https://prow.ci.openshift.org/view/gs/test-platform-results/logs/branch-ci-Azure-ARO-HCP-main-e2e-stage-e2e-parallel/2068055531859218432","occurred_at":"2026-06-19T19:37:16Z","failed_at":"e2e","signature_id":"6d5edd8ab6e7cfc527f8e300f7c735f488e5bccf6be8e97633ed998e0cc51aac","test_name":"Customer should be able to forward control plane logs to a storage account and Event Hub via shoebox diagnostic settings","test_suite":"stage/parallel","failure_text":"fail [github.com/Azure/ARO-HCP/test/e2e/shoebox_logs.go:269]: failed to create HCP cluster shoebox-hcp-cluster\nUnexpected error:\n    \u003c*fmt.wrapErrors | 0xc0007ce0f0\u003e: \n    failed to create HCP cluster shoebox-hcp-cluster, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster shoebox-hcp-cluster in resource group shoebox-cluster-tjm5kq, error: failed waiting for cluster=\"shoebox-hcp-cluster\" in resourcegroup=\"shoebox-cluster-tjm5kq\" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster shoebox-hcp-cluster in resource group shoebox-cluster-tjm5kq, error: context deadline exceeded\n    ...\noccurred\nfail [github.com/Azure/ARO-HCP/test/e2e/shoebox_logs.go:269]: failed to create HCP cluster shoebox-hcp-cluster\nUnexpected error:\n    \u003c*fmt.wrapErrors | 0xc0007ce0f0\u003e: \n    failed to create HCP cluster shoebox-hcp-cluster, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster shoebox-hcp-cluster in resource group shoebox-cluster-tjm5kq, error: failed waiting for cluster=\"shoebox-hcp-cluster\" in resourcegroup=\"shoebox-cluster-tjm5kq\" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster shoebox-hcp-cluster in resource group shoebox-cluster-tjm5kq, error: context deadline exceeded\n    ...\noccurred","failure_pattern_match":{"status":"clustered","failure_pattern_id":"6ed54686dd2f6d32054cbf144906e399a60d9c8b1a6d87d91850a99759063298","failure_pattern":"timeout during CreateHCPClusterAndWait; context deadline exceeded","search_query":"failed to create HCP cluster idms-e2e-hcp-cluster, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster idms-e2e-hcp-cluster in resource group idms-c2ddz2, error: failed waiting for cluster=\"idms-e2e-hcp-cluster\" in resourcegroup=\"idms-c2ddz2\" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster idms-e2e-hcp-cluster in resource group idms-c2ddz2, error: context deadline exceeded"}},{"row_id":"d5b12f066901e03a56883eda7af00cb2f4497aac3c470925fe9dbea3e5d1055d","run_url":"https://prow.ci.openshift.org/view/gs/test-platform-results/logs/branch-ci-Azure-ARO-HCP-main-e2e-stage-e2e-parallel/2068055531859218432","occurred_at":"2026-06-19T19:37:16Z","failed_at":"e2e","signature_id":"5899b7c91520cac7fc9528842b6c591771e7325d5c6a60b8ce86423a54cbd8fe","test_name":"Customer should be able to create a no-CNI private cluster with a private key vault, a nodepool and install cilium CNI successfully","test_suite":"stage/parallel","failure_text":"fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:190]: failed to verify nodes are Ready with Cilium CNI for cluster \"cilium-cluster\"\nUnexpected error:\n    \u003c*errors.joinError | 0xc00210a510\u003e: \n    VerifyNodesReady failed: no nodes found in the cluster\n    VerifyCiliumOperational failed: not all pods in kube-system namespace are running: no cilium pods found in kube-system namespace\n    failed to create NodePool cilium-np, caused by: timeout '20.000000' minutes exceeded during CreateNodePoolFromParam for node pool cilium-np in resource group complex-cilium-kv-kdtw5w, error: failed waiting for nodepool=\"cilium-np\" for cluster \"cilium-cluster\" in resourcegroup=\"complex-cilium-kv-kdtw5w\" to finish creating: context deadline exceeded\n    ...\noccurred\nfail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:190]: failed to verify nodes are Ready with Cilium CNI for cluster \"cilium-cluster\"\nUnexpected error:\n    \u003c*errors.joinError | 0xc00210a510\u003e: \n    VerifyNodesReady failed: no nodes found in the cluster\n    VerifyCiliumOperational failed: not all pods in kube-system namespace are running: no cilium pods found in kube-system namespace\n    failed to create NodePool cilium-np, caused by: timeout '20.000000' minutes exceeded during CreateNodePoolFromParam for node pool cilium-np in resource group complex-cilium-kv-kdtw5w, error: failed waiting for nodepool=\"cilium-np\" for cluster \"cilium-cluster\" in resourcegroup=\"complex-cilium-kv-kdtw5w\" to finish creating: context deadline exceeded\n    ...\noccurred","failure_pattern_match":{"status":"clustered","failure_pattern_id":"941cbd5c6e3f3d8ed572893a689b2dfded75f418444a1a0d01baa9fc9e30d9c0","failure_pattern":"CreateNodePoolFromParam timed out after \u003cminutes\u003e minutes; context deadline exceeded","search_query":"timeout '20.000000' minutes exceeded during CreateNodePoolFromParam for node pool cilium-np in resource group complex-cilium-kv-kdtw5w, error: failed waiting for nodepool=\"cilium-np\" for cluster \"cilium-cluster\" in resourcegroup=\"complex-cilium-kv-kdtw5w\" to finish creating: context deadline exceeded"}},{"row_id":"f402e4c2158c98958d68f1a8c5ecdd410a604c6b03a2cb3302742d5418d819c6","run_url":"https://prow.ci.openshift.org/view/gs/test-platform-results/logs/branch-ci-Azure-ARO-HCP-main-e2e-stage-e2e-parallel/2068055531859218432","occurred_at":"2026-06-19T19:37:16Z","failed_at":"e2e","signature_id":"3a0a66ac994c97e119500446e9e57f0b9747f9585025e8af52a7de99bd4ae068","test_name":"Customer should respect cluster-wide node limits with nodepool autoscaling","test_suite":"stage/parallel","failure_text":"fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:282]: failed to create HCP cluster node-limit-cluster with MaxNodesTotal limit\nUnexpected error:\n    \u003c*fmt.wrapErrors | 0xc000f60630\u003e: \n    failed to create HCP cluster node-limit-cluster, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster node-limit-cluster in resource group node-limit-test-kj8xs5, error: failed waiting for cluster=\"node-limit-cluster\" in resourcegroup=\"node-limit-test-kj8xs5\" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster node-limit-cluster in resource group node-limit-test-kj8xs5, error: context deadline exceeded\n    ...\noccurred\nfail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:282]: failed to create HCP cluster node-limit-cluster with MaxNodesTotal limit\nUnexpected error:\n    \u003c*fmt.wrapErrors | 0xc000f60630\u003e: \n    failed to create HCP cluster node-limit-cluster, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster node-limit-cluster in resource group node-limit-test-kj8xs5, error: failed waiting for cluster=\"node-limit-cluster\" in resourcegroup=\"node-limit-test-kj8xs5\" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster node-limit-cluster in resource group node-limit-test-kj8xs5, error: context deadline exceeded\n    ...\noccurred","failure_pattern_match":{"status":"clustered","failure_pattern_id":"6ed54686dd2f6d32054cbf144906e399a60d9c8b1a6d87d91850a99759063298","failure_pattern":"timeout during CreateHCPClusterAndWait; context deadline exceeded","search_query":"failed to create HCP cluster idms-e2e-hcp-cluster, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster idms-e2e-hcp-cluster in resource group idms-c2ddz2, error: failed waiting for cluster=\"idms-e2e-hcp-cluster\" in resourcegroup=\"idms-c2ddz2\" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster idms-e2e-hcp-cluster in resource group idms-c2ddz2, error: context deadline exceeded"}}],"failure_pattern_summary":{"signature_count":8,"failure_pattern_ids":["6ed54686dd2f6d32054cbf144906e399a60d9c8b1a6d87d91850a99759063298","941cbd5c6e3f3d8ed572893a689b2dfded75f418444a1a0d01baa9fc9e30d9c0"],"matched_occurrences":8,"unmatched_occurrences":0,"match_summary":"multiple_clustered"}},{"run":{"environment":"stg","run_url":"https://prow.ci.openshift.org/view/gs/test-platform-results/logs/branch-ci-Azure-ARO-HCP-main-e2e-stage-e2e-parallel/2068019178161311744","job_name":"branch-ci-Azure-ARO-HCP-main-e2e-stage-e2e-parallel","pr_number":0,"pr_state":"","pr_sha":"","final_merged_sha":"","merged_pr":true,"post_good_commit":true,"failed":true,"occurred_at":"2026-06-19T17:12:48Z"},"failed_at":["e2e"],"failed_test_count":2,"occurrences":[{"row_id":"2b3aeea78851a18eeb2b0b162d2db21866cb42ef0bee18a91af952d11d275111","run_url":"https://prow.ci.openshift.org/view/gs/test-platform-results/logs/branch-ci-Azure-ARO-HCP-main-e2e-stage-e2e-parallel/2068019178161311744","occurred_at":"2026-06-19T17:12:48Z","failed_at":"e2e","signature_id":"79c6ce5a68894dbe77658c9178971929cc55c11f98625fa016e5bca99323e6c8","test_name":"Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 4.22","test_suite":"stage/parallel","failure_text":"fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:102]: failed to create HCP cluster \"cluster-zstream-4-22-pwd5hd\" with version 4.22.0 on candidate channel\nUnexpected error:\n    \u003c*fmt.wrapErrors | 0xc0008f8720\u003e: \n    failed to create HCP cluster cluster-zstream-4-22-pwd5hd, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster cluster-zstream-4-22-pwd5hd in resource group rg-zstream-upgrade-4-22-zpgp48, error: failed waiting for cluster=\"cluster-zstream-4-22-pwd5hd\" in resourcegroup=\"rg-zstream-upgrade-4-22-zpgp48\" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster cluster-zstream-4-22-pwd5hd in resource group rg-zstream-upgrade-4-22-zpgp48, error: context deadline exceeded\n    ...\noccurred\nfail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:102]: failed to create HCP cluster \"cluster-zstream-4-22-pwd5hd\" with version 4.22.0 on candidate channel\nUnexpected error:\n    \u003c*fmt.wrapErrors | 0xc0008f8720\u003e: \n    failed to create HCP cluster cluster-zstream-4-22-pwd5hd, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster cluster-zstream-4-22-pwd5hd in resource group rg-zstream-upgrade-4-22-zpgp48, error: failed waiting for cluster=\"cluster-zstream-4-22-pwd5hd\" in resourcegroup=\"rg-zstream-upgrade-4-22-zpgp48\" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster cluster-zstream-4-22-pwd5hd in resource group rg-zstream-upgrade-4-22-zpgp48, error: context deadline exceeded\n    ...\noccurred","failure_pattern_match":{"status":"clustered","failure_pattern_id":"6ed54686dd2f6d32054cbf144906e399a60d9c8b1a6d87d91850a99759063298","failure_pattern":"timeout during CreateHCPClusterAndWait; context deadline exceeded","search_query":"failed to create HCP cluster idms-e2e-hcp-cluster, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster idms-e2e-hcp-cluster in resource group idms-c2ddz2, error: failed waiting for cluster=\"idms-e2e-hcp-cluster\" in resourcegroup=\"idms-c2ddz2\" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster idms-e2e-hcp-cluster in resource group idms-c2ddz2, error: context deadline exceeded"}},{"row_id":"ddc15129345f8e265f1750d50af19b061b5a0ffe70aea95a4dd8c17f3d19a323","run_url":"https://prow.ci.openshift.org/view/gs/test-platform-results/logs/branch-ci-Azure-ARO-HCP-main-e2e-stage-e2e-parallel/2068019178161311744","occurred_at":"2026-06-19T17:12:48Z","failed_at":"e2e","signature_id":"3be0c3387f6d7d8cef7cf300c58d91b3ee1412e0ad27b6f54dec13fc924ef2be","test_name":"Create HCPOpenShiftCluster with Private KeyVault should create a cluster with private keyvault using v20251223preview API","test_suite":"stage/parallel","failure_text":"fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:136]: failed to create node pool \"np-1\" for private keyvault cluster \"private-kv-cluster\"\nUnexpected error:\n    \u003c*fmt.wrapErrors | 0xc001555890\u003e: \n    failed to create NodePool np-1, caused by: timeout '20.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group private-keyvault-xz2vs2, error: failed waiting for nodepool=\"np-1\" for cluster \"private-kv-cluster\" in resourcegroup=\"private-keyvault-xz2vs2\" to finish creating: context deadline exceeded\n    ...\noccurred\nfail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:136]: failed to create node pool \"np-1\" for private keyvault cluster \"private-kv-cluster\"\nUnexpected error:\n    \u003c*fmt.wrapErrors | 0xc001555890\u003e: \n    failed to create NodePool np-1, caused by: timeout '20.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group private-keyvault-xz2vs2, error: failed waiting for nodepool=\"np-1\" for cluster \"private-kv-cluster\" in resourcegroup=\"private-keyvault-xz2vs2\" to finish creating: context deadline exceeded\n    ...\noccurred","failure_pattern_match":{"status":"clustered","failure_pattern_id":"941cbd5c6e3f3d8ed572893a689b2dfded75f418444a1a0d01baa9fc9e30d9c0","failure_pattern":"CreateNodePoolFromParam timed out after \u003cminutes\u003e minutes; context deadline exceeded","search_query":"timeout '20.000000' minutes exceeded during CreateNodePoolFromParam for node pool cilium-np in resource group complex-cilium-kv-kdtw5w, error: failed waiting for nodepool=\"cilium-np\" for cluster \"cilium-cluster\" in resourcegroup=\"complex-cilium-kv-kdtw5w\" to finish creating: context deadline exceeded"}}],"failure_pattern_summary":{"signature_count":2,"failure_pattern_ids":["6ed54686dd2f6d32054cbf144906e399a60d9c8b1a6d87d91850a99759063298","941cbd5c6e3f3d8ed572893a689b2dfded75f418444a1a0d01baa9fc9e30d9c0"],"matched_occurrences":2,"unmatched_occurrences":0,"match_summary":"multiple_clustered"}},{"run":{"environment":"stg","run_url":"https://prow.ci.openshift.org/view/gs/test-platform-results/logs/branch-ci-Azure-ARO-HCP-main-e2e-stage-e2e-parallel/2067901432899571712","job_name":"branch-ci-Azure-ARO-HCP-main-e2e-stage-e2e-parallel","pr_number":0,"pr_state":"","pr_sha":"","final_merged_sha":"","merged_pr":true,"post_good_commit":true,"failed":true,"occurred_at":"2026-06-19T09:24:54Z"},"failed_at":["e2e"],"failed_test_count":10,"occurrences":[{"row_id":"01ea37bce223fb53ed215f7f7dfaf4d2ad741d0bc0b4ad2bc6b17f68d7c89938","run_url":"https://prow.ci.openshift.org/view/gs/test-platform-results/logs/branch-ci-Azure-ARO-HCP-main-e2e-stage-e2e-parallel/2067901432899571712","occurred_at":"2026-06-19T09:24:54Z","failed_at":"e2e","signature_id":"31b613cf5d673bf5bd806a26a0e5ca838819b4ddc0d43a12c3d65de13e3fa8a2","test_name":"Customer should be able to create a cluster with an external auth config and get the external auth config","test_suite":"stage/parallel","failure_text":"fail [github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:285] A node timeout occurred and then the following failure was recorded in the timedout node before it exited:\nUnexpected error:\n    \u003c*errors.joinError | 0xc000eca540\u003e: \n    failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster=\"ea-cluster\" in resourcegroup=\"external-auth-cluster-8xzs29\" to finish deleting: context canceled\n    ...\noccurred  \nfail [:0]: A node timeout occurred\nfail [github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:285] A node timeout occurred and then the following failure was recorded in the timedout node before it exited:\nUnexpected error:\n    \u003c*errors.joinError | 0xc000eca540\u003e: \n    failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster=\"ea-cluster\" in resourcegroup=\"external-auth-cluster-8xzs29\" to finish deleting: context canceled\n    ...\noccurred  \nfail [:0]: A node timeout occurred","failure_pattern_match":{"status":"clustered","failure_pattern_id":"7f0e44476112ce4e675f687843407c03482d80d265239de5b2ddbd45c441a04e","failure_pattern":"failed to cleanup resource group: at least one HCP cluster \u003ccluster\u003e to delete: failed waiting for hcpcluster=\"\u003ccluster\u003e\" in resourcegroup=\"\u003cresource-group\u003e\" to finish deleting: context canceled","search_query":"failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster=\"arm64-vm-hcp-cluster\" in resourcegroup=\"arm64-vm-cluster-6hj75l\" to finish deleting: context canceled"}},{"row_id":"6774760b9a2a939e86d7b4784689f9476396e8192588c6cc97a3447b5c1af17c","run_url":"https://prow.ci.openshift.org/view/gs/test-platform-results/logs/branch-ci-Azure-ARO-HCP-main-e2e-stage-e2e-parallel/2067901432899571712","occurred_at":"2026-06-19T09:24:54Z","failed_at":"e2e","signature_id":"b7571795d3deb98b07408317659f2d5d60574597919cc8f4387db6d1bcb4c411","test_name":"Customer should be able to lifecycle and confirm external auth on a cluster","test_suite":"stage/parallel","failure_text":"fail [github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:285] A node timeout occurred and then the following failure was recorded in the timedout node before it exited:\nUnexpected error:\n    \u003c*errors.joinError | 0xc00117c240\u003e: \n    failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster=\"ea-list55g6ng\" in resourcegroup=\"ea-list-vp8mzk\" to finish deleting: context canceled\n    ...\noccurred  \nfail [:0]: A node timeout occurred\nfail [github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:285] A node timeout occurred and then the following failure was recorded in the timedout node before it exited:\nUnexpected error:\n    \u003c*errors.joinError | 0xc00117c240\u003e: \n    failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster=\"ea-list55g6ng\" in resourcegroup=\"ea-list-vp8mzk\" to finish deleting: context canceled\n    ...\noccurred  \nfail [:0]: A node timeout occurred","failure_pattern_match":{"status":"clustered","failure_pattern_id":"7f0e44476112ce4e675f687843407c03482d80d265239de5b2ddbd45c441a04e","failure_pattern":"failed to cleanup resource group: at least one HCP cluster \u003ccluster\u003e to delete: failed waiting for hcpcluster=\"\u003ccluster\u003e\" in resourcegroup=\"\u003cresource-group\u003e\" to finish deleting: context canceled","search_query":"failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster=\"arm64-vm-hcp-cluster\" in resourcegroup=\"arm64-vm-cluster-6hj75l\" to finish deleting: context canceled"}},{"row_id":"777f7caf3e95249eacbe2728d94e88bc4762875074d659d11ba0069c51b07230","run_url":"https://prow.ci.openshift.org/view/gs/test-platform-results/logs/branch-ci-Azure-ARO-HCP-main-e2e-stage-e2e-parallel/2067901432899571712","occurred_at":"2026-06-19T09:24:54Z","failed_at":"e2e","signature_id":"5397dbb1fbe98d1f2d0743076074210dfb88103a7aa8ab00ed825be01bb339ec","test_name":"Authorized CIDRs Connectivity should allow API access only from authorized VM IP","test_suite":"stage/parallel","failure_text":"fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:380]: Timed out after 300.006s.\nThe function passed to Eventually failed at /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:365 with:\nkubectl should execute successfully from authorized VM with OIDC token\nUnexpected error:\n    \u003c*errors.errorString | 0xc000a5a270\u003e: \n    E0619 10:12:30.439124    2434 memcache.go:265] \"Unhandled Error\" err=\"couldn't get current server API group list: the server has asked for the client to provide credentials\"\n    E0619 10:12:30.456274    2434 memcache.go:265] \"Unhandled Error\" err=\"couldn't get current server API group list: the server has asked for the client to provide credentials\"\n    E0619 10:12:30.472320    2434 memcache.go:265] \"Unhandled Error\" err=\"couldn't get current server API group list: the server has asked for the client to provide credentials\"\n    E0619 10:12:30.488740    2434 memcache.go:265] \"Unhandled Error\" err=\"couldn't get current server API group list: the server has asked for the client to provide credentials\"\n    E0619 10:12:30.509879    2434 memcache.go:265] \"Unhandled Error\" err=\"couldn't get current server API group list: the server has asked for the client to provide credentials\"\n    error: You must be logged in to the server (the server has asked for the client to provide credentials)\n    ...\noccurred\nfail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:380]: Timed out after 300.006s.\nThe function passed to Eventually failed at /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:365 with:\nkubectl should execute successfully from authorized VM with OIDC token\nUnexpected error:\n    \u003c*errors.errorString | 0xc000a5a270\u003e: \n    E0619 10:12:30.439124    2434 memcache.go:265] \"Unhandled Error\" err=\"couldn't get current server API group list: the server has asked for the client to provide credentials\"\n    E0619 10:12:30.456274    2434 memcache.go:265] \"Unhandled Error\" err=\"couldn't get current server API group list: the server has asked for the client to provide credentials\"\n    E0619 10:12:30.472320    2434 memcache.go:265] \"Unhandled Error\" err=\"couldn't get current server API group list: the server has asked for the client to provide credentials\"\n    E0619 10:12:30.488740    2434 memcache.go:265] \"Unhandled Error\" err=\"couldn't get current server API group list: the server has asked for the client to provide credentials\"\n    E0619 10:12:30.509879    2434 memcache.go:265] \"Unhandled Error\" err=\"couldn't get current server API group list: the server has asked for the client to provide credentials\"\n    error: You must be logged in to the server (the server has asked for the client to provide credentials)\n    ...\noccurred","failure_pattern_match":{"status":"clustered","failure_pattern_id":"09af6a8d28e7b0d3f3b7f296bf6332c077f2f9c03b6d3d6d167ff2e411eb11ca","failure_pattern":"couldn't get current server API group list: the server has asked for the client to provide credentials","search_query":"couldn't get current server API group list: the server has asked for the client to provide credentials"}},{"row_id":"7cadab642331bffc68c75d7115bb05b52ac6c5d1f77a93cff86eff0a50e0a79e","run_url":"https://prow.ci.openshift.org/view/gs/test-platform-results/logs/branch-ci-Azure-ARO-HCP-main-e2e-stage-e2e-parallel/2067901432899571712","occurred_at":"2026-06-19T09:24:54Z","failed_at":"e2e","signature_id":"1117a8b53dc279e5830c029662237c5c98fe8918375358e70b0ebb163256bc03","test_name":"Customer should be able to create node pools with ARM64-based VMs","test_suite":"stage/parallel","failure_text":"fail [github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:285] A node timeout occurred and then the following failure was recorded in the timedout node before it exited:\nUnexpected error:\n    \u003c*errors.joinError | 0xc0013ea180\u003e: \n    failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster=\"arm64-vm-hcp-cluster\" in resourcegroup=\"arm64-vm-cluster-6hj75l\" to finish deleting: context canceled\n    ...\noccurred  \nfail [:0]: A node timeout occurred\nfail [github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:285] A node timeout occurred and then the following failure was recorded in the timedout node before it exited:\nUnexpected error:\n    \u003c*errors.joinError | 0xc0013ea180\u003e: \n    failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster=\"arm64-vm-hcp-cluster\" in resourcegroup=\"arm64-vm-cluster-6hj75l\" to finish deleting: context canceled\n    ...\noccurred  \nfail [:0]: A node timeout occurred","failure_pattern_match":{"status":"clustered","failure_pattern_id":"7f0e44476112ce4e675f687843407c03482d80d265239de5b2ddbd45c441a04e","failure_pattern":"failed to cleanup resource group: at least one HCP cluster \u003ccluster\u003e to delete: failed waiting for hcpcluster=\"\u003ccluster\u003e\" in resourcegroup=\"\u003cresource-group\u003e\" to finish deleting: context canceled","search_query":"failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster=\"arm64-vm-hcp-cluster\" in resourcegroup=\"arm64-vm-cluster-6hj75l\" to finish deleting: context canceled"}},{"row_id":"9ef4178980ab5c023f89f5dab6a9726e32be0a8126b7367ac0c17b44826099b0","run_url":"https://prow.ci.openshift.org/view/gs/test-platform-results/logs/branch-ci-Azure-ARO-HCP-main-e2e-stage-e2e-parallel/2067901432899571712","occurred_at":"2026-06-19T09:24:54Z","failed_at":"e2e","signature_id":"6ccf461a1af6aa17cfa86e7486289d4d607d54bf9c1223ac3dacefc970840102","test_name":"Create HCPOpenShiftCluster with Private KeyVault should create a cluster with private keyvault using v20251223preview API","test_suite":"stage/parallel","failure_text":"fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:136]: failed to create node pool \"np-1\" for private keyvault cluster \"private-kv-cluster\"\nUnexpected error:\n    \u003c*fmt.wrapErrors | 0xc000efed80\u003e: \n    failed to create NodePool np-1, caused by: timeout '20.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group private-keyvault-2w2v28, error: failed waiting for nodepool=\"np-1\" for cluster \"private-kv-cluster\" in resourcegroup=\"private-keyvault-2w2v28\" to finish creating: context deadline exceeded\n    ...\noccurred\nfail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:136]: failed to create node pool \"np-1\" for private keyvault cluster \"private-kv-cluster\"\nUnexpected error:\n    \u003c*fmt.wrapErrors | 0xc000efed80\u003e: \n    failed to create NodePool np-1, caused by: timeout '20.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group private-keyvault-2w2v28, error: failed waiting for nodepool=\"np-1\" for cluster \"private-kv-cluster\" in resourcegroup=\"private-keyvault-2w2v28\" to finish creating: context deadline exceeded\n    ...\noccurred","failure_pattern_match":{"status":"clustered","failure_pattern_id":"941cbd5c6e3f3d8ed572893a689b2dfded75f418444a1a0d01baa9fc9e30d9c0","failure_pattern":"CreateNodePoolFromParam timed out after \u003cminutes\u003e minutes; context deadline exceeded","search_query":"timeout '20.000000' minutes exceeded during CreateNodePoolFromParam for node pool cilium-np in resource group complex-cilium-kv-kdtw5w, error: failed waiting for nodepool=\"cilium-np\" for cluster \"cilium-cluster\" in resourcegroup=\"complex-cilium-kv-kdtw5w\" to finish creating: context deadline exceeded"}},{"row_id":"a2a8118bc6de7e1b70b7d0d45a97e758fbd9ce716fe58a62b915270bdd2c39ea","run_url":"https://prow.ci.openshift.org/view/gs/test-platform-results/logs/branch-ci-Azure-ARO-HCP-main-e2e-stage-e2e-parallel/2067901432899571712","occurred_at":"2026-06-19T09:24:54Z","failed_at":"e2e","signature_id":"fb3bcfee6dc00774308219d016be9e615daa88cd9fca051875f068a6dd4034eb","test_name":"Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 4.19","test_suite":"stage/parallel","failure_text":"fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:124]: Timed out after 2700.003s.\nExpected success, but got an error:\n    \u003c*errors.joinError | 0xc000fadb48\u003e: \n    VerifyHostedControlPlaneZStreamUpgradeOnly(initial=4.19.33) failed: no version in clusterversion/version status.history is greater than initial \"4.19.33\"\n    ...\nfail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:124]: Timed out after 2700.003s.\nExpected success, but got an error:\n    \u003c*errors.joinError | 0xc000fadb48\u003e: \n    VerifyHostedControlPlaneZStreamUpgradeOnly(initial=4.19.33) failed: no version in clusterversion/version status.history is greater than initial \"4.19.33\"\n    ...","failure_pattern_match":{"status":"clustered","failure_pattern_id":"604745dc1eb894c4ac98add4f5ede2030b2008c5ab8972455c0f282bc06425ad","failure_pattern":"VerifyHostedControlPlaneZStreamUpgradeOnly(initial=4","search_query":"VerifyHostedControlPlaneZStreamUpgradeOnly(initial=4"}},{"row_id":"d41e4fe89854296e6d4c0284ed6a7ec22a0a0f503c80f36281774e248b76e4ed","run_url":"https://prow.ci.openshift.org/view/gs/test-platform-results/logs/branch-ci-Azure-ARO-HCP-main-e2e-stage-e2e-parallel/2067901432899571712","occurred_at":"2026-06-19T09:24:54Z","failed_at":"e2e","signature_id":"3ce65a0c90bda6aef7e93518a6e64f115b0f4f55aebb31ef8bf03c03aea2d36b","test_name":"Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 4.22","test_suite":"stage/parallel","failure_text":"fail [github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:285] A node timeout occurred and then the following failure was recorded in the timedout node before it exited:\nUnexpected error:\n    \u003c*errors.joinError | 0xc00100adf8\u003e: \n    failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster=\"cluster-zstream-4-22-w4nwvj\" in resourcegroup=\"rg-zstream-upgrade-4-22-bzhsdz\" to finish deleting: context canceled\n    ...\noccurred  \nfail [:0]: A node timeout occurred\nfail [github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:285] A node timeout occurred and then the following failure was recorded in the timedout node before it exited:\nUnexpected error:\n    \u003c*errors.joinError | 0xc00100adf8\u003e: \n    failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster=\"cluster-zstream-4-22-w4nwvj\" in resourcegroup=\"rg-zstream-upgrade-4-22-bzhsdz\" to finish deleting: context canceled\n    ...\noccurred  \nfail [:0]: A node timeout occurred","failure_pattern_match":{"status":"clustered","failure_pattern_id":"7f0e44476112ce4e675f687843407c03482d80d265239de5b2ddbd45c441a04e","failure_pattern":"failed to cleanup resource group: at least one HCP cluster \u003ccluster\u003e to delete: failed waiting for hcpcluster=\"\u003ccluster\u003e\" in resourcegroup=\"\u003cresource-group\u003e\" to finish deleting: context canceled","search_query":"failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster=\"arm64-vm-hcp-cluster\" in resourcegroup=\"arm64-vm-cluster-6hj75l\" to finish deleting: context canceled"}},{"row_id":"d6fc7ab27ae966cb9aabab148fc75964c696db0e018b30c50d5ff102adb86195","run_url":"https://prow.ci.openshift.org/view/gs/test-platform-results/logs/branch-ci-Azure-ARO-HCP-main-e2e-stage-e2e-parallel/2067901432899571712","occurred_at":"2026-06-19T09:24:54Z","failed_at":"e2e","signature_id":"d9a1a7ba504fc8019189033f938eecf808e3431a10a63f741065deaba2a433fe","test_name":"Customer should be able to create a HCP cluster and use cilium CNI plugin","test_suite":"stage/parallel","failure_text":"fail [github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:285] A node timeout occurred and then the following failure was recorded in the timedout node before it exited:\nUnexpected error:\n    \u003c*errors.joinError | 0xc00195c258\u003e: \n    failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster=\"cilium-cl\" in resourcegroup=\"cni-cilium-kvfgtb\" to finish deleting: context canceled\n    ...\noccurred  \nfail [:0]: A node timeout occurred\nfail [github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:285] A node timeout occurred and then the following failure was recorded in the timedout node before it exited:\nUnexpected error:\n    \u003c*errors.joinError | 0xc00195c258\u003e: \n    failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster=\"cilium-cl\" in resourcegroup=\"cni-cilium-kvfgtb\" to finish deleting: context canceled\n    ...\noccurred  \nfail [:0]: A node timeout occurred","failure_pattern_match":{"status":"clustered","failure_pattern_id":"7f0e44476112ce4e675f687843407c03482d80d265239de5b2ddbd45c441a04e","failure_pattern":"failed to cleanup resource group: at least one HCP cluster \u003ccluster\u003e to delete: failed waiting for hcpcluster=\"\u003ccluster\u003e\" in resourcegroup=\"\u003cresource-group\u003e\" to finish deleting: context canceled","search_query":"failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster=\"arm64-vm-hcp-cluster\" in resourcegroup=\"arm64-vm-cluster-6hj75l\" to finish deleting: context canceled"}},{"row_id":"dc34ed70bc808bd507ec72cbdfad5a1396f455acc78ca1565cf34427e40f2fde","run_url":"https://prow.ci.openshift.org/view/gs/test-platform-results/logs/branch-ci-Azure-ARO-HCP-main-e2e-stage-e2e-parallel/2067901432899571712","occurred_at":"2026-06-19T09:24:54Z","failed_at":"e2e","signature_id":"1e97fa199908195dff30b8f21a45f704d430818b4db92c1c37b6848392eb4ca3","test_name":"Customer should be able to create an HCP cluster and manage pull secrets","test_suite":"stage/parallel","failure_text":"fail [github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:285] A node timeout occurred and then the following failure was recorded in the timedout node before it exited:\nUnexpected error:\n    \u003c*errors.joinError | 0xc000cf47e0\u003e: \n    failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster=\"pullsecret-hcp-cluster\" in resourcegroup=\"pullsecret-test-rsq4mt\" to finish deleting: context canceled\n    ...\noccurred  \nfail [:0]: A node timeout occurred\nfail [github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:285] A node timeout occurred and then the following failure was recorded in the timedout node before it exited:\nUnexpected error:\n    \u003c*errors.joinError | 0xc000cf47e0\u003e: \n    failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster=\"pullsecret-hcp-cluster\" in resourcegroup=\"pullsecret-test-rsq4mt\" to finish deleting: context canceled\n    ...\noccurred  \nfail [:0]: A node timeout occurred","failure_pattern_match":{"status":"clustered","failure_pattern_id":"7f0e44476112ce4e675f687843407c03482d80d265239de5b2ddbd45c441a04e","failure_pattern":"failed to cleanup resource group: at least one HCP cluster \u003ccluster\u003e to delete: failed waiting for hcpcluster=\"\u003ccluster\u003e\" in resourcegroup=\"\u003cresource-group\u003e\" to finish deleting: context canceled","search_query":"failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster=\"arm64-vm-hcp-cluster\" in resourcegroup=\"arm64-vm-cluster-6hj75l\" to finish deleting: context canceled"}},{"row_id":"ed664d312c5f2439365278f4d99bc390bc326df1d796ad6393fd462e8e833593","run_url":"https://prow.ci.openshift.org/view/gs/test-platform-results/logs/branch-ci-Azure-ARO-HCP-main-e2e-stage-e2e-parallel/2067901432899571712","occurred_at":"2026-06-19T09:24:54Z","failed_at":"e2e","signature_id":"eaef53bf5b80e5e2da2ba9672f0c276cacdf93dedd9b3b1e84d377a3b08e62dc","test_name":"Customer should be able to create an HCP cluster and custom node pool osDisk size","test_suite":"stage/parallel","failure_text":"fail [github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:285] A node timeout occurred and then the following failure was recorded in the timedout node before it exited:\nUnexpected error:\n    \u003c*errors.joinError | 0xc0014164f8\u003e: \n    failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster=\"hcp-cluster-np-128\" in resourcegroup=\"clusternp128-b9wv2w\" to finish deleting: context canceled\n    ...\noccurred  \nfail [:0]: A node timeout occurred\nfail [github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:285] A node timeout occurred and then the following failure was recorded in the timedout node before it exited:\nUnexpected error:\n    \u003c*errors.joinError | 0xc0014164f8\u003e: \n    failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster=\"hcp-cluster-np-128\" in resourcegroup=\"clusternp128-b9wv2w\" to finish deleting: context canceled\n    ...\noccurred  \nfail [:0]: A node timeout occurred","failure_pattern_match":{"status":"clustered","failure_pattern_id":"7f0e44476112ce4e675f687843407c03482d80d265239de5b2ddbd45c441a04e","failure_pattern":"failed to cleanup resource group: at least one HCP cluster \u003ccluster\u003e to delete: failed waiting for hcpcluster=\"\u003ccluster\u003e\" in resourcegroup=\"\u003cresource-group\u003e\" to finish deleting: context canceled","search_query":"failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster=\"arm64-vm-hcp-cluster\" in resourcegroup=\"arm64-vm-cluster-6hj75l\" to finish deleting: context canceled"}}],"failure_pattern_summary":{"signature_count":10,"failure_pattern_ids":["09af6a8d28e7b0d3f3b7f296bf6332c077f2f9c03b6d3d6d167ff2e411eb11ca","604745dc1eb894c4ac98add4f5ede2030b2008c5ab8972455c0f282bc06425ad","7f0e44476112ce4e675f687843407c03482d80d265239de5b2ddbd45c441a04e","941cbd5c6e3f3d8ed572893a689b2dfded75f418444a1a0d01baa9fc9e30d9c0"],"matched_occurrences":10,"unmatched_occurrences":0,"match_summary":"multiple_clustered"}},{"run":{"environment":"stg","run_url":"https://prow.ci.openshift.org/view/gs/test-platform-results/logs/branch-ci-Azure-ARO-HCP-main-e2e-stage-e2e-parallel/2067839032129228800","job_name":"branch-ci-Azure-ARO-HCP-main-e2e-stage-e2e-parallel","pr_number":0,"pr_state":"","pr_sha":"","final_merged_sha":"","merged_pr":true,"post_good_commit":true,"failed":false,"occurred_at":"2026-06-19T05:16:58Z"},"failed_test_count":0,"failure_pattern_summary":{"signature_count":0,"matched_occurrences":0,"unmatched_occurrences":0,"match_summary":"no_failures"}},{"run":{"environment":"stg","run_url":"https://prow.ci.openshift.org/view/gs/test-platform-results/logs/periodic-ci-Azure-ARO-HCP-main-periodic-stage-e2e-parallel/2067789556920356864","job_name":"periodic-ci-Azure-ARO-HCP-main-periodic-stage-e2e-parallel","pr_number":0,"pr_state":"","pr_sha":"","final_merged_sha":"","merged_pr":true,"post_good_commit":true,"failed":true,"occurred_at":"2026-06-19T02:00:22Z"},"failed_at":["e2e"],"failed_test_count":15,"occurrences":[{"row_id":"0cb4fcbfafbd9bef5bb91878a013bcd0a690919f150ad158b86eff1546c4c440","run_url":"https://prow.ci.openshift.org/view/gs/test-platform-results/logs/periodic-ci-Azure-ARO-HCP-main-periodic-stage-e2e-parallel/2067789556920356864","occurred_at":"2026-06-19T02:00:22Z","failed_at":"e2e","signature_id":"b6a2fbf3bc1113ef4d28d0a3b2bfe7463bd6906ff38ff6b7be308c46e64815cc","test_name":"Customer should be able to use workload identity via the cluster OIDC issuer URL","test_suite":"stage/parallel","failure_text":"fail [github.com/Azure/ARO-HCP/test/e2e/oidc_issuer_workload_identity.go:216]: failed to create HCP cluster oidc-wi-cluster\nUnexpected error:\n    \u003c*fmt.wrapErrors | 0xc0009dcfc0\u003e: \n    failed to create HCP cluster oidc-wi-cluster, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster oidc-wi-cluster in resource group oidc-wi-tdlprd, error: failed waiting for cluster=\"oidc-wi-cluster\" in resourcegroup=\"oidc-wi-tdlprd\" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster oidc-wi-cluster in resource group oidc-wi-tdlprd, error: context deadline exceeded\n    ...\noccurred\nfail [github.com/Azure/ARO-HCP/test/e2e/oidc_issuer_workload_identity.go:216]: failed to create HCP cluster oidc-wi-cluster\nUnexpected error:\n    \u003c*fmt.wrapErrors | 0xc0009dcfc0\u003e: \n    failed to create HCP cluster oidc-wi-cluster, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster oidc-wi-cluster in resource group oidc-wi-tdlprd, error: failed waiting for cluster=\"oidc-wi-cluster\" in resourcegroup=\"oidc-wi-tdlprd\" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster oidc-wi-cluster in resource group oidc-wi-tdlprd, error: context deadline exceeded\n    ...\noccurred","failure_pattern_match":{"status":"clustered","failure_pattern_id":"6ed54686dd2f6d32054cbf144906e399a60d9c8b1a6d87d91850a99759063298","failure_pattern":"timeout during CreateHCPClusterAndWait; context deadline exceeded","search_query":"failed to create HCP cluster idms-e2e-hcp-cluster, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster idms-e2e-hcp-cluster in resource group idms-c2ddz2, error: failed waiting for cluster=\"idms-e2e-hcp-cluster\" in resourcegroup=\"idms-c2ddz2\" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster idms-e2e-hcp-cluster in resource group idms-c2ddz2, error: context deadline exceeded"}},{"row_id":"30bb466ed0b51286833b81dc1e61ebf9695cbcc0b4e20d21d3cafd7f32b4dc63","run_url":"https://prow.ci.openshift.org/view/gs/test-platform-results/logs/periodic-ci-Azure-ARO-HCP-main-periodic-stage-e2e-parallel/2067789556920356864","occurred_at":"2026-06-19T02:00:22Z","failed_at":"e2e","signature_id":"3cc10714b893c36b85d8a4cbcead78dfcd87b031c23eaec8a6ac73642d819ea8","test_name":"Customer should upgrade and update a nodepool from 4.21.z to 4.21.zLatest","test_suite":"stage/parallel","failure_text":"fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:132]: failed to create HCP cluster np-version-upgrade-cluster-qcslvf with version 4.21.21\nUnexpected error:\n    \u003c*fmt.wrapErrors | 0xc0003857a0\u003e: \n    failed to create HCP cluster np-version-upgrade-cluster-qcslvf, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster np-version-upgrade-cluster-qcslvf in resource group rg-np-version-upgrade-qcslvf-s488tr, error: failed waiting for cluster=\"np-version-upgrade-cluster-qcslvf\" in resourcegroup=\"rg-np-version-upgrade-qcslvf-s488tr\" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster np-version-upgrade-cluster-qcslvf in resource group rg-np-version-upgrade-qcslvf-s488tr, error: context deadline exceeded\n    ...\noccurred\nfail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:132]: failed to create HCP cluster np-version-upgrade-cluster-qcslvf with version 4.21.21\nUnexpected error:\n    \u003c*fmt.wrapErrors | 0xc0003857a0\u003e: \n    failed to create HCP cluster np-version-upgrade-cluster-qcslvf, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster np-version-upgrade-cluster-qcslvf in resource group rg-np-version-upgrade-qcslvf-s488tr, error: failed waiting for cluster=\"np-version-upgrade-cluster-qcslvf\" in resourcegroup=\"rg-np-version-upgrade-qcslvf-s488tr\" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster np-version-upgrade-cluster-qcslvf in resource group rg-np-version-upgrade-qcslvf-s488tr, error: context deadline exceeded\n    ...\noccurred","failure_pattern_match":{"status":"clustered","failure_pattern_id":"6ed54686dd2f6d32054cbf144906e399a60d9c8b1a6d87d91850a99759063298","failure_pattern":"timeout during CreateHCPClusterAndWait; context deadline exceeded","search_query":"failed to create HCP cluster idms-e2e-hcp-cluster, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster idms-e2e-hcp-cluster in resource group idms-c2ddz2, error: failed waiting for cluster=\"idms-e2e-hcp-cluster\" in resourcegroup=\"idms-c2ddz2\" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster idms-e2e-hcp-cluster in resource group idms-c2ddz2, error: context deadline exceeded"}},{"row_id":"3ad1715c0b5c405027650a451f5d5d353c2b50ce418a4fb4fe72b8eb152cebf1","run_url":"https://prow.ci.openshift.org/view/gs/test-platform-results/logs/periodic-ci-Azure-ARO-HCP-main-periodic-stage-e2e-parallel/2067789556920356864","occurred_at":"2026-06-19T02:00:22Z","failed_at":"e2e","signature_id":"8aba20e82165edc99cd629694344802ce1e7260711cba712a5c35e04f4da953a","test_name":"Customer should be able to forward control plane logs to a storage account and Event Hub via shoebox diagnostic settings","test_suite":"stage/parallel","failure_text":"fail [github.com/Azure/ARO-HCP/test/e2e/shoebox_logs.go:269]: failed to create HCP cluster shoebox-hcp-cluster\nUnexpected error:\n    \u003c*fmt.wrapErrors | 0xc0015be330\u003e: \n    failed to create HCP cluster shoebox-hcp-cluster, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster shoebox-hcp-cluster in resource group shoebox-cluster-k8h4mh, error: failed waiting for cluster=\"shoebox-hcp-cluster\" in resourcegroup=\"shoebox-cluster-k8h4mh\" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster shoebox-hcp-cluster in resource group shoebox-cluster-k8h4mh, error: context deadline exceeded\n    ...\noccurred\nfail [github.com/Azure/ARO-HCP/test/e2e/shoebox_logs.go:269]: failed to create HCP cluster shoebox-hcp-cluster\nUnexpected error:\n    \u003c*fmt.wrapErrors | 0xc0015be330\u003e: \n    failed to create HCP cluster shoebox-hcp-cluster, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster shoebox-hcp-cluster in resource group shoebox-cluster-k8h4mh, error: failed waiting for cluster=\"shoebox-hcp-cluster\" in resourcegroup=\"shoebox-cluster-k8h4mh\" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster shoebox-hcp-cluster in resource group shoebox-cluster-k8h4mh, error: context deadline exceeded\n    ...\noccurred","failure_pattern_match":{"status":"clustered","failure_pattern_id":"6ed54686dd2f6d32054cbf144906e399a60d9c8b1a6d87d91850a99759063298","failure_pattern":"timeout during CreateHCPClusterAndWait; context deadline exceeded","search_query":"failed to create HCP cluster idms-e2e-hcp-cluster, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster idms-e2e-hcp-cluster in resource group idms-c2ddz2, error: failed waiting for cluster=\"idms-e2e-hcp-cluster\" in resourcegroup=\"idms-c2ddz2\" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster idms-e2e-hcp-cluster in resource group idms-c2ddz2, error: context deadline exceeded"}},{"row_id":"3b97d6144213a69a1454052723648ff5b9a64156c2506b1bb4de84f81325fe03","run_url":"https://prow.ci.openshift.org/view/gs/test-platform-results/logs/periodic-ci-Azure-ARO-HCP-main-periodic-stage-e2e-parallel/2067789556920356864","occurred_at":"2026-06-19T02:00:22Z","failed_at":"e2e","signature_id":"55b3408693b6cdf7b71853feed56789088a942cb8abdf6bdb1a0b70d99b11fe2","test_name":"Customer should upgrade and update a nodepool from 4.20.z to 4.21.zLatest","test_suite":"stage/parallel","failure_text":"fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:132]: failed to create HCP cluster np-version-upgrade-cluster-6vplhz with version 4.21.21\nUnexpected error:\n    \u003c*fmt.wrapErrors | 0xc0009cff20\u003e: \n    failed to create HCP cluster np-version-upgrade-cluster-6vplhz, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster np-version-upgrade-cluster-6vplhz in resource group rg-np-version-upgrade-6vplhz-c2sl2f, error: failed waiting for cluster=\"np-version-upgrade-cluster-6vplhz\" in resourcegroup=\"rg-np-version-upgrade-6vplhz-c2sl2f\" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster np-version-upgrade-cluster-6vplhz in resource group rg-np-version-upgrade-6vplhz-c2sl2f, error: context deadline exceeded\n    ...\noccurred\nfail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:132]: failed to create HCP cluster np-version-upgrade-cluster-6vplhz with version 4.21.21\nUnexpected error:\n    \u003c*fmt.wrapErrors | 0xc0009cff20\u003e: \n    failed to create HCP cluster np-version-upgrade-cluster-6vplhz, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster np-version-upgrade-cluster-6vplhz in resource group rg-np-version-upgrade-6vplhz-c2sl2f, error: failed waiting for cluster=\"np-version-upgrade-cluster-6vplhz\" in resourcegroup=\"rg-np-version-upgrade-6vplhz-c2sl2f\" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster np-version-upgrade-cluster-6vplhz in resource group rg-np-version-upgrade-6vplhz-c2sl2f, error: context deadline exceeded\n    ...\noccurred","failure_pattern_match":{"status":"clustered","failure_pattern_id":"6ed54686dd2f6d32054cbf144906e399a60d9c8b1a6d87d91850a99759063298","failure_pattern":"timeout during CreateHCPClusterAndWait; context deadline exceeded","search_query":"failed to create HCP cluster idms-e2e-hcp-cluster, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster idms-e2e-hcp-cluster in resource group idms-c2ddz2, error: failed waiting for cluster=\"idms-e2e-hcp-cluster\" in resourcegroup=\"idms-c2ddz2\" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster idms-e2e-hcp-cluster in resource group idms-c2ddz2, error: context deadline exceeded"}},{"row_id":"3dbe9f631a5d333207080362d894faf25312b2e07ce93ff05836d31677249d89","run_url":"https://prow.ci.openshift.org/view/gs/test-platform-results/logs/periodic-ci-Azure-ARO-HCP-main-periodic-stage-e2e-parallel/2067789556920356864","occurred_at":"2026-06-19T02:00:22Z","failed_at":"e2e","signature_id":"7e15605c28cf56395c8b25e35318b595b0886eb41d95aa87ddafa7a6cefab68c","test_name":"Customer should be able to create an HCP cluster with Image Registry not present","test_suite":"stage/parallel","failure_text":"fail [github.com/Azure/ARO-HCP/test/e2e/image_registry_cluster_create.go:88]: failed to create HCP cluster with image registry disabled\nUnexpected error:\n    \u003c*fmt.wrapErrors | 0xc0016442d0\u003e: \n    failed to create HCP cluster disabled-image-registry-hcp-cluster, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster disabled-image-registry-hcp-cluster in resource group disabled-image-registry-kfplz2, error: failed waiting for cluster=\"disabled-image-registry-hcp-cluster\" in resourcegroup=\"disabled-image-registry-kfplz2\" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster disabled-image-registry-hcp-cluster in resource group disabled-image-registry-kfplz2, error: context deadline exceeded\n    ...\noccurred\nfail [github.com/Azure/ARO-HCP/test/e2e/image_registry_cluster_create.go:88]: failed to create HCP cluster with image registry disabled\nUnexpected error:\n    \u003c*fmt.wrapErrors | 0xc0016442d0\u003e: \n    failed to create HCP cluster disabled-image-registry-hcp-cluster, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster disabled-image-registry-hcp-cluster in resource group disabled-image-registry-kfplz2, error: failed waiting for cluster=\"disabled-image-registry-hcp-cluster\" in resourcegroup=\"disabled-image-registry-kfplz2\" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster disabled-image-registry-hcp-cluster in resource group disabled-image-registry-kfplz2, error: context deadline exceeded\n    ...\noccurred","failure_pattern_match":{"status":"clustered","failure_pattern_id":"6ed54686dd2f6d32054cbf144906e399a60d9c8b1a6d87d91850a99759063298","failure_pattern":"timeout during CreateHCPClusterAndWait; context deadline exceeded","search_query":"failed to create HCP cluster idms-e2e-hcp-cluster, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster idms-e2e-hcp-cluster in resource group idms-c2ddz2, error: failed waiting for cluster=\"idms-e2e-hcp-cluster\" in resourcegroup=\"idms-c2ddz2\" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster idms-e2e-hcp-cluster in resource group idms-c2ddz2, error: context deadline exceeded"}},{"row_id":"473dac0750e6505d85e600628500dd8a80d3c0f01a397e28e677ad0d03bb9fc2","run_url":"https://prow.ci.openshift.org/view/gs/test-platform-results/logs/periodic-ci-Azure-ARO-HCP-main-periodic-stage-e2e-parallel/2067789556920356864","occurred_at":"2026-06-19T02:00:22Z","failed_at":"e2e","signature_id":"b09a51a11d623e667e3fcf957909c7ada66556841add6c2a966eb816b7af4c13","test_name":"Customer should be able to create a cluster with default autoscaling and a nodepool with autoscaling enabled up to replica limits","test_suite":"stage/parallel","failure_text":"fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:95]: failed to create HCP cluster np-autoscale-cluster\nUnexpected error:\n    \u003c*fmt.wrapErrors | 0xc0012482d0\u003e: \n    failed to create HCP cluster np-autoscale-cluster, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster np-autoscale-cluster in resource group np-autoscaling-c6ldl7, error: failed waiting for cluster=\"np-autoscale-cluster\" in resourcegroup=\"np-autoscaling-c6ldl7\" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster np-autoscale-cluster in resource group np-autoscaling-c6ldl7, error: context deadline exceeded\n    ...\noccurred\nfail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:95]: failed to create HCP cluster np-autoscale-cluster\nUnexpected error:\n    \u003c*fmt.wrapErrors | 0xc0012482d0\u003e: \n    failed to create HCP cluster np-autoscale-cluster, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster np-autoscale-cluster in resource group np-autoscaling-c6ldl7, error: failed waiting for cluster=\"np-autoscale-cluster\" in resourcegroup=\"np-autoscaling-c6ldl7\" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster np-autoscale-cluster in resource group np-autoscaling-c6ldl7, error: context deadline exceeded\n    ...\noccurred","failure_pattern_match":{"status":"clustered","failure_pattern_id":"6ed54686dd2f6d32054cbf144906e399a60d9c8b1a6d87d91850a99759063298","failure_pattern":"timeout during CreateHCPClusterAndWait; context deadline exceeded","search_query":"failed to create HCP cluster idms-e2e-hcp-cluster, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster idms-e2e-hcp-cluster in resource group idms-c2ddz2, error: failed waiting for cluster=\"idms-e2e-hcp-cluster\" in resourcegroup=\"idms-c2ddz2\" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster idms-e2e-hcp-cluster in resource group idms-c2ddz2, error: context deadline exceeded"}},{"row_id":"6d60ac9807264d0a5d6247164202adca9afd4299605b6d3936c34da81b057870","run_url":"https://prow.ci.openshift.org/view/gs/test-platform-results/logs/periodic-ci-Azure-ARO-HCP-main-periodic-stage-e2e-parallel/2067789556920356864","occurred_at":"2026-06-19T02:00:22Z","failed_at":"e2e","signature_id":"ad7cd392157133110346bf1bede5a5334f2e248315b1213563e8d7c256af419e","test_name":"Nodepool Ephemeral OS Disk should create a nodepool with ephemeral OS disk when autoRepair is enabled","test_suite":"stage/parallel","failure_text":"fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_ephemeral_osdisk.go:111]: failed to create HCP cluster ephemeral-disk\nUnexpected error:\n    \u003c*fmt.wrapErrors | 0xc001546300\u003e: \n    failed to create HCP cluster ephemeral-disk, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster ephemeral-disk in resource group ephemeral-osdisk-n8z5ww, error: failed waiting for cluster=\"ephemeral-disk\" in resourcegroup=\"ephemeral-osdisk-n8z5ww\" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster ephemeral-disk in resource group ephemeral-osdisk-n8z5ww, error: context deadline exceeded\n    ...\noccurred\nfail [github.com/Azure/ARO-HCP/test/e2e/nodepool_ephemeral_osdisk.go:111]: failed to create HCP cluster ephemeral-disk\nUnexpected error:\n    \u003c*fmt.wrapErrors | 0xc001546300\u003e: \n    failed to create HCP cluster ephemeral-disk, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster ephemeral-disk in resource group ephemeral-osdisk-n8z5ww, error: failed waiting for cluster=\"ephemeral-disk\" in resourcegroup=\"ephemeral-osdisk-n8z5ww\" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster ephemeral-disk in resource group ephemeral-osdisk-n8z5ww, error: context deadline exceeded\n    ...\noccurred","failure_pattern_match":{"status":"clustered","failure_pattern_id":"6ed54686dd2f6d32054cbf144906e399a60d9c8b1a6d87d91850a99759063298","failure_pattern":"timeout during CreateHCPClusterAndWait; context deadline exceeded","search_query":"failed to create HCP cluster idms-e2e-hcp-cluster, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster idms-e2e-hcp-cluster in resource group idms-c2ddz2, error: failed waiting for cluster=\"idms-e2e-hcp-cluster\" in resourcegroup=\"idms-c2ddz2\" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster idms-e2e-hcp-cluster in resource group idms-c2ddz2, error: context deadline exceeded"}},{"row_id":"75e81f23d802c6a698f8c3d215f50d16bec55fd15e2dd1dbb3981cc823959797","run_url":"https://prow.ci.openshift.org/view/gs/test-platform-results/logs/periodic-ci-Azure-ARO-HCP-main-periodic-stage-e2e-parallel/2067789556920356864","occurred_at":"2026-06-19T02:00:22Z","failed_at":"e2e","signature_id":"e063a7698640f804264cb5fc23acba1189a7b38b6ae02851a6951ad6431944d9","test_name":"Customer should respect cluster-wide node limits with nodepool autoscaling","test_suite":"stage/parallel","failure_text":"fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:282]: failed to create HCP cluster node-limit-cluster with MaxNodesTotal limit\nUnexpected error:\n    \u003c*fmt.wrapErrors | 0xc000a304b0\u003e: \n    failed to create HCP cluster node-limit-cluster, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster node-limit-cluster in resource group node-limit-test-x6r5rr, error: failed waiting for cluster=\"node-limit-cluster\" in resourcegroup=\"node-limit-test-x6r5rr\" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster node-limit-cluster in resource group node-limit-test-x6r5rr, error: context deadline exceeded\n    ...\noccurred\nfail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:282]: failed to create HCP cluster node-limit-cluster with MaxNodesTotal limit\nUnexpected error:\n    \u003c*fmt.wrapErrors | 0xc000a304b0\u003e: \n    failed to create HCP cluster node-limit-cluster, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster node-limit-cluster in resource group node-limit-test-x6r5rr, error: failed waiting for cluster=\"node-limit-cluster\" in resourcegroup=\"node-limit-test-x6r5rr\" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster node-limit-cluster in resource group node-limit-test-x6r5rr, error: context deadline exceeded\n    ...\noccurred","failure_pattern_match":{"status":"clustered","failure_pattern_id":"6ed54686dd2f6d32054cbf144906e399a60d9c8b1a6d87d91850a99759063298","failure_pattern":"timeout during CreateHCPClusterAndWait; context deadline exceeded","search_query":"failed to create HCP cluster idms-e2e-hcp-cluster, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster idms-e2e-hcp-cluster in resource group idms-c2ddz2, error: failed waiting for cluster=\"idms-e2e-hcp-cluster\" in resourcegroup=\"idms-c2ddz2\" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster idms-e2e-hcp-cluster in resource group idms-c2ddz2, error: context deadline exceeded"}},{"row_id":"7fe93e9d170ea0e302231dcf8c7191d58030b1aa55dd09e95f37160a47f3ddd7","run_url":"https://prow.ci.openshift.org/view/gs/test-platform-results/logs/periodic-ci-Azure-ARO-HCP-main-periodic-stage-e2e-parallel/2067789556920356864","occurred_at":"2026-06-19T02:00:22Z","failed_at":"e2e","signature_id":"da9f39a85b17a6d6d8fa7e62d47869762cbecb9d23f6b4e96f12ed40d3d9ff51","test_name":"Customer should be able to create an HCP cluster and manage ImageDigestMirrors","test_suite":"stage/parallel","failure_text":"fail [github.com/Azure/ARO-HCP/test/e2e/idms_lifecycle.go:112]: failed to create HCP cluster with ImageDigestMirrors\nUnexpected error:\n    \u003c*fmt.wrapErrors | 0xc0016f4300\u003e: \n    failed to create HCP cluster idms-e2e-hcp-cluster, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster idms-e2e-hcp-cluster in resource group idms-c2ddz2, error: failed waiting for cluster=\"idms-e2e-hcp-cluster\" in resourcegroup=\"idms-c2ddz2\" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster idms-e2e-hcp-cluster in resource group idms-c2ddz2, error: context deadline exceeded\n    ...\noccurred\nfail [github.com/Azure/ARO-HCP/test/e2e/idms_lifecycle.go:112]: failed to create HCP cluster with ImageDigestMirrors\nUnexpected error:\n    \u003c*fmt.wrapErrors | 0xc0016f4300\u003e: \n    failed to create HCP cluster idms-e2e-hcp-cluster, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster idms-e2e-hcp-cluster in resource group idms-c2ddz2, error: failed waiting for cluster=\"idms-e2e-hcp-cluster\" in resourcegroup=\"idms-c2ddz2\" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster idms-e2e-hcp-cluster in resource group idms-c2ddz2, error: context deadline exceeded\n    ...\noccurred","failure_pattern_match":{"status":"clustered","failure_pattern_id":"6ed54686dd2f6d32054cbf144906e399a60d9c8b1a6d87d91850a99759063298","failure_pattern":"timeout during CreateHCPClusterAndWait; context deadline exceeded","search_query":"failed to create HCP cluster idms-e2e-hcp-cluster, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster idms-e2e-hcp-cluster in resource group idms-c2ddz2, error: failed waiting for cluster=\"idms-e2e-hcp-cluster\" in resourcegroup=\"idms-c2ddz2\" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster idms-e2e-hcp-cluster in resource group idms-c2ddz2, error: context deadline exceeded"}},{"row_id":"86d9829aa6b1c6dafe14e48ce324d9e6cd5263a326f6187474d6c0822a60350d","run_url":"https://prow.ci.openshift.org/view/gs/test-platform-results/logs/periodic-ci-Azure-ARO-HCP-main-periodic-stage-e2e-parallel/2067789556920356864","occurred_at":"2026-06-19T02:00:22Z","failed_at":"e2e","signature_id":"8ff5d062d6236573108c43c6c99ca4a3f676366baab1d4e5539f1d74eedf7ec1","test_name":"Customer should upgrade and update a nodepool from 4.20.z to 4.20.zLatest","test_suite":"stage/parallel","failure_text":"fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:132]: failed to create HCP cluster np-version-upgrade-cluster-mdfjlj with version 4.20.26\nUnexpected error:\n    \u003c*fmt.wrapErrors | 0xc00188c4e0\u003e: \n    failed to create HCP cluster np-version-upgrade-cluster-mdfjlj, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster np-version-upgrade-cluster-mdfjlj in resource group rg-np-version-upgrade-mdfjlj-dnmmnx, error: failed waiting for cluster=\"np-version-upgrade-cluster-mdfjlj\" in resourcegroup=\"rg-np-version-upgrade-mdfjlj-dnmmnx\" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster np-version-upgrade-cluster-mdfjlj in resource group rg-np-version-upgrade-mdfjlj-dnmmnx, error: context deadline exceeded\n    ...\noccurred\nfail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:132]: failed to create HCP cluster np-version-upgrade-cluster-mdfjlj with version 4.20.26\nUnexpected error:\n    \u003c*fmt.wrapErrors | 0xc00188c4e0\u003e: \n    failed to create HCP cluster np-version-upgrade-cluster-mdfjlj, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster np-version-upgrade-cluster-mdfjlj in resource group rg-np-version-upgrade-mdfjlj-dnmmnx, error: failed waiting for cluster=\"np-version-upgrade-cluster-mdfjlj\" in resourcegroup=\"rg-np-version-upgrade-mdfjlj-dnmmnx\" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster np-version-upgrade-cluster-mdfjlj in resource group rg-np-version-upgrade-mdfjlj-dnmmnx, error: context deadline exceeded\n    ...\noccurred","failure_pattern_match":{"status":"clustered","failure_pattern_id":"6ed54686dd2f6d32054cbf144906e399a60d9c8b1a6d87d91850a99759063298","failure_pattern":"timeout during CreateHCPClusterAndWait; context deadline exceeded","search_query":"failed to create HCP cluster idms-e2e-hcp-cluster, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster idms-e2e-hcp-cluster in resource group idms-c2ddz2, error: failed waiting for cluster=\"idms-e2e-hcp-cluster\" in resourcegroup=\"idms-c2ddz2\" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster idms-e2e-hcp-cluster in resource group idms-c2ddz2, error: context deadline exceeded"}},{"row_id":"89941d932f1e4957828ad0a81d434f7faeb092dde261e84251cddbbcb80e5167","run_url":"https://prow.ci.openshift.org/view/gs/test-platform-results/logs/periodic-ci-Azure-ARO-HCP-main-periodic-stage-e2e-parallel/2067789556920356864","occurred_at":"2026-06-19T02:00:22Z","failed_at":"e2e","signature_id":"b1a41213edd1b6fe4f374c76c66a9c53bc79afa9ba7b372b40e21df654d42f29","test_name":"Customer should downgrade a nodepool to a lower minor version from 4.21.zLatest to 4.19.zLatest","test_suite":"stage/parallel","failure_text":"fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:750]: failed to create HCP cluster\nUnexpected error:\n    \u003c*fmt.wrapErrors | 0xc000382510\u003e: \n    failed to create HCP cluster np-dg-minor-zdl42z, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster np-dg-minor-zdl42z in resource group rg-np-dg-minor-zdl42z-qqhqtg, error: failed waiting for cluster=\"np-dg-minor-zdl42z\" in resourcegroup=\"rg-np-dg-minor-zdl42z-qqhqtg\" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster np-dg-minor-zdl42z in resource group rg-np-dg-minor-zdl42z-qqhqtg, error: context deadline exceeded\n    ...\noccurred\nfail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:750]: failed to create HCP cluster\nUnexpected error:\n    \u003c*fmt.wrapErrors | 0xc000382510\u003e: \n    failed to create HCP cluster np-dg-minor-zdl42z, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster np-dg-minor-zdl42z in resource group rg-np-dg-minor-zdl42z-qqhqtg, error: failed waiting for cluster=\"np-dg-minor-zdl42z\" in resourcegroup=\"rg-np-dg-minor-zdl42z-qqhqtg\" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster np-dg-minor-zdl42z in resource group rg-np-dg-minor-zdl42z-qqhqtg, error: context deadline exceeded\n    ...\noccurred","failure_pattern_match":{"status":"clustered","failure_pattern_id":"6ed54686dd2f6d32054cbf144906e399a60d9c8b1a6d87d91850a99759063298","failure_pattern":"timeout during CreateHCPClusterAndWait; context deadline exceeded","search_query":"failed to create HCP cluster idms-e2e-hcp-cluster, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster idms-e2e-hcp-cluster in resource group idms-c2ddz2, error: failed waiting for cluster=\"idms-e2e-hcp-cluster\" in resourcegroup=\"idms-c2ddz2\" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster idms-e2e-hcp-cluster in resource group idms-c2ddz2, error: context deadline exceeded"}},{"row_id":"9293c51a2462eb0835ac25039d2259e5245ba5630d8911b48b2a92f86468538a","run_url":"https://prow.ci.openshift.org/view/gs/test-platform-results/logs/periodic-ci-Azure-ARO-HCP-main-periodic-stage-e2e-parallel/2067789556920356864","occurred_at":"2026-06-19T02:00:22Z","failed_at":"e2e","signature_id":"6b69e1873f74dec674b66ff76e51c563f1e0b03b8ac31dfc9a14f8374ff023a1","test_name":"Customer should be able to update node pool labels and taints","test_suite":"stage/parallel","failure_text":"fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_labels_taints.go:91]: failed to create HCP cluster cluster-np-labels-taints\nUnexpected error:\n    \u003c*fmt.wrapErrors | 0xc0015940c0\u003e: \n    failed to create HCP cluster cluster-np-labels-taints, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster cluster-np-labels-taints in resource group rg-np-labels-taints-qkf9r2, error: failed waiting for cluster=\"cluster-np-labels-taints\" in resourcegroup=\"rg-np-labels-taints-qkf9r2\" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster cluster-np-labels-taints in resource group rg-np-labels-taints-qkf9r2, error: context deadline exceeded\n    ...\noccurred\nfail [github.com/Azure/ARO-HCP/test/e2e/nodepool_labels_taints.go:91]: failed to create HCP cluster cluster-np-labels-taints\nUnexpected error:\n    \u003c*fmt.wrapErrors | 0xc0015940c0\u003e: \n    failed to create HCP cluster cluster-np-labels-taints, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster cluster-np-labels-taints in resource group rg-np-labels-taints-qkf9r2, error: failed waiting for cluster=\"cluster-np-labels-taints\" in resourcegroup=\"rg-np-labels-taints-qkf9r2\" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster cluster-np-labels-taints in resource group rg-np-labels-taints-qkf9r2, error: context deadline exceeded\n    ...\noccurred","failure_pattern_match":{"status":"clustered","failure_pattern_id":"6ed54686dd2f6d32054cbf144906e399a60d9c8b1a6d87d91850a99759063298","failure_pattern":"timeout during CreateHCPClusterAndWait; context deadline exceeded","search_query":"failed to create HCP cluster idms-e2e-hcp-cluster, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster idms-e2e-hcp-cluster in resource group idms-c2ddz2, error: failed waiting for cluster=\"idms-e2e-hcp-cluster\" in resourcegroup=\"idms-c2ddz2\" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster idms-e2e-hcp-cluster in resource group idms-c2ddz2, error: context deadline exceeded"}},{"row_id":"c40d383ff0e09249f3c4d778cf6a6c796e8e7db41109921c1c9d4fcaf94bef1d","run_url":"https://prow.ci.openshift.org/view/gs/test-platform-results/logs/periodic-ci-Azure-ARO-HCP-main-periodic-stage-e2e-parallel/2067789556920356864","occurred_at":"2026-06-19T02:00:22Z","failed_at":"e2e","signature_id":"e28722693e9eb46c5ab1aa590cdb77fab39ff5b72a03481650856e2acdbb0b8e","test_name":"MISE Routing routes to the correct frontend based on version header MISE v2 when x-ms-mise-version header is set","test_suite":"stage/parallel","failure_text":"fail [github.com/Azure/ARO-HCP/test/e2e/mise_routing.go:132]: failed to create HCP cluster\nUnexpected error:\n    \u003c*fmt.wrapErrors | 0xc001338780\u003e: \n    failed waiting for cluster=\"mise-routing\" in resourcegroup=\"mise-v2-smoke-26pvp6\" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223AndWait for cluster mise-routing in resource group mise-v2-smoke-26pvp6, error: context deadline exceeded\n    ...\noccurred\nfail [github.com/Azure/ARO-HCP/test/e2e/mise_routing.go:132]: failed to create HCP cluster\nUnexpected error:\n    \u003c*fmt.wrapErrors | 0xc001338780\u003e: \n    failed waiting for cluster=\"mise-routing\" in resourcegroup=\"mise-v2-smoke-26pvp6\" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223AndWait for cluster mise-routing in resource group mise-v2-smoke-26pvp6, error: context deadline exceeded\n    ...\noccurred","failure_pattern_match":{"status":"clustered","failure_pattern_id":"6ed54686dd2f6d32054cbf144906e399a60d9c8b1a6d87d91850a99759063298","failure_pattern":"timeout during CreateHCPClusterAndWait; context deadline exceeded","search_query":"failed to create HCP cluster idms-e2e-hcp-cluster, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster idms-e2e-hcp-cluster in resource group idms-c2ddz2, error: failed waiting for cluster=\"idms-e2e-hcp-cluster\" in resourcegroup=\"idms-c2ddz2\" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster idms-e2e-hcp-cluster in resource group idms-c2ddz2, error: context deadline exceeded"}},{"row_id":"cbaf273a5b06c64069ae7066cbb6c8e5d1ac89bb6247c1780788401e5970755c","run_url":"https://prow.ci.openshift.org/view/gs/test-platform-results/logs/periodic-ci-Azure-ARO-HCP-main-periodic-stage-e2e-parallel/2067789556920356864","occurred_at":"2026-06-19T02:00:22Z","failed_at":"e2e","signature_id":"e870671147f0c21eaa331cc511b601b2f7392da5b634448deaa11ed388987091","test_name":"ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.21","test_suite":"stage/parallel","failure_text":"fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:159]: failed to create node pool \"np-d5plzj\" for cluster \"cluster-candidate-4-21-d5plzj\"\nUnexpected error:\n    \u003c*fmt.wrapErrors | 0xc001584420\u003e: \n    failed to create NodePool np-d5plzj, caused by: timeout '20.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-d5plzj in resource group rg-candidate-4-21-dqdtjf, error: failed waiting for nodepool=\"np-d5plzj\" for cluster \"cluster-candidate-4-21-d5plzj\" in resourcegroup=\"rg-candidate-4-21-dqdtjf\" to finish creating: context deadline exceeded\n    ...\noccurred\nfail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:159]: failed to create node pool \"np-d5plzj\" for cluster \"cluster-candidate-4-21-d5plzj\"\nUnexpected error:\n    \u003c*fmt.wrapErrors | 0xc001584420\u003e: \n    failed to create NodePool np-d5plzj, caused by: timeout '20.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-d5plzj in resource group rg-candidate-4-21-dqdtjf, error: failed waiting for nodepool=\"np-d5plzj\" for cluster \"cluster-candidate-4-21-d5plzj\" in resourcegroup=\"rg-candidate-4-21-dqdtjf\" to finish creating: context deadline exceeded\n    ...\noccurred","failure_pattern_match":{"status":"clustered","failure_pattern_id":"941cbd5c6e3f3d8ed572893a689b2dfded75f418444a1a0d01baa9fc9e30d9c0","failure_pattern":"CreateNodePoolFromParam timed out after \u003cminutes\u003e minutes; context deadline exceeded","search_query":"timeout '20.000000' minutes exceeded during CreateNodePoolFromParam for node pool cilium-np in resource group complex-cilium-kv-kdtw5w, error: failed waiting for nodepool=\"cilium-np\" for cluster \"cilium-cluster\" in resourcegroup=\"complex-cilium-kv-kdtw5w\" to finish creating: context deadline exceeded"}},{"row_id":"fe4066664ae3d4b74488734a50efcfda3d334526384c73ec9ae5d67803ab55be","run_url":"https://prow.ci.openshift.org/view/gs/test-platform-results/logs/periodic-ci-Azure-ARO-HCP-main-periodic-stage-e2e-parallel/2067789556920356864","occurred_at":"2026-06-19T02:00:22Z","failed_at":"e2e","signature_id":"dafa117376bb74cf58dea24c05c2869c59ef59b08fc880fc5c2d23820c294254","test_name":"MISE Routing routes to the correct frontend based on version header default route returns no version header","test_suite":"stage/parallel","failure_text":"fail [github.com/Azure/ARO-HCP/test/e2e/mise_routing.go:132]: failed to create HCP cluster\nUnexpected error:\n    \u003c*fmt.wrapErrors | 0xc0017824e0\u003e: \n    failed waiting for cluster=\"mise-routing\" in resourcegroup=\"mise-default-smoke-r8c549\" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223AndWait for cluster mise-routing in resource group mise-default-smoke-r8c549, error: context deadline exceeded\n    ...\noccurred\nfail [github.com/Azure/ARO-HCP/test/e2e/mise_routing.go:132]: failed to create HCP cluster\nUnexpected error:\n    \u003c*fmt.wrapErrors | 0xc0017824e0\u003e: \n    failed waiting for cluster=\"mise-routing\" in resourcegroup=\"mise-default-smoke-r8c549\" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223AndWait for cluster mise-routing in resource group mise-default-smoke-r8c549, error: context deadline exceeded\n    ...\noccurred","failure_pattern_match":{"status":"clustered","failure_pattern_id":"6ed54686dd2f6d32054cbf144906e399a60d9c8b1a6d87d91850a99759063298","failure_pattern":"timeout during CreateHCPClusterAndWait; context deadline exceeded","search_query":"failed to create HCP cluster idms-e2e-hcp-cluster, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster idms-e2e-hcp-cluster in resource group idms-c2ddz2, error: failed waiting for cluster=\"idms-e2e-hcp-cluster\" in resourcegroup=\"idms-c2ddz2\" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster idms-e2e-hcp-cluster in resource group idms-c2ddz2, error: context deadline exceeded"}}],"failure_pattern_summary":{"signature_count":15,"failure_pattern_ids":["6ed54686dd2f6d32054cbf144906e399a60d9c8b1a6d87d91850a99759063298","941cbd5c6e3f3d8ed572893a689b2dfded75f418444a1a0d01baa9fc9e30d9c0"],"matched_occurrences":15,"unmatched_occurrences":0,"match_summary":"multiple_clustered"}}]}]}