CIHealth Run Log

↗ Open Failure patterns for this day

Environment: PROD

Time (UTC)	Job	Failed at	Result	PR	Failed tests	Details
23:00:06	periodic-ci-Azure-ARO-HCP-main-periodic-prod-e2e-parallel-ocp-nightly post-goodmerged PR No additional run metadata captured	e2e	failed	n/a	3	e2e (3) ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift 23:00:06 UTC · failed at e2e · test Customer should be able to create a cluster and node pools with aggregated advanced features · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:184]: failed to create HCP cluster "agg-cluster" with aggregated settings Unexpected error: <fmt.wrapError \| 0xc000aed260>: failed waiting for cluster="agg-cluster" in resourcegroup="feature-aggregation-z9m2c6q2wt9f" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/2381acc6-3e26-4e23-b87b-3b89fac035ca -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/2381acc6-3e26-4e23-b87b-3b89fac035ca", "name": "2381acc6-3e26-4e23-b87b-3b89fac035ca", "status": "Failed", "startTime": "2026-08-18T23:06:57.972257037Z", "endTime": "2026-08-18T23:25:57.729049787Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:184]: failed to create HCP cluster "agg-cluster" with aggregated settings Unexpected error: <fmt.wrapError \| 0xc000aed260>: failed waiting for cluster="agg-cluster" in resourcegroup="feature-aggregation-z9m2c6q2wt9f" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/2381acc6-3e26-4e23-b87b-3b89fac035ca -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/2381acc6-3e26-4e23-b87b-3b89fac035ca", "name": "2381acc6-3e26-4e23-b87b-3b89fac035ca", "status": "Failed", "startTime": "2026-08-18T23:06:57.972257037Z", "endTime": "2026-08-18T23:25:57.729049787Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred timeout during CreateHCPClusterAndWait; context deadline exceeded 23:00:06 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP nightly channel for 5.1 · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:107]: HCP cluster rg-nightly-5-1-mds7bfdnr4mb/cluster-nightly-5-1-jkzd98 should provision Unexpected error: <fmt.wrapErrors \| 0xc0012549c0>: failed to create HCP cluster cluster-nightly-5-1-jkzd98, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster cluster-nightly-5-1-jkzd98 in resource group rg-nightly-5-1-mds7bfdnr4mb, error: failed waiting for cluster="cluster-nightly-5-1-jkzd98" in resourcegroup="rg-nightly-5-1-mds7bfdnr4mb" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster cluster-nightly-5-1-jkzd98 in resource group rg-nightly-5-1-mds7bfdnr4mb, error: context deadline exceeded ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:107]: HCP cluster rg-nightly-5-1-mds7bfdnr4mb/cluster-nightly-5-1-jkzd98 should provision Unexpected error: <fmt.wrapErrors \| 0xc0012549c0>: failed to create HCP cluster cluster-nightly-5-1-jkzd98, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster cluster-nightly-5-1-jkzd98 in resource group rg-nightly-5-1-mds7bfdnr4mb, error: failed waiting for cluster="cluster-nightly-5-1-jkzd98" in resourcegroup="rg-nightly-5-1-mds7bfdnr4mb" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster cluster-nightly-5-1-jkzd98 in resource group rg-nightly-5-1-mds7bfdnr4mb, error: context deadline exceeded ... occurred timeout during CreateHCPClusterAndWait; context deadline exceeded 23:00:06 UTC · failed at e2e · test Customer should create a cluster with private KAS and verify API server is only reachable from VNet while ingress remains public · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:90]: failed to create HCP cluster "private-kas" with private KAS Unexpected error: <fmt.wrapErrors \| 0xc001c36390>: failed to create HCP cluster private-kas, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster private-kas in resource group private-kas-n2w89h2n5hbn, error: failed waiting for cluster="private-kas" in resourcegroup="private-kas-n2w89h2n5hbn" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster private-kas in resource group private-kas-n2w89h2n5hbn, error: context deadline exceeded ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:90]: failed to create HCP cluster "private-kas" with private KAS Unexpected error: <fmt.wrapErrors \| 0xc001c36390>: failed to create HCP cluster private-kas, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster private-kas in resource group private-kas-n2w89h2n5hbn, error: failed waiting for cluster="private-kas" in resourcegroup="private-kas-n2w89h2n5hbn" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster private-kas in resource group private-kas-n2w89h2n5hbn, error: context deadline exceeded ... occurred
02:00:45	periodic-ci-Azure-ARO-HCP-main-periodic-prod-e2e-parallel post-goodmerged PR No additional run metadata captured	e2e	failed	n/a	1	e2e (1) VerifyVMOSDiskCustomerEncryption(nodePool=<nodepool>) failed: OS disk encryption verification failed for 2/2 VMs: disk des-encrypt-des-np-z8ktk-l4d4d_OSDisk for VM des-encrypt-des-np-z8ktk-l4d4d has encryption type EncryptionAtRestWithPlatformKey, expected 02:00:45 UTC · failed at e2e · test Nodepool OS Disk Encryption should create a nodepool with customer-managed disk encryption via DES · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_osdisk_encryption.go:163]: cluster verification failed Unexpected error: <errors.joinError \| 0xc0013905a0>: VerifyVMOSDiskCustomerEncryption(nodePool=des-np) failed: OS disk encryption verification failed for 2/2 VMs: disk des-encrypt-des-np-z8ktk-l4d4d_OSDisk for VM des-encrypt-des-np-z8ktk-l4d4d has encryption type EncryptionAtRestWithPlatformKey, expected EncryptionAtRestWithCustomerKey disk des-encrypt-des-np-z8ktk-t2cjq_OSDisk for VM des-encrypt-des-np-z8ktk-t2cjq has encryption type EncryptionAtRestWithPlatformKey, expected EncryptionAtRestWithCustomerKey ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_osdisk_encryption.go:163]: cluster verification failed Unexpected error: <errors.joinError \| 0xc0013905a0>: VerifyVMOSDiskCustomerEncryption(nodePool=des-np) failed: OS disk encryption verification failed for 2/2 VMs: disk des-encrypt-des-np-z8ktk-l4d4d_OSDisk for VM des-encrypt-des-np-z8ktk-l4d4d has encryption type EncryptionAtRestWithPlatformKey, expected EncryptionAtRestWithCustomerKey disk des-encrypt-des-np-z8ktk-t2cjq_OSDisk for VM des-encrypt-des-np-z8ktk-t2cjq has encryption type EncryptionAtRestWithPlatformKey, expected EncryptionAtRestWithCustomerKey ... occurred
01:17:15	branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel post-goodmerged PR No additional run metadata captured	e2e	failed	n/a	48	e2e (48) ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift 01:17:15 UTC · failed at e2e · test Customer should be able to successfully upgrade control plane minor version from 4.20 minor to 4.21 minor · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:182]: failed to create HCP cluster "cp-ystream-upgrade-4-21-pl2828" with install version 4.20.34 Unexpected error: <fmt.wrapError \| 0xc001408920>: failed to create HCP cluster cp-ystream-upgrade-4-21-pl2828: failed waiting for cluster="cp-ystream-upgrade-4-21-pl2828" in resourcegroup="rg-cp-ystream-upgrade-4-21-2nlwvsvrv9bp" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/8b146388-d74d-42e6-bdaf-8a3edf1b3beb -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/8b146388-d74d-42e6-bdaf-8a3edf1b3beb", "name": "8b146388-d74d-42e6-bdaf-8a3edf1b3beb", "status": "Failed", "startTime": "2026-08-18T02:21:03.015049333Z", "endTime": "2026-08-18T02:40:05.87439567Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:182]: failed to create HCP cluster "cp-ystream-upgrade-4-21-pl2828" with install version 4.20.34 Unexpected error: <fmt.wrapError \| 0xc001408920>: failed to create HCP cluster cp-ystream-upgrade-4-21-pl2828: failed waiting for cluster="cp-ystream-upgrade-4-21-pl2828" in resourcegroup="rg-cp-ystream-upgrade-4-21-2nlwvsvrv9bp" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/8b146388-d74d-42e6-bdaf-8a3edf1b3beb -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/8b146388-d74d-42e6-bdaf-8a3edf1b3beb", "name": "8b146388-d74d-42e6-bdaf-8a3edf1b3beb", "status": "Failed", "startTime": "2026-08-18T02:21:03.015049333Z", "endTime": "2026-08-18T02:40:05.87439567Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift 01:17:15 UTC · failed at e2e · test Customer should be able to create an HCP cluster with custom autoscaling and update the autoscaling configuration · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:108]: failed to create HCP cluster "autoscaling-hcp-cluster" with custom autoscaling Unexpected error: <fmt.wrapError \| 0xc000f62880>: failed to create HCP cluster autoscaling-hcp-cluster: failed waiting for cluster="autoscaling-hcp-cluster" in resourcegroup="autoscaling-cluster-dnvp8bsl7ccf" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/a5af20d3-0412-488c-bbe6-26b573f24f51 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/a5af20d3-0412-488c-bbe6-26b573f24f51", "name": "a5af20d3-0412-488c-bbe6-26b573f24f51", "status": "Failed", "startTime": "2026-08-18T01:34:47.053111478Z", "endTime": "2026-08-18T01:53:54.554086959Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:108]: failed to create HCP cluster "autoscaling-hcp-cluster" with custom autoscaling Unexpected error: <fmt.wrapError \| 0xc000f62880>: failed to create HCP cluster autoscaling-hcp-cluster: failed waiting for cluster="autoscaling-hcp-cluster" in resourcegroup="autoscaling-cluster-dnvp8bsl7ccf" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/a5af20d3-0412-488c-bbe6-26b573f24f51 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/a5af20d3-0412-488c-bbe6-26b573f24f51", "name": "a5af20d3-0412-488c-bbe6-26b573f24f51", "status": "Failed", "startTime": "2026-08-18T01:34:47.053111478Z", "endTime": "2026-08-18T01:53:54.554086959Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift 01:17:15 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 4.19 · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:110]: failed to create HCP cluster "cluster-zstream-4-19-pkvpqg" with version 4.19.41 on candidate channel Unexpected error: <fmt.wrapError \| 0xc000993420>: failed to create HCP cluster cluster-zstream-4-19-pkvpqg: failed waiting for cluster="cluster-zstream-4-19-pkvpqg" in resourcegroup="rg-zstream-upgrade-4-19-xq9w64kvmwvv" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/fd634c01-e142-477e-8a22-f4420d35f59e -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/fd634c01-e142-477e-8a22-f4420d35f59e", "name": "fd634c01-e142-477e-8a22-f4420d35f59e", "status": "Failed", "startTime": "2026-08-18T02:22:35.247158175Z", "endTime": "2026-08-18T02:41:35.737706721Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:110]: failed to create HCP cluster "cluster-zstream-4-19-pkvpqg" with version 4.19.41 on candidate channel Unexpected error: <fmt.wrapError \| 0xc000993420>: failed to create HCP cluster cluster-zstream-4-19-pkvpqg: failed waiting for cluster="cluster-zstream-4-19-pkvpqg" in resourcegroup="rg-zstream-upgrade-4-19-xq9w64kvmwvv" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/fd634c01-e142-477e-8a22-f4420d35f59e -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/fd634c01-e142-477e-8a22-f4420d35f59e", "name": "fd634c01-e142-477e-8a22-f4420d35f59e", "status": "Failed", "startTime": "2026-08-18T02:22:35.247158175Z", "endTime": "2026-08-18T02:41:35.737706721Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift 01:17:15 UTC · failed at e2e · test Authorized CIDRs Connectivity should allow API access only from authorized VM IP · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:111]: failed to create HCP cluster "cidr-connectivity-test" with authorized CIDRs Unexpected error: <fmt.wrapError \| 0xc0010c8b20>: failed to create HCP cluster cidr-connectivity-test: failed waiting for cluster="cidr-connectivity-test" in resourcegroup="e2e-cidr-connectivity-zrwbc572c7q6" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/f7f07a25-6078-4e6c-a5c2-a24eec38065d -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/f7f07a25-6078-4e6c-a5c2-a24eec38065d", "name": "f7f07a25-6078-4e6c-a5c2-a24eec38065d", "status": "Failed", "startTime": "2026-08-18T01:39:50.996749726Z", "endTime": "2026-08-18T01:58:54.431759835Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:111]: failed to create HCP cluster "cidr-connectivity-test" with authorized CIDRs Unexpected error: <fmt.wrapError \| 0xc0010c8b20>: failed to create HCP cluster cidr-connectivity-test: failed waiting for cluster="cidr-connectivity-test" in resourcegroup="e2e-cidr-connectivity-zrwbc572c7q6" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/f7f07a25-6078-4e6c-a5c2-a24eec38065d -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/f7f07a25-6078-4e6c-a5c2-a24eec38065d", "name": "f7f07a25-6078-4e6c-a5c2-a24eec38065d", "status": "Failed", "startTime": "2026-08-18T01:39:50.996749726Z", "endTime": "2026-08-18T01:58:54.431759835Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated; provider Microsoft.RedHatOpenShift 01:17:15 UTC · failed at e2e · test Customer should create a cluster with private KAS and verify API server is only reachable from VNet while ingress remains public · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:90]: failed to create HCP cluster "private-kas" with private KAS Unexpected error: <fmt.wrapError \| 0xc00073f060>: failed to create HCP cluster private-kas: failed waiting for cluster="private-kas" in resourcegroup="private-kas-w7rtqw6zhbhb" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/d58e4803-0126-47e9-845d-d1971bcc6eb1 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/d58e4803-0126-47e9-845d-d1971bcc6eb1", "name": "d58e4803-0126-47e9-845d-d1971bcc6eb1", "status": "Failed", "startTime": "2026-08-18T01:37:08.021690619Z", "endTime": "2026-08-18T01:56:14.403576829Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:90]: failed to create HCP cluster "private-kas" with private KAS Unexpected error: <fmt.wrapError \| 0xc00073f060>: failed to create HCP cluster private-kas: failed waiting for cluster="private-kas" in resourcegroup="private-kas-w7rtqw6zhbhb" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/d58e4803-0126-47e9-845d-d1971bcc6eb1 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/d58e4803-0126-47e9-845d-d1971bcc6eb1", "name": "d58e4803-0126-47e9-845d-d1971bcc6eb1", "status": "Failed", "startTime": "2026-08-18T01:37:08.021690619Z", "endTime": "2026-08-18T01:56:14.403576829Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift 01:17:15 UTC · failed at e2e · test Customer should be able to create an HCP cluster and manage pull secrets · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_pullsecret.go:113]: failed to create HCP cluster for pull secret test Unexpected error: <fmt.wrapError \| 0xc000f16620>: failed to create HCP cluster pullsecret-hcp-cluster: failed waiting for cluster="pullsecret-hcp-cluster" in resourcegroup="pullsecret-test-vdb7ww2x2vhj" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/9682c885-327c-4b57-bf19-db382361e77b -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/9682c885-327c-4b57-bf19-db382361e77b", "name": "9682c885-327c-4b57-bf19-db382361e77b", "status": "Failed", "startTime": "2026-08-18T01:34:15.525892211Z", "endTime": "2026-08-18T01:53:24.490752565Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_pullsecret.go:113]: failed to create HCP cluster for pull secret test Unexpected error: <fmt.wrapError \| 0xc000f16620>: failed to create HCP cluster pullsecret-hcp-cluster: failed waiting for cluster="pullsecret-hcp-cluster" in resourcegroup="pullsecret-test-vdb7ww2x2vhj" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/9682c885-327c-4b57-bf19-db382361e77b -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/9682c885-327c-4b57-bf19-db382361e77b", "name": "9682c885-327c-4b57-bf19-db382361e77b", "status": "Failed", "startTime": "2026-08-18T01:34:15.525892211Z", "endTime": "2026-08-18T01:53:24.490752565Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift 01:17:15 UTC · failed at e2e · test Customer should be able to lifecycle and confirm external auth on a cluster · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_list_and_verify.go:87]: failed to create HCP cluster for external auth list test Unexpected error: <fmt.wrapError \| 0xc00081b4c0>: failed to create HCP cluster ea-listz7drx2: failed waiting for cluster="ea-listz7drx2" in resourcegroup="ea-list-2s5lrmcqdq8j" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/64b6158a-ccf0-4864-a257-301af81fbf00 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/64b6158a-ccf0-4864-a257-301af81fbf00", "name": "64b6158a-ccf0-4864-a257-301af81fbf00", "status": "Failed", "startTime": "2026-08-18T02:22:09.794983918Z", "endTime": "2026-08-18T02:41:15.875400176Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_list_and_verify.go:87]: failed to create HCP cluster for external auth list test Unexpected error: <fmt.wrapError \| 0xc00081b4c0>: failed to create HCP cluster ea-listz7drx2: failed waiting for cluster="ea-listz7drx2" in resourcegroup="ea-list-2s5lrmcqdq8j" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/64b6158a-ccf0-4864-a257-301af81fbf00 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/64b6158a-ccf0-4864-a257-301af81fbf00", "name": "64b6158a-ccf0-4864-a257-301af81fbf00", "status": "Failed", "startTime": "2026-08-18T02:22:09.794983918Z", "endTime": "2026-08-18T02:41:15.875400176Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift 01:17:15 UTC · failed at e2e · test Customer should be able to update node pool labels and taints · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_labels_taints.go:92]: failed to create HCP cluster cluster-np-labels-taints Unexpected error: <fmt.wrapError \| 0xc0009dd040>: failed to create HCP cluster cluster-np-labels-taints: failed waiting for cluster="cluster-np-labels-taints" in resourcegroup="rg-np-labels-taints-b8wmfdpwzr4f" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/0de0c21a-0f71-4144-80e3-410d3cccafab -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/0de0c21a-0f71-4144-80e3-410d3cccafab", "name": "0de0c21a-0f71-4144-80e3-410d3cccafab", "status": "Failed", "startTime": "2026-08-18T02:47:28.122950359Z", "endTime": "2026-08-18T03:06:36.936036065Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_labels_taints.go:92]: failed to create HCP cluster cluster-np-labels-taints Unexpected error: <fmt.wrapError \| 0xc0009dd040>: failed to create HCP cluster cluster-np-labels-taints: failed waiting for cluster="cluster-np-labels-taints" in resourcegroup="rg-np-labels-taints-b8wmfdpwzr4f" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/0de0c21a-0f71-4144-80e3-410d3cccafab -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/0de0c21a-0f71-4144-80e3-410d3cccafab", "name": "0de0c21a-0f71-4144-80e3-410d3cccafab", "status": "Failed", "startTime": "2026-08-18T02:47:28.122950359Z", "endTime": "2026-08-18T03:06:36.936036065Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift 01:17:15 UTC · failed at e2e · test Customer should be able to create a cluster with an external auth config and get the external auth config · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:94]: failed to create HCP cluster for external auth test Unexpected error: <fmt.wrapError \| 0xc00054af40>: failed to create HCP cluster ea-cluster: failed waiting for cluster="ea-cluster" in resourcegroup="external-auth-cluster-vsndz7vcmkc2" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/8c055328-9528-4d95-bf30-f114c8824240 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/8c055328-9528-4d95-bf30-f114c8824240", "name": "8c055328-9528-4d95-bf30-f114c8824240", "status": "Failed", "startTime": "2026-08-18T02:21:25.465764241Z", "endTime": "2026-08-18T02:40:25.755703486Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:94]: failed to create HCP cluster for external auth test Unexpected error: <fmt.wrapError \| 0xc00054af40>: failed to create HCP cluster ea-cluster: failed waiting for cluster="ea-cluster" in resourcegroup="external-auth-cluster-vsndz7vcmkc2" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/8c055328-9528-4d95-bf30-f114c8824240 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/8c055328-9528-4d95-bf30-f114c8824240", "name": "8c055328-9528-4d95-bf30-f114c8824240", "status": "Failed", "startTime": "2026-08-18T02:21:25.465764241Z", "endTime": "2026-08-18T02:40:25.755703486Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift 01:17:15 UTC · failed at e2e · test Customer should be able to create an HCP cluster then delete it by deleting the customer resource group · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delete_cx_rg.go:90]: failed to create HCP cluster "cx-rg-hcp-cluster" Unexpected error: <fmt.wrapError \| 0xc0000ff9e0>: failed to create HCP cluster cx-rg-hcp-cluster: failed waiting for cluster="cx-rg-hcp-cluster" in resourcegroup="cx-rg-cluster-nm7frlj7wx8x" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/6c67b4db-e2e8-4147-ba66-6b76f9ea3698 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/6c67b4db-e2e8-4147-ba66-6b76f9ea3698", "name": "6c67b4db-e2e8-4147-ba66-6b76f9ea3698", "status": "Failed", "startTime": "2026-08-18T01:34:14.569006871Z", "endTime": "2026-08-18T01:53:14.520927967Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delete_cx_rg.go:90]: failed to create HCP cluster "cx-rg-hcp-cluster" Unexpected error: <fmt.wrapError \| 0xc0000ff9e0>: failed to create HCP cluster cx-rg-hcp-cluster: failed waiting for cluster="cx-rg-hcp-cluster" in resourcegroup="cx-rg-cluster-nm7frlj7wx8x" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/6c67b4db-e2e8-4147-ba66-6b76f9ea3698 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/6c67b4db-e2e8-4147-ba66-6b76f9ea3698", "name": "6c67b4db-e2e8-4147-ba66-6b76f9ea3698", "status": "Failed", "startTime": "2026-08-18T01:34:14.569006871Z", "endTime": "2026-08-18T01:53:14.520927967Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift 01:17:15 UTC · failed at e2e · test Customer should downgrade a nodepool version z-stream downgrade from 4.21.zLatest to 4.21.zPrevious · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:593]: failed to create HCP cluster Unexpected error: <fmt.wrapError \| 0xc000f9cd60>: failed to create HCP cluster np-downgrade-mcst4p: failed waiting for cluster="np-downgrade-mcst4p" in resourcegroup="rg-np-downgrade-mcst4p-6f78v8fbdxfl" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/c52fe4a1-d446-434e-ba20-149524aafd7b -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/c52fe4a1-d446-434e-ba20-149524aafd7b", "name": "c52fe4a1-d446-434e-ba20-149524aafd7b", "status": "Failed", "startTime": "2026-08-18T03:07:13.980026707Z", "endTime": "2026-08-18T03:26:17.641036396Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:593]: failed to create HCP cluster Unexpected error: <fmt.wrapError \| 0xc000f9cd60>: failed to create HCP cluster np-downgrade-mcst4p: failed waiting for cluster="np-downgrade-mcst4p" in resourcegroup="rg-np-downgrade-mcst4p-6f78v8fbdxfl" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/c52fe4a1-d446-434e-ba20-149524aafd7b -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/c52fe4a1-d446-434e-ba20-149524aafd7b", "name": "c52fe4a1-d446-434e-ba20-149524aafd7b", "status": "Failed", "startTime": "2026-08-18T03:07:13.980026707Z", "endTime": "2026-08-18T03:26:17.641036396Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated; provider Microsoft.RedHatOpenShift 01:17:15 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 5.0 · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:107]: HCP cluster rg-candidate-5-0-lln7p567vmfx/cluster-candidate-5-0-n9hg2s should provision Unexpected error: <fmt.wrapError \| 0xc000b8a840>: failed to create HCP cluster cluster-candidate-5-0-n9hg2s: failed waiting for cluster="cluster-candidate-5-0-n9hg2s" in resourcegroup="rg-candidate-5-0-lln7p567vmfx" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/9275203c-c388-4fdb-8585-24996057f18a -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/9275203c-c388-4fdb-8585-24996057f18a", "name": "9275203c-c388-4fdb-8585-24996057f18a", "status": "Failed", "startTime": "2026-08-18T02:20:20.646227309Z", "endTime": "2026-08-18T02:39:25.899893485Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:107]: HCP cluster rg-candidate-5-0-lln7p567vmfx/cluster-candidate-5-0-n9hg2s should provision Unexpected error: <fmt.wrapError \| 0xc000b8a840>: failed to create HCP cluster cluster-candidate-5-0-n9hg2s: failed waiting for cluster="cluster-candidate-5-0-n9hg2s" in resourcegroup="rg-candidate-5-0-lln7p567vmfx" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/9275203c-c388-4fdb-8585-24996057f18a -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/9275203c-c388-4fdb-8585-24996057f18a", "name": "9275203c-c388-4fdb-8585-24996057f18a", "status": "Failed", "startTime": "2026-08-18T02:20:20.646227309Z", "endTime": "2026-08-18T02:39:25.899893485Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift 01:17:15 UTC · failed at e2e · test Customer should be able to create node pools with ARM64-based VMs · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/arm64_nodepool.go:88]: failed to create HCP cluster "arm64-vm-hcp-cluster" Unexpected error: <fmt.wrapError \| 0xc000b404a0>: failed to create HCP cluster arm64-vm-hcp-cluster: failed waiting for cluster="arm64-vm-hcp-cluster" in resourcegroup="arm64-vm-cluster-dbtgnnnvcz2z" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/be950d1d-0ef4-4e89-8d67-64c5700365db -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/be950d1d-0ef4-4e89-8d67-64c5700365db", "name": "be950d1d-0ef4-4e89-8d67-64c5700365db", "status": "Failed", "startTime": "2026-08-18T01:34:05.037589846Z", "endTime": "2026-08-18T01:53:14.531114964Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/arm64_nodepool.go:88]: failed to create HCP cluster "arm64-vm-hcp-cluster" Unexpected error: <fmt.wrapError \| 0xc000b404a0>: failed to create HCP cluster arm64-vm-hcp-cluster: failed waiting for cluster="arm64-vm-hcp-cluster" in resourcegroup="arm64-vm-cluster-dbtgnnnvcz2z" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/be950d1d-0ef4-4e89-8d67-64c5700365db -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/be950d1d-0ef4-4e89-8d67-64c5700365db", "name": "be950d1d-0ef4-4e89-8d67-64c5700365db", "status": "Failed", "startTime": "2026-08-18T01:34:05.037589846Z", "endTime": "2026-08-18T01:53:14.531114964Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred Cluster provisioning failed 01:17:15 UTC · failed at e2e · test Customer should be able to test admin credentials before cluster ready, then full admin credential lifecycle · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:158]: Cluster provisioning failed fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:158]: Cluster provisioning failed ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift 01:17:15 UTC · failed at e2e · test Customer should create an HCP cluster and validate TLS certificates · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:96]: failed to create HCP cluster for TLS endpoint test Unexpected error: <fmt.wrapError \| 0xc000e00480>: failed to create HCP cluster tls-endpoint-hcp-cluster: failed waiting for cluster="tls-endpoint-hcp-cluster" in resourcegroup="tls-endpoint-cluster-b8f6jdz5vkr9" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/5529fe79-5551-4598-a548-b1f242876d20 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/5529fe79-5551-4598-a548-b1f242876d20", "name": "5529fe79-5551-4598-a548-b1f242876d20", "status": "Failed", "startTime": "2026-08-18T01:34:54.967119171Z", "endTime": "2026-08-18T01:53:54.734709374Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:96]: failed to create HCP cluster for TLS endpoint test Unexpected error: <fmt.wrapError \| 0xc000e00480>: failed to create HCP cluster tls-endpoint-hcp-cluster: failed waiting for cluster="tls-endpoint-hcp-cluster" in resourcegroup="tls-endpoint-cluster-b8f6jdz5vkr9" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/5529fe79-5551-4598-a548-b1f242876d20 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/5529fe79-5551-4598-a548-b1f242876d20", "name": "5529fe79-5551-4598-a548-b1f242876d20", "status": "Failed", "startTime": "2026-08-18T01:34:54.967119171Z", "endTime": "2026-08-18T01:53:54.734709374Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated; provider Microsoft.RedHatOpenShift 01:17:15 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.21 · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:107]: HCP cluster rg-candidate-4-21-xh9qrb947nb7/cluster-candidate-4-21-lvrbs2 should provision Unexpected error: <fmt.wrapError \| 0xc001046ee0>: failed to create HCP cluster cluster-candidate-4-21-lvrbs2: failed waiting for cluster="cluster-candidate-4-21-lvrbs2" in resourcegroup="rg-candidate-4-21-xh9qrb947nb7" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/b434160c-effb-4cc2-a595-1bb2db1f60ca -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/b434160c-effb-4cc2-a595-1bb2db1f60ca", "name": "b434160c-effb-4cc2-a595-1bb2db1f60ca", "status": "Failed", "startTime": "2026-08-18T02:20:00.98447986Z", "endTime": "2026-08-18T02:39:05.838171548Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:107]: HCP cluster rg-candidate-4-21-xh9qrb947nb7/cluster-candidate-4-21-lvrbs2 should provision Unexpected error: <fmt.wrapError \| 0xc001046ee0>: failed to create HCP cluster cluster-candidate-4-21-lvrbs2: failed waiting for cluster="cluster-candidate-4-21-lvrbs2" in resourcegroup="rg-candidate-4-21-xh9qrb947nb7" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/b434160c-effb-4cc2-a595-1bb2db1f60ca -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/b434160c-effb-4cc2-a595-1bb2db1f60ca", "name": "b434160c-effb-4cc2-a595-1bb2db1f60ca", "status": "Failed", "startTime": "2026-08-18T02:20:00.98447986Z", "endTime": "2026-08-18T02:39:05.838171548Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift 01:17:15 UTC · failed at e2e · test Customer should be able to create a HCP cluster and use cilium CNI plugin · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_cni_cilium.go:88]: failed to create HCP cluster "cilium-cl" without CNI Unexpected error: <fmt.wrapError \| 0xc0013b4180>: failed to create HCP cluster cilium-cl: failed waiting for cluster="cilium-cl" in resourcegroup="cni-cilium-lglb9vrqn8kz" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/32ec6679-a8ba-4e88-aac1-83469c53918c -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/32ec6679-a8ba-4e88-aac1-83469c53918c", "name": "32ec6679-a8ba-4e88-aac1-83469c53918c", "status": "Failed", "startTime": "2026-08-18T01:34:05.638284549Z", "endTime": "2026-08-18T01:53:14.514295274Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_cni_cilium.go:88]: failed to create HCP cluster "cilium-cl" without CNI Unexpected error: <fmt.wrapError \| 0xc0013b4180>: failed to create HCP cluster cilium-cl: failed waiting for cluster="cilium-cl" in resourcegroup="cni-cilium-lglb9vrqn8kz" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/32ec6679-a8ba-4e88-aac1-83469c53918c -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/32ec6679-a8ba-4e88-aac1-83469c53918c", "name": "32ec6679-a8ba-4e88-aac1-83469c53918c", "status": "Failed", "startTime": "2026-08-18T01:34:05.638284549Z", "endTime": "2026-08-18T01:53:14.514295274Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift 01:17:15 UTC · failed at e2e · test Customer should upgrade a nodepool skipping one minor version (+2) from 4.20.z to 4.22.zLatest · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:462]: failed to create HCP cluster np-skip-minor-4l8vkx at version 4.22.10 Unexpected error: <fmt.wrapError \| 0xc000f46b80>: failed to create HCP cluster np-skip-minor-4l8vkx: failed waiting for cluster="np-skip-minor-4l8vkx" in resourcegroup="rg-np-skip-minor-4l8vkx-tqb64hrlx82j" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/726ca7c3-5ea4-4a42-9377-15144aa74b54 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/726ca7c3-5ea4-4a42-9377-15144aa74b54", "name": "726ca7c3-5ea4-4a42-9377-15144aa74b54", "status": "Failed", "startTime": "2026-08-18T03:07:14.300447528Z", "endTime": "2026-08-18T03:26:17.639458601Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:462]: failed to create HCP cluster np-skip-minor-4l8vkx at version 4.22.10 Unexpected error: <fmt.wrapError \| 0xc000f46b80>: failed to create HCP cluster np-skip-minor-4l8vkx: failed waiting for cluster="np-skip-minor-4l8vkx" in resourcegroup="rg-np-skip-minor-4l8vkx-tqb64hrlx82j" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/726ca7c3-5ea4-4a42-9377-15144aa74b54 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/726ca7c3-5ea4-4a42-9377-15144aa74b54", "name": "726ca7c3-5ea4-4a42-9377-15144aa74b54", "status": "Failed", "startTime": "2026-08-18T03:07:14.300447528Z", "endTime": "2026-08-18T03:26:17.639458601Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift 01:17:15 UTC · failed at e2e · test Customer should upgrade a nodepool to a version without Cincinnati upgrade edge z-stream upgrade without Cincinnati edge in 4.20 · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:314]: failed to create HCP cluster np-noedge-g5fc5s Unexpected error: <fmt.wrapError \| 0xc00084e1e0>: failed to create HCP cluster np-noedge-g5fc5s: failed waiting for cluster="np-noedge-g5fc5s" in resourcegroup="rg-np-noedge-g5fc5s-8bfh6hb267dw" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/94bbeecd-3423-4010-8314-ff8e53a38982 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/94bbeecd-3423-4010-8314-ff8e53a38982", "name": "94bbeecd-3423-4010-8314-ff8e53a38982", "status": "Failed", "startTime": "2026-08-18T03:22:09.446229949Z", "endTime": "2026-08-18T03:41:17.713501122Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:314]: failed to create HCP cluster np-noedge-g5fc5s Unexpected error: <fmt.wrapError \| 0xc00084e1e0>: failed to create HCP cluster np-noedge-g5fc5s: failed waiting for cluster="np-noedge-g5fc5s" in resourcegroup="rg-np-noedge-g5fc5s-8bfh6hb267dw" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/94bbeecd-3423-4010-8314-ff8e53a38982 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/94bbeecd-3423-4010-8314-ff8e53a38982", "name": "94bbeecd-3423-4010-8314-ff8e53a38982", "status": "Failed", "startTime": "2026-08-18T03:22:09.446229949Z", "endTime": "2026-08-18T03:41:17.713501122Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift 01:17:15 UTC · failed at e2e · test Customer should be able to forward control plane logs to a storage account and Event Hub via shoebox diagnostic settings · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/shoebox_logs.go:270]: failed to create HCP cluster shoebox-hcp-cluster Unexpected error: <fmt.wrapError \| 0xc001580820>: failed to create HCP cluster shoebox-hcp-cluster: failed waiting for cluster="shoebox-hcp-cluster" in resourcegroup="shoebox-cluster-n9kkzn45w49n" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/2bc60d2b-0fbd-4ed7-895c-5b2d6486be40 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/2bc60d2b-0fbd-4ed7-895c-5b2d6486be40", "name": "2bc60d2b-0fbd-4ed7-895c-5b2d6486be40", "status": "Failed", "startTime": "2026-08-18T03:06:49.439367376Z", "endTime": "2026-08-18T03:25:57.638155323Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/shoebox_logs.go:270]: failed to create HCP cluster shoebox-hcp-cluster Unexpected error: <fmt.wrapError \| 0xc001580820>: failed to create HCP cluster shoebox-hcp-cluster: failed waiting for cluster="shoebox-hcp-cluster" in resourcegroup="shoebox-cluster-n9kkzn45w49n" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/2bc60d2b-0fbd-4ed7-895c-5b2d6486be40 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/2bc60d2b-0fbd-4ed7-895c-5b2d6486be40", "name": "2bc60d2b-0fbd-4ed7-895c-5b2d6486be40", "status": "Failed", "startTime": "2026-08-18T03:06:49.439367376Z", "endTime": "2026-08-18T03:25:57.638155323Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift 01:17:15 UTC · failed at e2e · test Customer should be able to create an HCP cluster and custom node pool osDisk size · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_nodepool_osdisk.go:80]: failed to create HCP cluster "hcp-cluster-np-128" Unexpected error: <fmt.wrapError \| 0xc001592820>: failed to create HCP cluster hcp-cluster-np-128: failed waiting for cluster="hcp-cluster-np-128" in resourcegroup="clusternp128-nr7gpbmjfgdn" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/8a0dbd68-0dcc-4587-a43f-63ee3a680700 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/8a0dbd68-0dcc-4587-a43f-63ee3a680700", "name": "8a0dbd68-0dcc-4587-a43f-63ee3a680700", "status": "Failed", "startTime": "2026-08-18T01:34:45.799991077Z", "endTime": "2026-08-18T01:53:54.557961621Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_nodepool_osdisk.go:80]: failed to create HCP cluster "hcp-cluster-np-128" Unexpected error: <fmt.wrapError \| 0xc001592820>: failed to create HCP cluster hcp-cluster-np-128: failed waiting for cluster="hcp-cluster-np-128" in resourcegroup="clusternp128-nr7gpbmjfgdn" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/8a0dbd68-0dcc-4587-a43f-63ee3a680700 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/8a0dbd68-0dcc-4587-a43f-63ee3a680700", "name": "8a0dbd68-0dcc-4587-a43f-63ee3a680700", "status": "Failed", "startTime": "2026-08-18T01:34:45.799991077Z", "endTime": "2026-08-18T01:53:54.557961621Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift 01:17:15 UTC · failed at e2e · test Update HCPOpenShiftCluster Negative creates a cluster and fails to update its name with a PATCH request · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:84]: failed to create HCP cluster for patch-name test Unexpected error: <fmt.wrapError \| 0xc000b7ad20>: failed to create HCP cluster patch-name-cluster: failed waiting for cluster="patch-name-cluster" in resourcegroup="patch-name-6jdjkfkzjj4q" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/84cf09bc-9ccf-4c52-ad0b-3215eecddc43 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/84cf09bc-9ccf-4c52-ad0b-3215eecddc43", "name": "84cf09bc-9ccf-4c52-ad0b-3215eecddc43", "status": "Failed", "startTime": "2026-08-18T01:35:14.689867232Z", "endTime": "2026-08-18T01:54:14.533556615Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:84]: failed to create HCP cluster for patch-name test Unexpected error: <fmt.wrapError \| 0xc000b7ad20>: failed to create HCP cluster patch-name-cluster: failed waiting for cluster="patch-name-cluster" in resourcegroup="patch-name-6jdjkfkzjj4q" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/84cf09bc-9ccf-4c52-ad0b-3215eecddc43 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/84cf09bc-9ccf-4c52-ad0b-3215eecddc43", "name": "84cf09bc-9ccf-4c52-ad0b-3215eecddc43", "status": "Failed", "startTime": "2026-08-18T01:35:14.689867232Z", "endTime": "2026-08-18T01:54:14.533556615Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated; provider Microsoft.RedHatOpenShift 01:17:15 UTC · failed at e2e · test Customer should be able to create an HCP cluster and manage ImageDigestMirrors · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/idms_lifecycle.go:113]: failed to create HCP cluster with ImageDigestMirrors Unexpected error: <fmt.wrapError \| 0xc000a94b00>: failed to create HCP cluster idms-e2e-hcp-cluster: failed waiting for cluster="idms-e2e-hcp-cluster" in resourcegroup="idms-rl7558nqx6x5" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/6c83ecdf-321d-4c22-8c0e-0e67dfc51b95 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/6c83ecdf-321d-4c22-8c0e-0e67dfc51b95", "name": "6c83ecdf-321d-4c22-8c0e-0e67dfc51b95", "status": "Failed", "startTime": "2026-08-18T02:22:38.138134816Z", "endTime": "2026-08-18T02:41:45.799493023Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/idms_lifecycle.go:113]: failed to create HCP cluster with ImageDigestMirrors Unexpected error: <fmt.wrapError \| 0xc000a94b00>: failed to create HCP cluster idms-e2e-hcp-cluster: failed waiting for cluster="idms-e2e-hcp-cluster" in resourcegroup="idms-rl7558nqx6x5" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/6c83ecdf-321d-4c22-8c0e-0e67dfc51b95 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/6c83ecdf-321d-4c22-8c0e-0e67dfc51b95", "name": "6c83ecdf-321d-4c22-8c0e-0e67dfc51b95", "status": "Failed", "startTime": "2026-08-18T02:22:38.138134816Z", "endTime": "2026-08-18T02:41:45.799493023Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift 01:17:15 UTC · failed at e2e · test Customer should be able to use workload identity via the cluster OIDC issuer URL · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/oidc_issuer_workload_identity.go:217]: failed to create HCP cluster oidc-wi-cluster Unexpected error: <fmt.wrapError \| 0xc001454fa0>: failed to create HCP cluster oidc-wi-cluster: failed waiting for cluster="oidc-wi-cluster" in resourcegroup="oidc-wi-rlpxm5hnr4qk" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/9204c2aa-afaa-4aec-929c-673c0f6e633e -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/9204c2aa-afaa-4aec-929c-673c0f6e633e", "name": "9204c2aa-afaa-4aec-929c-673c0f6e633e", "status": "Failed", "startTime": "2026-08-18T03:06:29.321278577Z", "endTime": "2026-08-18T03:25:37.629557591Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/oidc_issuer_workload_identity.go:217]: failed to create HCP cluster oidc-wi-cluster Unexpected error: <fmt.wrapError \| 0xc001454fa0>: failed to create HCP cluster oidc-wi-cluster: failed waiting for cluster="oidc-wi-cluster" in resourcegroup="oidc-wi-rlpxm5hnr4qk" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/9204c2aa-afaa-4aec-929c-673c0f6e633e -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/9204c2aa-afaa-4aec-929c-673c0f6e633e", "name": "9204c2aa-afaa-4aec-929c-673c0f6e633e", "status": "Failed", "startTime": "2026-08-18T03:06:29.321278577Z", "endTime": "2026-08-18T03:25:37.629557591Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift 01:17:15 UTC · failed at e2e · test Customer should not be able to perform various invalid operations on cluster resources · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/simple_negative_cases.go:110]: failed to create HCP cluster negative-tests-cluster Unexpected error: <fmt.wrapError \| 0xc000881e40>: failed to create HCP cluster negative-tests-cluster: failed waiting for cluster="negative-tests-cluster" in resourcegroup="rg-negative-tests-55jnpgdk2xqq" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/d2316f39-ccb1-485c-bc80-ad87973b6715 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/d2316f39-ccb1-485c-bc80-ad87973b6715", "name": "d2316f39-ccb1-485c-bc80-ad87973b6715", "status": "Failed", "startTime": "2026-08-18T03:07:26.053560789Z", "endTime": "2026-08-18T03:26:27.648974782Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/simple_negative_cases.go:110]: failed to create HCP cluster negative-tests-cluster Unexpected error: <fmt.wrapError \| 0xc000881e40>: failed to create HCP cluster negative-tests-cluster: failed waiting for cluster="negative-tests-cluster" in resourcegroup="rg-negative-tests-55jnpgdk2xqq" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/d2316f39-ccb1-485c-bc80-ad87973b6715 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/d2316f39-ccb1-485c-bc80-ad87973b6715", "name": "d2316f39-ccb1-485c-bc80-ad87973b6715", "status": "Failed", "startTime": "2026-08-18T03:07:26.053560789Z", "endTime": "2026-08-18T03:26:27.648974782Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred timeout during CreateHCPClusterAndWait; context deadline exceeded 01:17:15 UTC · failed at e2e · test Customer should be able to create a no-CNI private cluster with a private key vault, a nodepool and install cilium CNI successfully · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:106]: failed to create HCP cluster "cilium-cluster" with no CNI and private etcd Unexpected error: <fmt.wrapErrors \| 0xc00127f380>: failed waiting for cluster="cilium-cluster" in resourcegroup="complex-cilium-kv-nz8jdkqt97lt" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223AndWait for cluster cilium-cluster in resource group complex-cilium-kv-nz8jdkqt97lt, error: context deadline exceeded ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:106]: failed to create HCP cluster "cilium-cluster" with no CNI and private etcd Unexpected error: <fmt.wrapErrors \| 0xc00127f380>: failed waiting for cluster="cilium-cluster" in resourcegroup="complex-cilium-kv-nz8jdkqt97lt" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223AndWait for cluster cilium-cluster in resource group complex-cilium-kv-nz8jdkqt97lt, error: context deadline exceeded ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated; provider Microsoft.RedHatOpenShift 01:17:15 UTC · failed at e2e · test Create HCPOpenShiftCluster with Private KeyVault should create a cluster with private keyvault using v20251223preview API · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:98]: failed to create HCP cluster "private-kv-cluster" with private keyvault Unexpected error: <fmt.wrapError \| 0xc0008c74a0>: failed waiting for cluster="private-kv-cluster" in resourcegroup="private-keyvault-klf4n4fvlz26" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/d5fa142d-cb72-43e2-a5ac-bdaf2c8ab514 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/d5fa142d-cb72-43e2-a5ac-bdaf2c8ab514", "name": "d5fa142d-cb72-43e2-a5ac-bdaf2c8ab514", "status": "Failed", "startTime": "2026-08-18T01:35:55.986111403Z", "endTime": "2026-08-18T01:55:04.389818569Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:98]: failed to create HCP cluster "private-kv-cluster" with private keyvault Unexpected error: <fmt.wrapError \| 0xc0008c74a0>: failed waiting for cluster="private-kv-cluster" in resourcegroup="private-keyvault-klf4n4fvlz26" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/d5fa142d-cb72-43e2-a5ac-bdaf2c8ab514 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/d5fa142d-cb72-43e2-a5ac-bdaf2c8ab514", "name": "d5fa142d-cb72-43e2-a5ac-bdaf2c8ab514", "status": "Failed", "startTime": "2026-08-18T01:35:55.986111403Z", "endTime": "2026-08-18T01:55:04.389818569Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift 01:17:15 UTC · failed at e2e · test Nodepool Ephemeral OS Disk should create a nodepool with ephemeral OS disk when autoRepair is enabled · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_ephemeral_osdisk.go:98]: failed to create HCP cluster ephemeral-disk Unexpected error: <fmt.wrapError \| 0xc000a90600>: failed to create HCP cluster ephemeral-disk: failed waiting for cluster="ephemeral-disk" in resourcegroup="ephemeral-osdisk-dskdjvfx5j7t" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/0b9c0bcd-a92a-43db-a991-ea9328e1f923 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/0b9c0bcd-a92a-43db-a991-ea9328e1f923", "name": "0b9c0bcd-a92a-43db-a991-ea9328e1f923", "status": "Failed", "startTime": "2026-08-18T02:25:33.035856103Z", "endTime": "2026-08-18T02:44:36.584407006Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_ephemeral_osdisk.go:98]: failed to create HCP cluster ephemeral-disk Unexpected error: <fmt.wrapError \| 0xc000a90600>: failed to create HCP cluster ephemeral-disk: failed waiting for cluster="ephemeral-disk" in resourcegroup="ephemeral-osdisk-dskdjvfx5j7t" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/0b9c0bcd-a92a-43db-a991-ea9328e1f923 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/0b9c0bcd-a92a-43db-a991-ea9328e1f923", "name": "0b9c0bcd-a92a-43db-a991-ea9328e1f923", "status": "Failed", "startTime": "2026-08-18T02:25:33.035856103Z", "endTime": "2026-08-18T02:44:36.584407006Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift 01:17:15 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 4.20 · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:110]: failed to create HCP cluster "cluster-zstream-4-20-jrsvwm" with version 4.20.33 on candidate channel Unexpected error: <fmt.wrapError \| 0xc000fe6440>: failed to create HCP cluster cluster-zstream-4-20-jrsvwm: failed waiting for cluster="cluster-zstream-4-20-jrsvwm" in resourcegroup="rg-zstream-upgrade-4-20-qpmcvtcvj9dv" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/1687c7d6-bf31-410c-8390-111504ab636c -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/1687c7d6-bf31-410c-8390-111504ab636c", "name": "1687c7d6-bf31-410c-8390-111504ab636c", "status": "Failed", "startTime": "2026-08-18T02:22:59.495869545Z", "endTime": "2026-08-18T02:42:05.746097106Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:110]: failed to create HCP cluster "cluster-zstream-4-20-jrsvwm" with version 4.20.33 on candidate channel Unexpected error: <fmt.wrapError \| 0xc000fe6440>: failed to create HCP cluster cluster-zstream-4-20-jrsvwm: failed waiting for cluster="cluster-zstream-4-20-jrsvwm" in resourcegroup="rg-zstream-upgrade-4-20-qpmcvtcvj9dv" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/1687c7d6-bf31-410c-8390-111504ab636c -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/1687c7d6-bf31-410c-8390-111504ab636c", "name": "1687c7d6-bf31-410c-8390-111504ab636c", "status": "Failed", "startTime": "2026-08-18T02:22:59.495869545Z", "endTime": "2026-08-18T02:42:05.746097106Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift 01:17:15 UTC · failed at e2e · test Customer should not be able to reuse subnets and NSGs between clusters · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_nsg_subnet_reuse.go:107]: failed to create seed HCP cluster basic-cluster Unexpected error: <fmt.wrapError \| 0xc0008b94a0>: failed to create HCP cluster basic-cluster: failed waiting for cluster="basic-cluster" in resourcegroup="rg-cluster-nsg-subnet-reuse-t67ltk24vhhl" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/4b27045c-87f0-45f9-bb70-024ec2e52cbe -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/4b27045c-87f0-45f9-bb70-024ec2e52cbe", "name": "4b27045c-87f0-45f9-bb70-024ec2e52cbe", "status": "Failed", "startTime": "2026-08-18T01:35:16.063344861Z", "endTime": "2026-08-18T01:54:24.672728151Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_nsg_subnet_reuse.go:107]: failed to create seed HCP cluster basic-cluster Unexpected error: <fmt.wrapError \| 0xc0008b94a0>: failed to create HCP cluster basic-cluster: failed waiting for cluster="basic-cluster" in resourcegroup="rg-cluster-nsg-subnet-reuse-t67ltk24vhhl" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/4b27045c-87f0-45f9-bb70-024ec2e52cbe -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/4b27045c-87f0-45f9-bb70-024ec2e52cbe", "name": "4b27045c-87f0-45f9-bb70-024ec2e52cbe", "status": "Failed", "startTime": "2026-08-18T01:35:16.063344861Z", "endTime": "2026-08-18T01:54:24.672728151Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated; provider Microsoft.RedHatOpenShift 01:17:15 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.20 · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:107]: HCP cluster rg-candidate-4-20-hgtq5vkjmtng/cluster-candidate-4-20-7ntjrh should provision Unexpected error: <fmt.wrapError \| 0xc0005ac5a0>: failed to create HCP cluster cluster-candidate-4-20-7ntjrh: failed waiting for cluster="cluster-candidate-4-20-7ntjrh" in resourcegroup="rg-candidate-4-20-hgtq5vkjmtng" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/7498cc37-bb15-4954-891a-544689c57fd9 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/7498cc37-bb15-4954-891a-544689c57fd9", "name": "7498cc37-bb15-4954-891a-544689c57fd9", "status": "Failed", "startTime": "2026-08-18T02:20:22.991368241Z", "endTime": "2026-08-18T02:39:25.761048408Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:107]: HCP cluster rg-candidate-4-20-hgtq5vkjmtng/cluster-candidate-4-20-7ntjrh should provision Unexpected error: <fmt.wrapError \| 0xc0005ac5a0>: failed to create HCP cluster cluster-candidate-4-20-7ntjrh: failed waiting for cluster="cluster-candidate-4-20-7ntjrh" in resourcegroup="rg-candidate-4-20-hgtq5vkjmtng" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/7498cc37-bb15-4954-891a-544689c57fd9 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/7498cc37-bb15-4954-891a-544689c57fd9", "name": "7498cc37-bb15-4954-891a-544689c57fd9", "status": "Failed", "startTime": "2026-08-18T02:20:22.991368241Z", "endTime": "2026-08-18T02:39:25.761048408Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift 01:17:15 UTC · failed at e2e · test Customer should upgrade and update a nodepool from 4.20.z to 4.21.zLatest · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:133]: failed to create HCP cluster np-version-upgrade-cluster-ktwcf4 with version 4.21.29 Unexpected error: <fmt.wrapError \| 0xc001088e20>: failed to create HCP cluster np-version-upgrade-cluster-ktwcf4: failed waiting for cluster="np-version-upgrade-cluster-ktwcf4" in resourcegroup="rg-np-version-upgrade-ktwcf4-k458jgrddqcq" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/b4c0e4e5-8627-4490-88d5-2632a2a2f8fc -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/b4c0e4e5-8627-4490-88d5-2632a2a2f8fc", "name": "b4c0e4e5-8627-4490-88d5-2632a2a2f8fc", "status": "Failed", "startTime": "2026-08-18T03:05:31.516468484Z", "endTime": "2026-08-18T03:24:37.697618815Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:133]: failed to create HCP cluster np-version-upgrade-cluster-ktwcf4 with version 4.21.29 Unexpected error: <fmt.wrapError \| 0xc001088e20>: failed to create HCP cluster np-version-upgrade-cluster-ktwcf4: failed waiting for cluster="np-version-upgrade-cluster-ktwcf4" in resourcegroup="rg-np-version-upgrade-ktwcf4-k458jgrddqcq" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/b4c0e4e5-8627-4490-88d5-2632a2a2f8fc -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/b4c0e4e5-8627-4490-88d5-2632a2a2f8fc", "name": "b4c0e4e5-8627-4490-88d5-2632a2a2f8fc", "status": "Failed", "startTime": "2026-08-18T03:05:31.516468484Z", "endTime": "2026-08-18T03:24:37.697618815Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred cluster entered terminal Failed state after role assignment deployment 01:17:15 UTC · failed at e2e · test ARO HCP Service should successfully provision cluster when MI role assignments are added after cluster creation · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:220]: Timed out after 2520.000s. cluster should eventually succeed after role assignments are created The function passed to Eventually failed at /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:216 with: cluster entered terminal Failed state after role assignment deployment Expected <armredhatopenshifthcp.ProvisioningState>: ... not to equal <armredhatopenshifthcp.ProvisioningState>: ... fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:220]: Timed out after 2520.000s. cluster should eventually succeed after role assignments are created The function passed to Eventually failed at /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:216 with: cluster entered terminal Failed state after role assignment deployment Expected <armredhatopenshifthcp.ProvisioningState>: ... not to equal <armredhatopenshifthcp.ProvisioningState>: ... ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated; provider Microsoft.RedHatOpenShift 01:17:15 UTC · failed at e2e · test Customer should create a cluster with private ingress using v20260630preview and verify the ingress is internal · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_ingress.go:103]: failed to create HCP cluster "private-ingress" with private ingress Unexpected error: <fmt.wrapError \| 0xc0008bdf60>: failed to create HCP cluster private-ingress: failed waiting for cluster="private-ingress" in resourcegroup="private-ingress-nctgpthtz9f4" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/af1fee24-3049-47ef-b920-97189cf8fcee -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/af1fee24-3049-47ef-b920-97189cf8fcee", "name": "af1fee24-3049-47ef-b920-97189cf8fcee", "status": "Failed", "startTime": "2026-08-18T01:34:08.580882055Z", "endTime": "2026-08-18T01:53:14.490660293Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_ingress.go:103]: failed to create HCP cluster "private-ingress" with private ingress Unexpected error: <fmt.wrapError \| 0xc0008bdf60>: failed to create HCP cluster private-ingress: failed waiting for cluster="private-ingress" in resourcegroup="private-ingress-nctgpthtz9f4" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/af1fee24-3049-47ef-b920-97189cf8fcee -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/af1fee24-3049-47ef-b920-97189cf8fcee", "name": "af1fee24-3049-47ef-b920-97189cf8fcee", "status": "Failed", "startTime": "2026-08-18T01:34:08.580882055Z", "endTime": "2026-08-18T01:53:14.490660293Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift 01:17:15 UTC · failed at e2e · test Customer should downgrade a nodepool to a lower minor version from 4.21.zLatest to 4.19.zLatest · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:741]: failed to create HCP cluster Unexpected error: <fmt.wrapError \| 0xc000facbc0>: failed to create HCP cluster np-dg-minor-szpgrq: failed waiting for cluster="np-dg-minor-szpgrq" in resourcegroup="rg-np-dg-minor-szpgrq-t9m426rffsgp" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/3db25e9a-b224-465f-9d23-7922b114d810 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/3db25e9a-b224-465f-9d23-7922b114d810", "name": "3db25e9a-b224-465f-9d23-7922b114d810", "status": "Failed", "startTime": "2026-08-18T03:06:49.429771196Z", "endTime": "2026-08-18T03:25:57.711979405Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:741]: failed to create HCP cluster Unexpected error: <fmt.wrapError \| 0xc000facbc0>: failed to create HCP cluster np-dg-minor-szpgrq: failed waiting for cluster="np-dg-minor-szpgrq" in resourcegroup="rg-np-dg-minor-szpgrq-t9m426rffsgp" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/3db25e9a-b224-465f-9d23-7922b114d810 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/3db25e9a-b224-465f-9d23-7922b114d810", "name": "3db25e9a-b224-465f-9d23-7922b114d810", "status": "Failed", "startTime": "2026-08-18T03:06:49.429771196Z", "endTime": "2026-08-18T03:25:57.711979405Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred timeout during CreateHCPClusterAndWait; context deadline exceeded 01:17:15 UTC · failed at e2e · test Customer should be able to create an HCP cluster with back-level version 4.19 · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:151]: failed to create HCP cluster version 4.19 Unexpected error: <fmt.wrapErrors \| 0xc0013d8240>: failed waiting for cluster="cluster-ver-4-19" in resourcegroup="rg-cluster-back-version-qfcrpn6hrh9n" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterAndWait for cluster cluster-ver-4-19 in resource group rg-cluster-back-version-qfcrpn6hrh9n, error: context deadline exceeded ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:151]: failed to create HCP cluster version 4.19 Unexpected error: <fmt.wrapErrors \| 0xc0013d8240>: failed waiting for cluster="cluster-ver-4-19" in resourcegroup="rg-cluster-back-version-qfcrpn6hrh9n" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterAndWait for cluster cluster-ver-4-19 in resource group rg-cluster-back-version-qfcrpn6hrh9n, error: context deadline exceeded ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift 01:17:15 UTC · failed at e2e · test Customer should be able to create several HCP clusters in their customer resource group, but not in the same managed resource group · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:131]: failed to wait for first cluster "basic-hcp-cluster" to complete creation (timeout '20.000000' minutes) Unexpected error: <exported.ResponseError \| 0xc00074cf00>: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/14b9d201-ed20-4b0a-9a6e-e010ba8f28af -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/14b9d201-ed20-4b0a-9a6e-e010ba8f28af", "name": "14b9d201-ed20-4b0a-9a6e-e010ba8f28af", "status": "Failed", "startTime": "2026-08-18T01:35:11.854338314Z", "endTime": "2026-08-18T01:54:14.528632911Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:131]: failed to wait for first cluster "basic-hcp-cluster" to complete creation (timeout '20.000000' minutes) Unexpected error: <exported.ResponseError \| 0xc00074cf00>: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/14b9d201-ed20-4b0a-9a6e-e010ba8f28af -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/14b9d201-ed20-4b0a-9a6e-e010ba8f28af", "name": "14b9d201-ed20-4b0a-9a6e-e010ba8f28af", "status": "Failed", "startTime": "2026-08-18T01:35:11.854338314Z", "endTime": "2026-08-18T01:54:14.528632911Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift 01:17:15 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 4.21 · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:110]: failed to create HCP cluster "cluster-zstream-4-21-4rw7qw" with version 4.21.28 on candidate channel Unexpected error: <fmt.wrapError \| 0xc000e09420>: failed to create HCP cluster cluster-zstream-4-21-4rw7qw: failed waiting for cluster="cluster-zstream-4-21-4rw7qw" in resourcegroup="rg-zstream-upgrade-4-21-6st9sr5lfd6j" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/328eb91e-47ee-4dae-98a5-c5c57ddda57d -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/328eb91e-47ee-4dae-98a5-c5c57ddda57d", "name": "328eb91e-47ee-4dae-98a5-c5c57ddda57d", "status": "Failed", "startTime": "2026-08-18T02:20:42.085524691Z", "endTime": "2026-08-18T02:39:45.847398158Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:110]: failed to create HCP cluster "cluster-zstream-4-21-4rw7qw" with version 4.21.28 on candidate channel Unexpected error: <fmt.wrapError \| 0xc000e09420>: failed to create HCP cluster cluster-zstream-4-21-4rw7qw: failed waiting for cluster="cluster-zstream-4-21-4rw7qw" in resourcegroup="rg-zstream-upgrade-4-21-6st9sr5lfd6j" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/328eb91e-47ee-4dae-98a5-c5c57ddda57d -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/328eb91e-47ee-4dae-98a5-c5c57ddda57d", "name": "328eb91e-47ee-4dae-98a5-c5c57ddda57d", "status": "Failed", "startTime": "2026-08-18T02:20:42.085524691Z", "endTime": "2026-08-18T02:39:45.847398158Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift 01:17:15 UTC · failed at e2e · test Customer should be able to successfully upgrade control plane minor version from 4.21 minor to 4.22 minor · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:182]: failed to create HCP cluster "cp-ystream-upgrade-4-22-h482f9" with install version 4.21.28 Unexpected error: <fmt.wrapError \| 0xc0004aca80>: failed to create HCP cluster cp-ystream-upgrade-4-22-h482f9: failed waiting for cluster="cp-ystream-upgrade-4-22-h482f9" in resourcegroup="rg-cp-ystream-upgrade-4-22-cns7sgjvqbjf" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/12192dc8-297c-457e-9c26-cdda4e94430f -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/12192dc8-297c-457e-9c26-cdda4e94430f", "name": "12192dc8-297c-457e-9c26-cdda4e94430f", "status": "Failed", "startTime": "2026-08-18T02:21:25.103882504Z", "endTime": "2026-08-18T02:40:25.795531616Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:182]: failed to create HCP cluster "cp-ystream-upgrade-4-22-h482f9" with install version 4.21.28 Unexpected error: <fmt.wrapError \| 0xc0004aca80>: failed to create HCP cluster cp-ystream-upgrade-4-22-h482f9: failed waiting for cluster="cp-ystream-upgrade-4-22-h482f9" in resourcegroup="rg-cp-ystream-upgrade-4-22-cns7sgjvqbjf" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/12192dc8-297c-457e-9c26-cdda4e94430f -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/12192dc8-297c-457e-9c26-cdda4e94430f", "name": "12192dc8-297c-457e-9c26-cdda4e94430f", "status": "Failed", "startTime": "2026-08-18T02:21:25.103882504Z", "endTime": "2026-08-18T02:40:25.795531616Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift 01:17:15 UTC · failed at e2e · test Customer should upgrade and update a nodepool from 4.21.z to 4.21.zLatest · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:133]: failed to create HCP cluster np-version-upgrade-cluster-phxfbm with version 4.21.29 Unexpected error: <fmt.wrapError \| 0xc001092ce0>: failed to create HCP cluster np-version-upgrade-cluster-phxfbm: failed waiting for cluster="np-version-upgrade-cluster-phxfbm" in resourcegroup="rg-np-version-upgrade-phxfbm-v96mg7kkv48z" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/3af2530c-5f72-43c5-8c8d-ed0a7e8448bd -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/3af2530c-5f72-43c5-8c8d-ed0a7e8448bd", "name": "3af2530c-5f72-43c5-8c8d-ed0a7e8448bd", "status": "Failed", "startTime": "2026-08-18T03:05:52.501093606Z", "endTime": "2026-08-18T03:24:57.820442571Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:133]: failed to create HCP cluster np-version-upgrade-cluster-phxfbm with version 4.21.29 Unexpected error: <fmt.wrapError \| 0xc001092ce0>: failed to create HCP cluster np-version-upgrade-cluster-phxfbm: failed waiting for cluster="np-version-upgrade-cluster-phxfbm" in resourcegroup="rg-np-version-upgrade-phxfbm-v96mg7kkv48z" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/3af2530c-5f72-43c5-8c8d-ed0a7e8448bd -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/3af2530c-5f72-43c5-8c8d-ed0a7e8448bd", "name": "3af2530c-5f72-43c5-8c8d-ed0a7e8448bd", "status": "Failed", "startTime": "2026-08-18T03:05:52.501093606Z", "endTime": "2026-08-18T03:24:57.820442571Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift 01:17:15 UTC · failed at e2e · test Customer should upgrade and update a nodepool from 4.20.z to 4.20.zLatest · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:133]: failed to create HCP cluster np-version-upgrade-cluster-5stxsv with version 4.20.34 Unexpected error: <fmt.wrapError \| 0xc000902a00>: failed to create HCP cluster np-version-upgrade-cluster-5stxsv: failed waiting for cluster="np-version-upgrade-cluster-5stxsv" in resourcegroup="rg-np-version-upgrade-5stxsv-pd9lkcxvmsqc" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/23bca7d1-d34e-45cb-bcf2-aa6a76f71651 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/23bca7d1-d34e-45cb-bcf2-aa6a76f71651", "name": "23bca7d1-d34e-45cb-bcf2-aa6a76f71651", "status": "Failed", "startTime": "2026-08-18T03:08:57.14601964Z", "endTime": "2026-08-18T03:27:57.63550179Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:133]: failed to create HCP cluster np-version-upgrade-cluster-5stxsv with version 4.20.34 Unexpected error: <fmt.wrapError \| 0xc000902a00>: failed to create HCP cluster np-version-upgrade-cluster-5stxsv: failed waiting for cluster="np-version-upgrade-cluster-5stxsv" in resourcegroup="rg-np-version-upgrade-5stxsv-pd9lkcxvmsqc" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/23bca7d1-d34e-45cb-bcf2-aa6a76f71651 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/23bca7d1-d34e-45cb-bcf2-aa6a76f71651", "name": "23bca7d1-d34e-45cb-bcf2-aa6a76f71651", "status": "Failed", "startTime": "2026-08-18T03:08:57.14601964Z", "endTime": "2026-08-18T03:27:57.63550179Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift 01:17:15 UTC · failed at e2e · test Customer should respect cluster-wide node limits with nodepool autoscaling · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:291]: failed to create HCP cluster node-limit-cluster with MaxNodesTotal limit Unexpected error: <fmt.wrapError \| 0xc00150c400>: failed to create HCP cluster node-limit-cluster: failed waiting for cluster="node-limit-cluster" in resourcegroup="node-limit-test-xsc659bqwsjv" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/ee89d445-6931-48de-beed-2caf5b9d9058 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/ee89d445-6931-48de-beed-2caf5b9d9058", "name": "ee89d445-6931-48de-beed-2caf5b9d9058", "status": "Failed", "startTime": "2026-08-18T02:26:39.318377569Z", "endTime": "2026-08-18T02:45:46.637467319Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:291]: failed to create HCP cluster node-limit-cluster with MaxNodesTotal limit Unexpected error: <fmt.wrapError \| 0xc00150c400>: failed to create HCP cluster node-limit-cluster: failed waiting for cluster="node-limit-cluster" in resourcegroup="node-limit-test-xsc659bqwsjv" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/ee89d445-6931-48de-beed-2caf5b9d9058 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/ee89d445-6931-48de-beed-2caf5b9d9058", "name": "ee89d445-6931-48de-beed-2caf5b9d9058", "status": "Failed", "startTime": "2026-08-18T02:26:39.318377569Z", "endTime": "2026-08-18T02:45:46.637467319Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred timeout during CreateHCPClusterAndWait; context deadline exceeded 01:17:15 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.22 · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:107]: HCP cluster rg-candidate-4-22-9rh5s6cz8tvc/cluster-candidate-4-22-cxtdzk should provision Unexpected error: <fmt.wrapErrors \| 0xc00180c810>: failed to create HCP cluster cluster-candidate-4-22-cxtdzk, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster cluster-candidate-4-22-cxtdzk in resource group rg-candidate-4-22-9rh5s6cz8tvc, error: failed waiting for cluster="cluster-candidate-4-22-cxtdzk" in resourcegroup="rg-candidate-4-22-9rh5s6cz8tvc" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster cluster-candidate-4-22-cxtdzk in resource group rg-candidate-4-22-9rh5s6cz8tvc, error: context deadline exceeded ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:107]: HCP cluster rg-candidate-4-22-9rh5s6cz8tvc/cluster-candidate-4-22-cxtdzk should provision Unexpected error: <fmt.wrapErrors \| 0xc00180c810>: failed to create HCP cluster cluster-candidate-4-22-cxtdzk, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster cluster-candidate-4-22-cxtdzk in resource group rg-candidate-4-22-9rh5s6cz8tvc, error: failed waiting for cluster="cluster-candidate-4-22-cxtdzk" in resourcegroup="rg-candidate-4-22-9rh5s6cz8tvc" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster cluster-candidate-4-22-cxtdzk in resource group rg-candidate-4-22-9rh5s6cz8tvc, error: context deadline exceeded ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift 01:17:15 UTC · failed at e2e · test Update HCPOpenShiftCluster Positive creates a cluster and updates tags with a PATCH request · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:161]: failed to create HCP cluster for patch-tags test Unexpected error: <fmt.wrapError \| 0xc00141ef60>: failed to create HCP cluster patch-tags-cluster: failed waiting for cluster="patch-tags-cluster" in resourcegroup="patch-tags-hx8xlpfzk9mg" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/1009cfae-4b95-4d7c-84aa-7bd528b272f2 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/1009cfae-4b95-4d7c-84aa-7bd528b272f2", "name": "1009cfae-4b95-4d7c-84aa-7bd528b272f2", "status": "Failed", "startTime": "2026-08-18T01:39:31.881688747Z", "endTime": "2026-08-18T01:58:34.438297863Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:161]: failed to create HCP cluster for patch-tags test Unexpected error: <fmt.wrapError \| 0xc00141ef60>: failed to create HCP cluster patch-tags-cluster: failed waiting for cluster="patch-tags-cluster" in resourcegroup="patch-tags-hx8xlpfzk9mg" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/1009cfae-4b95-4d7c-84aa-7bd528b272f2 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/1009cfae-4b95-4d7c-84aa-7bd528b272f2", "name": "1009cfae-4b95-4d7c-84aa-7bd528b272f2", "status": "Failed", "startTime": "2026-08-18T01:39:31.881688747Z", "endTime": "2026-08-18T01:58:34.438297863Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated; provider Microsoft.RedHatOpenShift 01:17:15 UTC · failed at e2e · test FIPS Mode Support with v2026-06-30 API should create an HCP cluster with FIPS mode enabled via cryptoRestrictions property · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_fips_mode.go:106]: failed to create HCP cluster "fips-enabled-cluster" with cryptoRestrictions set to FIPS Unexpected error: <fmt.wrapError \| 0xc000eec640>: failed waiting for cluster="fips-enabled-cluster" in resourcegroup="fips-enabled-js6s9lhv9qhv" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/82a24a4c-d210-4a62-8673-c0f5a51b1448 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/82a24a4c-d210-4a62-8673-c0f5a51b1448", "name": "82a24a4c-d210-4a62-8673-c0f5a51b1448", "status": "Failed", "startTime": "2026-08-18T01:34:47.523636047Z", "endTime": "2026-08-18T01:53:54.569768124Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_fips_mode.go:106]: failed to create HCP cluster "fips-enabled-cluster" with cryptoRestrictions set to FIPS Unexpected error: <fmt.wrapError \| 0xc000eec640>: failed waiting for cluster="fips-enabled-cluster" in resourcegroup="fips-enabled-js6s9lhv9qhv" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/82a24a4c-d210-4a62-8673-c0f5a51b1448 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/82a24a4c-d210-4a62-8673-c0f5a51b1448", "name": "82a24a4c-d210-4a62-8673-c0f5a51b1448", "status": "Failed", "startTime": "2026-08-18T01:34:47.523636047Z", "endTime": "2026-08-18T01:53:54.569768124Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift 01:17:15 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 4.22 · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:110]: failed to create HCP cluster "cluster-zstream-4-22-xztxk8" with version 4.22.8 on candidate channel Unexpected error: <fmt.wrapError \| 0xc000e90a00>: failed to create HCP cluster cluster-zstream-4-22-xztxk8: failed waiting for cluster="cluster-zstream-4-22-xztxk8" in resourcegroup="rg-zstream-upgrade-4-22-9n47t7ndzgk9" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/386cfeca-2d47-4438-8194-e04e52779c85 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/386cfeca-2d47-4438-8194-e04e52779c85", "name": "386cfeca-2d47-4438-8194-e04e52779c85", "status": "Failed", "startTime": "2026-08-18T02:22:48.738999859Z", "endTime": "2026-08-18T02:41:55.833760991Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:110]: failed to create HCP cluster "cluster-zstream-4-22-xztxk8" with version 4.22.8 on candidate channel Unexpected error: <fmt.wrapError \| 0xc000e90a00>: failed to create HCP cluster cluster-zstream-4-22-xztxk8: failed waiting for cluster="cluster-zstream-4-22-xztxk8" in resourcegroup="rg-zstream-upgrade-4-22-9n47t7ndzgk9" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/386cfeca-2d47-4438-8194-e04e52779c85 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/386cfeca-2d47-4438-8194-e04e52779c85", "name": "386cfeca-2d47-4438-8194-e04e52779c85", "status": "Failed", "startTime": "2026-08-18T02:22:48.738999859Z", "endTime": "2026-08-18T02:41:55.833760991Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift 01:17:15 UTC · failed at e2e · test Customer should be able to create an HCP cluster with Image Registry not present · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/image_registry_cluster_create.go:89]: failed to create HCP cluster with image registry disabled Unexpected error: <fmt.wrapError \| 0xc000dd70c0>: failed to create HCP cluster disabled-image-registry-hcp-cluster: failed waiting for cluster="disabled-image-registry-hcp-cluster" in resourcegroup="disabled-image-registry-mxnr5h7mxxnk" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/29126f72-891d-444f-87f5-fdcc7d248d85 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/29126f72-891d-444f-87f5-fdcc7d248d85", "name": "29126f72-891d-444f-87f5-fdcc7d248d85", "status": "Failed", "startTime": "2026-08-18T02:22:43.089171061Z", "endTime": "2026-08-18T02:41:45.735845882Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/image_registry_cluster_create.go:89]: failed to create HCP cluster with image registry disabled Unexpected error: <fmt.wrapError \| 0xc000dd70c0>: failed to create HCP cluster disabled-image-registry-hcp-cluster: failed waiting for cluster="disabled-image-registry-hcp-cluster" in resourcegroup="disabled-image-registry-mxnr5h7mxxnk" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/29126f72-891d-444f-87f5-fdcc7d248d85 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/29126f72-891d-444f-87f5-fdcc7d248d85", "name": "29126f72-891d-444f-87f5-fdcc7d248d85", "status": "Failed", "startTime": "2026-08-18T02:22:43.089171061Z", "endTime": "2026-08-18T02:41:45.735845882Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift 01:17:15 UTC · failed at e2e · test Customer should be able to create a cluster with default autoscaling and a nodepool with autoscaling enabled up to replica limits · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:102]: failed to create HCP cluster np-autoscale-cluster Unexpected error: <fmt.wrapError \| 0xc000efa660>: failed to create HCP cluster np-autoscale-cluster: failed waiting for cluster="np-autoscale-cluster" in resourcegroup="np-autoscaling-j45nlzgmgwsl" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/1522b984-8738-4ce3-a94c-ff57b760074b -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/1522b984-8738-4ce3-a94c-ff57b760074b", "name": "1522b984-8738-4ce3-a94c-ff57b760074b", "status": "Failed", "startTime": "2026-08-18T02:23:15.108288438Z", "endTime": "2026-08-18T02:42:15.72613301Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:102]: failed to create HCP cluster np-autoscale-cluster Unexpected error: <fmt.wrapError \| 0xc000efa660>: failed to create HCP cluster np-autoscale-cluster: failed waiting for cluster="np-autoscale-cluster" in resourcegroup="np-autoscaling-j45nlzgmgwsl" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/1522b984-8738-4ce3-a94c-ff57b760074b -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/1522b984-8738-4ce3-a94c-ff57b760074b", "name": "1522b984-8738-4ce3-a94c-ff57b760074b", "status": "Failed", "startTime": "2026-08-18T02:23:15.108288438Z", "endTime": "2026-08-18T02:42:15.72613301Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred

Time (UTC)

Job

Failed at

Result

Failed tests

Details

23:00:06

periodic-ci-Azure-ARO-HCP-main-periodic-prod-e2e-parallel-ocp-nightly

post-goodmerged PR

No additional run metadata captured

e2e

failed

n/a

e2e (3)

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift

23:00:06 UTC · failed at e2e · test Customer should be able to create a cluster and node pools with aggregated advanced features · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:184]: failed to create HCP cluster "agg-cluster" with aggregated settings
Unexpected error:
    <*fmt.wrapError | 0xc000aed260>: 
    failed waiting for cluster="agg-cluster" in resourcegroup="feature-aggregation-z9m2c6q2wt9f" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/2381acc6-3e26-4e23-b87b-3b89fac035ca
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/2381acc6-3e26-4e23-b87b-3b89fac035ca",
      "name": "2381acc6-3e26-4e23-b87b-3b89fac035ca",
      "status": "Failed",
      "startTime": "2026-08-18T23:06:57.972257037Z",
      "endTime": "2026-08-18T23:25:57.729049787Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:184]: failed to create HCP cluster "agg-cluster" with aggregated settings
Unexpected error:
    <*fmt.wrapError | 0xc000aed260>: 
    failed waiting for cluster="agg-cluster" in resourcegroup="feature-aggregation-z9m2c6q2wt9f" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/2381acc6-3e26-4e23-b87b-3b89fac035ca
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/2381acc6-3e26-4e23-b87b-3b89fac035ca",
      "name": "2381acc6-3e26-4e23-b87b-3b89fac035ca",
      "status": "Failed",
      "startTime": "2026-08-18T23:06:57.972257037Z",
      "endTime": "2026-08-18T23:25:57.729049787Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

timeout during CreateHCPClusterAndWait; context deadline exceeded

23:00:06 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP nightly channel for 5.1 · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:107]: HCP cluster rg-nightly-5-1-mds7bfdnr4mb/cluster-nightly-5-1-jkzd98 should provision
Unexpected error:
    <*fmt.wrapErrors | 0xc0012549c0>: 
    failed to create HCP cluster cluster-nightly-5-1-jkzd98, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster cluster-nightly-5-1-jkzd98 in resource group rg-nightly-5-1-mds7bfdnr4mb, error: failed waiting for cluster="cluster-nightly-5-1-jkzd98" in resourcegroup="rg-nightly-5-1-mds7bfdnr4mb" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster cluster-nightly-5-1-jkzd98 in resource group rg-nightly-5-1-mds7bfdnr4mb, error: context deadline exceeded
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:107]: HCP cluster rg-nightly-5-1-mds7bfdnr4mb/cluster-nightly-5-1-jkzd98 should provision
Unexpected error:
    <*fmt.wrapErrors | 0xc0012549c0>: 
    failed to create HCP cluster cluster-nightly-5-1-jkzd98, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster cluster-nightly-5-1-jkzd98 in resource group rg-nightly-5-1-mds7bfdnr4mb, error: failed waiting for cluster="cluster-nightly-5-1-jkzd98" in resourcegroup="rg-nightly-5-1-mds7bfdnr4mb" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster cluster-nightly-5-1-jkzd98 in resource group rg-nightly-5-1-mds7bfdnr4mb, error: context deadline exceeded
    ...
occurred

timeout during CreateHCPClusterAndWait; context deadline exceeded

23:00:06 UTC · failed at e2e · test Customer should create a cluster with private KAS and verify API server is only reachable from VNet while ingress remains public · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:90]: failed to create HCP cluster "private-kas" with private KAS
Unexpected error:
    <*fmt.wrapErrors | 0xc001c36390>: 
    failed to create HCP cluster private-kas, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster private-kas in resource group private-kas-n2w89h2n5hbn, error: failed waiting for cluster="private-kas" in resourcegroup="private-kas-n2w89h2n5hbn" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster private-kas in resource group private-kas-n2w89h2n5hbn, error: context deadline exceeded
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:90]: failed to create HCP cluster "private-kas" with private KAS
Unexpected error:
    <*fmt.wrapErrors | 0xc001c36390>: 
    failed to create HCP cluster private-kas, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster private-kas in resource group private-kas-n2w89h2n5hbn, error: failed waiting for cluster="private-kas" in resourcegroup="private-kas-n2w89h2n5hbn" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster private-kas in resource group private-kas-n2w89h2n5hbn, error: context deadline exceeded
    ...
occurred

02:00:45

periodic-ci-Azure-ARO-HCP-main-periodic-prod-e2e-parallel

post-goodmerged PR

No additional run metadata captured

e2e

failed

n/a

e2e (1)

VerifyVMOSDiskCustomerEncryption(nodePool=<nodepool>) failed: OS disk encryption verification failed for 2/2 VMs: disk des-encrypt-des-np-z8ktk-l4d4d_OSDisk for VM des-encrypt-des-np-z8ktk-l4d4d has encryption type EncryptionAtRestWithPlatformKey, expected

02:00:45 UTC · failed at e2e · test Nodepool OS Disk Encryption should create a nodepool with customer-managed disk encryption via DES · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_osdisk_encryption.go:163]: cluster verification failed
Unexpected error:
    <*errors.joinError | 0xc0013905a0>: 
    VerifyVMOSDiskCustomerEncryption(nodePool=des-np) failed: OS disk encryption verification failed for 2/2 VMs: disk des-encrypt-des-np-z8ktk-l4d4d_OSDisk for VM des-encrypt-des-np-z8ktk-l4d4d has encryption type EncryptionAtRestWithPlatformKey, expected EncryptionAtRestWithCustomerKey
    disk des-encrypt-des-np-z8ktk-t2cjq_OSDisk for VM des-encrypt-des-np-z8ktk-t2cjq has encryption type EncryptionAtRestWithPlatformKey, expected EncryptionAtRestWithCustomerKey
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_osdisk_encryption.go:163]: cluster verification failed
Unexpected error:
    <*errors.joinError | 0xc0013905a0>: 
    VerifyVMOSDiskCustomerEncryption(nodePool=des-np) failed: OS disk encryption verification failed for 2/2 VMs: disk des-encrypt-des-np-z8ktk-l4d4d_OSDisk for VM des-encrypt-des-np-z8ktk-l4d4d has encryption type EncryptionAtRestWithPlatformKey, expected EncryptionAtRestWithCustomerKey
    disk des-encrypt-des-np-z8ktk-t2cjq_OSDisk for VM des-encrypt-des-np-z8ktk-t2cjq has encryption type EncryptionAtRestWithPlatformKey, expected EncryptionAtRestWithCustomerKey
    ...
occurred

01:17:15

branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel

post-goodmerged PR

No additional run metadata captured

e2e

failed

n/a

e2e (48)

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift

01:17:15 UTC · failed at e2e · test Customer should be able to successfully upgrade control plane minor version from 4.20 minor to 4.21 minor · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:182]: failed to create HCP cluster "cp-ystream-upgrade-4-21-pl2828" with install version 4.20.34
Unexpected error:
    <*fmt.wrapError | 0xc001408920>: 
    failed to create HCP cluster cp-ystream-upgrade-4-21-pl2828: failed waiting for cluster="cp-ystream-upgrade-4-21-pl2828" in resourcegroup="rg-cp-ystream-upgrade-4-21-2nlwvsvrv9bp" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/8b146388-d74d-42e6-bdaf-8a3edf1b3beb
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/8b146388-d74d-42e6-bdaf-8a3edf1b3beb",
      "name": "8b146388-d74d-42e6-bdaf-8a3edf1b3beb",
      "status": "Failed",
      "startTime": "2026-08-18T02:21:03.015049333Z",
      "endTime": "2026-08-18T02:40:05.87439567Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:182]: failed to create HCP cluster "cp-ystream-upgrade-4-21-pl2828" with install version 4.20.34
Unexpected error:
    <*fmt.wrapError | 0xc001408920>: 
    failed to create HCP cluster cp-ystream-upgrade-4-21-pl2828: failed waiting for cluster="cp-ystream-upgrade-4-21-pl2828" in resourcegroup="rg-cp-ystream-upgrade-4-21-2nlwvsvrv9bp" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/8b146388-d74d-42e6-bdaf-8a3edf1b3beb
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/8b146388-d74d-42e6-bdaf-8a3edf1b3beb",
      "name": "8b146388-d74d-42e6-bdaf-8a3edf1b3beb",
      "status": "Failed",
      "startTime": "2026-08-18T02:21:03.015049333Z",
      "endTime": "2026-08-18T02:40:05.87439567Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift

01:17:15 UTC · failed at e2e · test Customer should be able to create an HCP cluster with custom autoscaling and update the autoscaling configuration · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:108]: failed to create HCP cluster "autoscaling-hcp-cluster" with custom autoscaling
Unexpected error:
    <*fmt.wrapError | 0xc000f62880>: 
    failed to create HCP cluster autoscaling-hcp-cluster: failed waiting for cluster="autoscaling-hcp-cluster" in resourcegroup="autoscaling-cluster-dnvp8bsl7ccf" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/a5af20d3-0412-488c-bbe6-26b573f24f51
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/a5af20d3-0412-488c-bbe6-26b573f24f51",
      "name": "a5af20d3-0412-488c-bbe6-26b573f24f51",
      "status": "Failed",
      "startTime": "2026-08-18T01:34:47.053111478Z",
      "endTime": "2026-08-18T01:53:54.554086959Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:108]: failed to create HCP cluster "autoscaling-hcp-cluster" with custom autoscaling
Unexpected error:
    <*fmt.wrapError | 0xc000f62880>: 
    failed to create HCP cluster autoscaling-hcp-cluster: failed waiting for cluster="autoscaling-hcp-cluster" in resourcegroup="autoscaling-cluster-dnvp8bsl7ccf" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/a5af20d3-0412-488c-bbe6-26b573f24f51
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/a5af20d3-0412-488c-bbe6-26b573f24f51",
      "name": "a5af20d3-0412-488c-bbe6-26b573f24f51",
      "status": "Failed",
      "startTime": "2026-08-18T01:34:47.053111478Z",
      "endTime": "2026-08-18T01:53:54.554086959Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift

01:17:15 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 4.19 · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:110]: failed to create HCP cluster "cluster-zstream-4-19-pkvpqg" with version 4.19.41 on candidate channel
Unexpected error:
    <*fmt.wrapError | 0xc000993420>: 
    failed to create HCP cluster cluster-zstream-4-19-pkvpqg: failed waiting for cluster="cluster-zstream-4-19-pkvpqg" in resourcegroup="rg-zstream-upgrade-4-19-xq9w64kvmwvv" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/fd634c01-e142-477e-8a22-f4420d35f59e
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/fd634c01-e142-477e-8a22-f4420d35f59e",
      "name": "fd634c01-e142-477e-8a22-f4420d35f59e",
      "status": "Failed",
      "startTime": "2026-08-18T02:22:35.247158175Z",
      "endTime": "2026-08-18T02:41:35.737706721Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:110]: failed to create HCP cluster "cluster-zstream-4-19-pkvpqg" with version 4.19.41 on candidate channel
Unexpected error:
    <*fmt.wrapError | 0xc000993420>: 
    failed to create HCP cluster cluster-zstream-4-19-pkvpqg: failed waiting for cluster="cluster-zstream-4-19-pkvpqg" in resourcegroup="rg-zstream-upgrade-4-19-xq9w64kvmwvv" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/fd634c01-e142-477e-8a22-f4420d35f59e
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/fd634c01-e142-477e-8a22-f4420d35f59e",
      "name": "fd634c01-e142-477e-8a22-f4420d35f59e",
      "status": "Failed",
      "startTime": "2026-08-18T02:22:35.247158175Z",
      "endTime": "2026-08-18T02:41:35.737706721Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift

01:17:15 UTC · failed at e2e · test Authorized CIDRs Connectivity should allow API access only from authorized VM IP · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:111]: failed to create HCP cluster "cidr-connectivity-test" with authorized CIDRs
Unexpected error:
    <*fmt.wrapError | 0xc0010c8b20>: 
    failed to create HCP cluster cidr-connectivity-test: failed waiting for cluster="cidr-connectivity-test" in resourcegroup="e2e-cidr-connectivity-zrwbc572c7q6" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/f7f07a25-6078-4e6c-a5c2-a24eec38065d
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/f7f07a25-6078-4e6c-a5c2-a24eec38065d",
      "name": "f7f07a25-6078-4e6c-a5c2-a24eec38065d",
      "status": "Failed",
      "startTime": "2026-08-18T01:39:50.996749726Z",
      "endTime": "2026-08-18T01:58:54.431759835Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:111]: failed to create HCP cluster "cidr-connectivity-test" with authorized CIDRs
Unexpected error:
    <*fmt.wrapError | 0xc0010c8b20>: 
    failed to create HCP cluster cidr-connectivity-test: failed waiting for cluster="cidr-connectivity-test" in resourcegroup="e2e-cidr-connectivity-zrwbc572c7q6" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/f7f07a25-6078-4e6c-a5c2-a24eec38065d
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/f7f07a25-6078-4e6c-a5c2-a24eec38065d",
      "name": "f7f07a25-6078-4e6c-a5c2-a24eec38065d",
      "status": "Failed",
      "startTime": "2026-08-18T01:39:50.996749726Z",
      "endTime": "2026-08-18T01:58:54.431759835Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated; provider Microsoft.RedHatOpenShift

01:17:15 UTC · failed at e2e · test Customer should create a cluster with private KAS and verify API server is only reachable from VNet while ingress remains public · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:90]: failed to create HCP cluster "private-kas" with private KAS
Unexpected error:
    <*fmt.wrapError | 0xc00073f060>: 
    failed to create HCP cluster private-kas: failed waiting for cluster="private-kas" in resourcegroup="private-kas-w7rtqw6zhbhb" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/d58e4803-0126-47e9-845d-d1971bcc6eb1
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/d58e4803-0126-47e9-845d-d1971bcc6eb1",
      "name": "d58e4803-0126-47e9-845d-d1971bcc6eb1",
      "status": "Failed",
      "startTime": "2026-08-18T01:37:08.021690619Z",
      "endTime": "2026-08-18T01:56:14.403576829Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:90]: failed to create HCP cluster "private-kas" with private KAS
Unexpected error:
    <*fmt.wrapError | 0xc00073f060>: 
    failed to create HCP cluster private-kas: failed waiting for cluster="private-kas" in resourcegroup="private-kas-w7rtqw6zhbhb" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/d58e4803-0126-47e9-845d-d1971bcc6eb1
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/d58e4803-0126-47e9-845d-d1971bcc6eb1",
      "name": "d58e4803-0126-47e9-845d-d1971bcc6eb1",
      "status": "Failed",
      "startTime": "2026-08-18T01:37:08.021690619Z",
      "endTime": "2026-08-18T01:56:14.403576829Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift

01:17:15 UTC · failed at e2e · test Customer should be able to create an HCP cluster and manage pull secrets · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_pullsecret.go:113]: failed to create HCP cluster for pull secret test
Unexpected error:
    <*fmt.wrapError | 0xc000f16620>: 
    failed to create HCP cluster pullsecret-hcp-cluster: failed waiting for cluster="pullsecret-hcp-cluster" in resourcegroup="pullsecret-test-vdb7ww2x2vhj" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/9682c885-327c-4b57-bf19-db382361e77b
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/9682c885-327c-4b57-bf19-db382361e77b",
      "name": "9682c885-327c-4b57-bf19-db382361e77b",
      "status": "Failed",
      "startTime": "2026-08-18T01:34:15.525892211Z",
      "endTime": "2026-08-18T01:53:24.490752565Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_pullsecret.go:113]: failed to create HCP cluster for pull secret test
Unexpected error:
    <*fmt.wrapError | 0xc000f16620>: 
    failed to create HCP cluster pullsecret-hcp-cluster: failed waiting for cluster="pullsecret-hcp-cluster" in resourcegroup="pullsecret-test-vdb7ww2x2vhj" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/9682c885-327c-4b57-bf19-db382361e77b
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/9682c885-327c-4b57-bf19-db382361e77b",
      "name": "9682c885-327c-4b57-bf19-db382361e77b",
      "status": "Failed",
      "startTime": "2026-08-18T01:34:15.525892211Z",
      "endTime": "2026-08-18T01:53:24.490752565Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift

01:17:15 UTC · failed at e2e · test Customer should be able to lifecycle and confirm external auth on a cluster · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_list_and_verify.go:87]: failed to create HCP cluster for external auth list test
Unexpected error:
    <*fmt.wrapError | 0xc00081b4c0>: 
    failed to create HCP cluster ea-listz7drx2: failed waiting for cluster="ea-listz7drx2" in resourcegroup="ea-list-2s5lrmcqdq8j" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/64b6158a-ccf0-4864-a257-301af81fbf00
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/64b6158a-ccf0-4864-a257-301af81fbf00",
      "name": "64b6158a-ccf0-4864-a257-301af81fbf00",
      "status": "Failed",
      "startTime": "2026-08-18T02:22:09.794983918Z",
      "endTime": "2026-08-18T02:41:15.875400176Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_list_and_verify.go:87]: failed to create HCP cluster for external auth list test
Unexpected error:
    <*fmt.wrapError | 0xc00081b4c0>: 
    failed to create HCP cluster ea-listz7drx2: failed waiting for cluster="ea-listz7drx2" in resourcegroup="ea-list-2s5lrmcqdq8j" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/64b6158a-ccf0-4864-a257-301af81fbf00
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/64b6158a-ccf0-4864-a257-301af81fbf00",
      "name": "64b6158a-ccf0-4864-a257-301af81fbf00",
      "status": "Failed",
      "startTime": "2026-08-18T02:22:09.794983918Z",
      "endTime": "2026-08-18T02:41:15.875400176Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift

01:17:15 UTC · failed at e2e · test Customer should be able to update node pool labels and taints · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_labels_taints.go:92]: failed to create HCP cluster cluster-np-labels-taints
Unexpected error:
    <*fmt.wrapError | 0xc0009dd040>: 
    failed to create HCP cluster cluster-np-labels-taints: failed waiting for cluster="cluster-np-labels-taints" in resourcegroup="rg-np-labels-taints-b8wmfdpwzr4f" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/0de0c21a-0f71-4144-80e3-410d3cccafab
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/0de0c21a-0f71-4144-80e3-410d3cccafab",
      "name": "0de0c21a-0f71-4144-80e3-410d3cccafab",
      "status": "Failed",
      "startTime": "2026-08-18T02:47:28.122950359Z",
      "endTime": "2026-08-18T03:06:36.936036065Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_labels_taints.go:92]: failed to create HCP cluster cluster-np-labels-taints
Unexpected error:
    <*fmt.wrapError | 0xc0009dd040>: 
    failed to create HCP cluster cluster-np-labels-taints: failed waiting for cluster="cluster-np-labels-taints" in resourcegroup="rg-np-labels-taints-b8wmfdpwzr4f" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/0de0c21a-0f71-4144-80e3-410d3cccafab
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/0de0c21a-0f71-4144-80e3-410d3cccafab",
      "name": "0de0c21a-0f71-4144-80e3-410d3cccafab",
      "status": "Failed",
      "startTime": "2026-08-18T02:47:28.122950359Z",
      "endTime": "2026-08-18T03:06:36.936036065Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift

01:17:15 UTC · failed at e2e · test Customer should be able to create a cluster with an external auth config and get the external auth config · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:94]: failed to create HCP cluster for external auth test
Unexpected error:
    <*fmt.wrapError | 0xc00054af40>: 
    failed to create HCP cluster ea-cluster: failed waiting for cluster="ea-cluster" in resourcegroup="external-auth-cluster-vsndz7vcmkc2" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/8c055328-9528-4d95-bf30-f114c8824240
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/8c055328-9528-4d95-bf30-f114c8824240",
      "name": "8c055328-9528-4d95-bf30-f114c8824240",
      "status": "Failed",
      "startTime": "2026-08-18T02:21:25.465764241Z",
      "endTime": "2026-08-18T02:40:25.755703486Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:94]: failed to create HCP cluster for external auth test
Unexpected error:
    <*fmt.wrapError | 0xc00054af40>: 
    failed to create HCP cluster ea-cluster: failed waiting for cluster="ea-cluster" in resourcegroup="external-auth-cluster-vsndz7vcmkc2" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/8c055328-9528-4d95-bf30-f114c8824240
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/8c055328-9528-4d95-bf30-f114c8824240",
      "name": "8c055328-9528-4d95-bf30-f114c8824240",
      "status": "Failed",
      "startTime": "2026-08-18T02:21:25.465764241Z",
      "endTime": "2026-08-18T02:40:25.755703486Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift

01:17:15 UTC · failed at e2e · test Customer should be able to create an HCP cluster then delete it by deleting the customer resource group · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delete_cx_rg.go:90]: failed to create HCP cluster "cx-rg-hcp-cluster"
Unexpected error:
    <*fmt.wrapError | 0xc0000ff9e0>: 
    failed to create HCP cluster cx-rg-hcp-cluster: failed waiting for cluster="cx-rg-hcp-cluster" in resourcegroup="cx-rg-cluster-nm7frlj7wx8x" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/6c67b4db-e2e8-4147-ba66-6b76f9ea3698
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/6c67b4db-e2e8-4147-ba66-6b76f9ea3698",
      "name": "6c67b4db-e2e8-4147-ba66-6b76f9ea3698",
      "status": "Failed",
      "startTime": "2026-08-18T01:34:14.569006871Z",
      "endTime": "2026-08-18T01:53:14.520927967Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delete_cx_rg.go:90]: failed to create HCP cluster "cx-rg-hcp-cluster"
Unexpected error:
    <*fmt.wrapError | 0xc0000ff9e0>: 
    failed to create HCP cluster cx-rg-hcp-cluster: failed waiting for cluster="cx-rg-hcp-cluster" in resourcegroup="cx-rg-cluster-nm7frlj7wx8x" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/6c67b4db-e2e8-4147-ba66-6b76f9ea3698
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/6c67b4db-e2e8-4147-ba66-6b76f9ea3698",
      "name": "6c67b4db-e2e8-4147-ba66-6b76f9ea3698",
      "status": "Failed",
      "startTime": "2026-08-18T01:34:14.569006871Z",
      "endTime": "2026-08-18T01:53:14.520927967Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift

01:17:15 UTC · failed at e2e · test Customer should downgrade a nodepool version z-stream downgrade from 4.21.zLatest to 4.21.zPrevious · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:593]: failed to create HCP cluster
Unexpected error:
    <*fmt.wrapError | 0xc000f9cd60>: 
    failed to create HCP cluster np-downgrade-mcst4p: failed waiting for cluster="np-downgrade-mcst4p" in resourcegroup="rg-np-downgrade-mcst4p-6f78v8fbdxfl" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/c52fe4a1-d446-434e-ba20-149524aafd7b
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/c52fe4a1-d446-434e-ba20-149524aafd7b",
      "name": "c52fe4a1-d446-434e-ba20-149524aafd7b",
      "status": "Failed",
      "startTime": "2026-08-18T03:07:13.980026707Z",
      "endTime": "2026-08-18T03:26:17.641036396Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:593]: failed to create HCP cluster
Unexpected error:
    <*fmt.wrapError | 0xc000f9cd60>: 
    failed to create HCP cluster np-downgrade-mcst4p: failed waiting for cluster="np-downgrade-mcst4p" in resourcegroup="rg-np-downgrade-mcst4p-6f78v8fbdxfl" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/c52fe4a1-d446-434e-ba20-149524aafd7b
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/c52fe4a1-d446-434e-ba20-149524aafd7b",
      "name": "c52fe4a1-d446-434e-ba20-149524aafd7b",
      "status": "Failed",
      "startTime": "2026-08-18T03:07:13.980026707Z",
      "endTime": "2026-08-18T03:26:17.641036396Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

01:17:15 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 5.0 · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:107]: HCP cluster rg-candidate-5-0-lln7p567vmfx/cluster-candidate-5-0-n9hg2s should provision
Unexpected error:
    <*fmt.wrapError | 0xc000b8a840>: 
    failed to create HCP cluster cluster-candidate-5-0-n9hg2s: failed waiting for cluster="cluster-candidate-5-0-n9hg2s" in resourcegroup="rg-candidate-5-0-lln7p567vmfx" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/9275203c-c388-4fdb-8585-24996057f18a
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/9275203c-c388-4fdb-8585-24996057f18a",
      "name": "9275203c-c388-4fdb-8585-24996057f18a",
      "status": "Failed",
      "startTime": "2026-08-18T02:20:20.646227309Z",
      "endTime": "2026-08-18T02:39:25.899893485Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:107]: HCP cluster rg-candidate-5-0-lln7p567vmfx/cluster-candidate-5-0-n9hg2s should provision
Unexpected error:
    <*fmt.wrapError | 0xc000b8a840>: 
    failed to create HCP cluster cluster-candidate-5-0-n9hg2s: failed waiting for cluster="cluster-candidate-5-0-n9hg2s" in resourcegroup="rg-candidate-5-0-lln7p567vmfx" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/9275203c-c388-4fdb-8585-24996057f18a
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/9275203c-c388-4fdb-8585-24996057f18a",
      "name": "9275203c-c388-4fdb-8585-24996057f18a",
      "status": "Failed",
      "startTime": "2026-08-18T02:20:20.646227309Z",
      "endTime": "2026-08-18T02:39:25.899893485Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift

01:17:15 UTC · failed at e2e · test Customer should be able to create node pools with ARM64-based VMs · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/arm64_nodepool.go:88]: failed to create HCP cluster "arm64-vm-hcp-cluster"
Unexpected error:
    <*fmt.wrapError | 0xc000b404a0>: 
    failed to create HCP cluster arm64-vm-hcp-cluster: failed waiting for cluster="arm64-vm-hcp-cluster" in resourcegroup="arm64-vm-cluster-dbtgnnnvcz2z" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/be950d1d-0ef4-4e89-8d67-64c5700365db
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/be950d1d-0ef4-4e89-8d67-64c5700365db",
      "name": "be950d1d-0ef4-4e89-8d67-64c5700365db",
      "status": "Failed",
      "startTime": "2026-08-18T01:34:05.037589846Z",
      "endTime": "2026-08-18T01:53:14.531114964Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/arm64_nodepool.go:88]: failed to create HCP cluster "arm64-vm-hcp-cluster"
Unexpected error:
    <*fmt.wrapError | 0xc000b404a0>: 
    failed to create HCP cluster arm64-vm-hcp-cluster: failed waiting for cluster="arm64-vm-hcp-cluster" in resourcegroup="arm64-vm-cluster-dbtgnnnvcz2z" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/be950d1d-0ef4-4e89-8d67-64c5700365db
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/be950d1d-0ef4-4e89-8d67-64c5700365db",
      "name": "be950d1d-0ef4-4e89-8d67-64c5700365db",
      "status": "Failed",
      "startTime": "2026-08-18T01:34:05.037589846Z",
      "endTime": "2026-08-18T01:53:14.531114964Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

Cluster provisioning failed

01:17:15 UTC · failed at e2e · test Customer should be able to test admin credentials before cluster ready, then full admin credential lifecycle · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:158]: Cluster provisioning failed
fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:158]: Cluster provisioning failed

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift

01:17:15 UTC · failed at e2e · test Customer should create an HCP cluster and validate TLS certificates · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:96]: failed to create HCP cluster for TLS endpoint test
Unexpected error:
    <*fmt.wrapError | 0xc000e00480>: 
    failed to create HCP cluster tls-endpoint-hcp-cluster: failed waiting for cluster="tls-endpoint-hcp-cluster" in resourcegroup="tls-endpoint-cluster-b8f6jdz5vkr9" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/5529fe79-5551-4598-a548-b1f242876d20
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/5529fe79-5551-4598-a548-b1f242876d20",
      "name": "5529fe79-5551-4598-a548-b1f242876d20",
      "status": "Failed",
      "startTime": "2026-08-18T01:34:54.967119171Z",
      "endTime": "2026-08-18T01:53:54.734709374Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:96]: failed to create HCP cluster for TLS endpoint test
Unexpected error:
    <*fmt.wrapError | 0xc000e00480>: 
    failed to create HCP cluster tls-endpoint-hcp-cluster: failed waiting for cluster="tls-endpoint-hcp-cluster" in resourcegroup="tls-endpoint-cluster-b8f6jdz5vkr9" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/5529fe79-5551-4598-a548-b1f242876d20
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/5529fe79-5551-4598-a548-b1f242876d20",
      "name": "5529fe79-5551-4598-a548-b1f242876d20",
      "status": "Failed",
      "startTime": "2026-08-18T01:34:54.967119171Z",
      "endTime": "2026-08-18T01:53:54.734709374Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

01:17:15 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.21 · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:107]: HCP cluster rg-candidate-4-21-xh9qrb947nb7/cluster-candidate-4-21-lvrbs2 should provision
Unexpected error:
    <*fmt.wrapError | 0xc001046ee0>: 
    failed to create HCP cluster cluster-candidate-4-21-lvrbs2: failed waiting for cluster="cluster-candidate-4-21-lvrbs2" in resourcegroup="rg-candidate-4-21-xh9qrb947nb7" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/b434160c-effb-4cc2-a595-1bb2db1f60ca
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/b434160c-effb-4cc2-a595-1bb2db1f60ca",
      "name": "b434160c-effb-4cc2-a595-1bb2db1f60ca",
      "status": "Failed",
      "startTime": "2026-08-18T02:20:00.98447986Z",
      "endTime": "2026-08-18T02:39:05.838171548Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:107]: HCP cluster rg-candidate-4-21-xh9qrb947nb7/cluster-candidate-4-21-lvrbs2 should provision
Unexpected error:
    <*fmt.wrapError | 0xc001046ee0>: 
    failed to create HCP cluster cluster-candidate-4-21-lvrbs2: failed waiting for cluster="cluster-candidate-4-21-lvrbs2" in resourcegroup="rg-candidate-4-21-xh9qrb947nb7" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/b434160c-effb-4cc2-a595-1bb2db1f60ca
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/b434160c-effb-4cc2-a595-1bb2db1f60ca",
      "name": "b434160c-effb-4cc2-a595-1bb2db1f60ca",
      "status": "Failed",
      "startTime": "2026-08-18T02:20:00.98447986Z",
      "endTime": "2026-08-18T02:39:05.838171548Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift

01:17:15 UTC · failed at e2e · test Customer should be able to create a HCP cluster and use cilium CNI plugin · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_cni_cilium.go:88]: failed to create HCP cluster "cilium-cl" without CNI
Unexpected error:
    <*fmt.wrapError | 0xc0013b4180>: 
    failed to create HCP cluster cilium-cl: failed waiting for cluster="cilium-cl" in resourcegroup="cni-cilium-lglb9vrqn8kz" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/32ec6679-a8ba-4e88-aac1-83469c53918c
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/32ec6679-a8ba-4e88-aac1-83469c53918c",
      "name": "32ec6679-a8ba-4e88-aac1-83469c53918c",
      "status": "Failed",
      "startTime": "2026-08-18T01:34:05.638284549Z",
      "endTime": "2026-08-18T01:53:14.514295274Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_cni_cilium.go:88]: failed to create HCP cluster "cilium-cl" without CNI
Unexpected error:
    <*fmt.wrapError | 0xc0013b4180>: 
    failed to create HCP cluster cilium-cl: failed waiting for cluster="cilium-cl" in resourcegroup="cni-cilium-lglb9vrqn8kz" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/32ec6679-a8ba-4e88-aac1-83469c53918c
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/32ec6679-a8ba-4e88-aac1-83469c53918c",
      "name": "32ec6679-a8ba-4e88-aac1-83469c53918c",
      "status": "Failed",
      "startTime": "2026-08-18T01:34:05.638284549Z",
      "endTime": "2026-08-18T01:53:14.514295274Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift

01:17:15 UTC · failed at e2e · test Customer should upgrade a nodepool skipping one minor version (+2) from 4.20.z to 4.22.zLatest · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:462]: failed to create HCP cluster np-skip-minor-4l8vkx at version 4.22.10
Unexpected error:
    <*fmt.wrapError | 0xc000f46b80>: 
    failed to create HCP cluster np-skip-minor-4l8vkx: failed waiting for cluster="np-skip-minor-4l8vkx" in resourcegroup="rg-np-skip-minor-4l8vkx-tqb64hrlx82j" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/726ca7c3-5ea4-4a42-9377-15144aa74b54
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/726ca7c3-5ea4-4a42-9377-15144aa74b54",
      "name": "726ca7c3-5ea4-4a42-9377-15144aa74b54",
      "status": "Failed",
      "startTime": "2026-08-18T03:07:14.300447528Z",
      "endTime": "2026-08-18T03:26:17.639458601Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:462]: failed to create HCP cluster np-skip-minor-4l8vkx at version 4.22.10
Unexpected error:
    <*fmt.wrapError | 0xc000f46b80>: 
    failed to create HCP cluster np-skip-minor-4l8vkx: failed waiting for cluster="np-skip-minor-4l8vkx" in resourcegroup="rg-np-skip-minor-4l8vkx-tqb64hrlx82j" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/726ca7c3-5ea4-4a42-9377-15144aa74b54
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/726ca7c3-5ea4-4a42-9377-15144aa74b54",
      "name": "726ca7c3-5ea4-4a42-9377-15144aa74b54",
      "status": "Failed",
      "startTime": "2026-08-18T03:07:14.300447528Z",
      "endTime": "2026-08-18T03:26:17.639458601Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift

01:17:15 UTC · failed at e2e · test Customer should upgrade a nodepool to a version without Cincinnati upgrade edge z-stream upgrade without Cincinnati edge in 4.20 · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:314]: failed to create HCP cluster np-noedge-g5fc5s
Unexpected error:
    <*fmt.wrapError | 0xc00084e1e0>: 
    failed to create HCP cluster np-noedge-g5fc5s: failed waiting for cluster="np-noedge-g5fc5s" in resourcegroup="rg-np-noedge-g5fc5s-8bfh6hb267dw" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/94bbeecd-3423-4010-8314-ff8e53a38982
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/94bbeecd-3423-4010-8314-ff8e53a38982",
      "name": "94bbeecd-3423-4010-8314-ff8e53a38982",
      "status": "Failed",
      "startTime": "2026-08-18T03:22:09.446229949Z",
      "endTime": "2026-08-18T03:41:17.713501122Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:314]: failed to create HCP cluster np-noedge-g5fc5s
Unexpected error:
    <*fmt.wrapError | 0xc00084e1e0>: 
    failed to create HCP cluster np-noedge-g5fc5s: failed waiting for cluster="np-noedge-g5fc5s" in resourcegroup="rg-np-noedge-g5fc5s-8bfh6hb267dw" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/94bbeecd-3423-4010-8314-ff8e53a38982
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/94bbeecd-3423-4010-8314-ff8e53a38982",
      "name": "94bbeecd-3423-4010-8314-ff8e53a38982",
      "status": "Failed",
      "startTime": "2026-08-18T03:22:09.446229949Z",
      "endTime": "2026-08-18T03:41:17.713501122Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift

01:17:15 UTC · failed at e2e · test Customer should be able to forward control plane logs to a storage account and Event Hub via shoebox diagnostic settings · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/shoebox_logs.go:270]: failed to create HCP cluster shoebox-hcp-cluster
Unexpected error:
    <*fmt.wrapError | 0xc001580820>: 
    failed to create HCP cluster shoebox-hcp-cluster: failed waiting for cluster="shoebox-hcp-cluster" in resourcegroup="shoebox-cluster-n9kkzn45w49n" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/2bc60d2b-0fbd-4ed7-895c-5b2d6486be40
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/2bc60d2b-0fbd-4ed7-895c-5b2d6486be40",
      "name": "2bc60d2b-0fbd-4ed7-895c-5b2d6486be40",
      "status": "Failed",
      "startTime": "2026-08-18T03:06:49.439367376Z",
      "endTime": "2026-08-18T03:25:57.638155323Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/shoebox_logs.go:270]: failed to create HCP cluster shoebox-hcp-cluster
Unexpected error:
    <*fmt.wrapError | 0xc001580820>: 
    failed to create HCP cluster shoebox-hcp-cluster: failed waiting for cluster="shoebox-hcp-cluster" in resourcegroup="shoebox-cluster-n9kkzn45w49n" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/2bc60d2b-0fbd-4ed7-895c-5b2d6486be40
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/2bc60d2b-0fbd-4ed7-895c-5b2d6486be40",
      "name": "2bc60d2b-0fbd-4ed7-895c-5b2d6486be40",
      "status": "Failed",
      "startTime": "2026-08-18T03:06:49.439367376Z",
      "endTime": "2026-08-18T03:25:57.638155323Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift

01:17:15 UTC · failed at e2e · test Customer should be able to create an HCP cluster and custom node pool osDisk size · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_nodepool_osdisk.go:80]: failed to create HCP cluster "hcp-cluster-np-128"
Unexpected error:
    <*fmt.wrapError | 0xc001592820>: 
    failed to create HCP cluster hcp-cluster-np-128: failed waiting for cluster="hcp-cluster-np-128" in resourcegroup="clusternp128-nr7gpbmjfgdn" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/8a0dbd68-0dcc-4587-a43f-63ee3a680700
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/8a0dbd68-0dcc-4587-a43f-63ee3a680700",
      "name": "8a0dbd68-0dcc-4587-a43f-63ee3a680700",
      "status": "Failed",
      "startTime": "2026-08-18T01:34:45.799991077Z",
      "endTime": "2026-08-18T01:53:54.557961621Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_nodepool_osdisk.go:80]: failed to create HCP cluster "hcp-cluster-np-128"
Unexpected error:
    <*fmt.wrapError | 0xc001592820>: 
    failed to create HCP cluster hcp-cluster-np-128: failed waiting for cluster="hcp-cluster-np-128" in resourcegroup="clusternp128-nr7gpbmjfgdn" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/8a0dbd68-0dcc-4587-a43f-63ee3a680700
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/8a0dbd68-0dcc-4587-a43f-63ee3a680700",
      "name": "8a0dbd68-0dcc-4587-a43f-63ee3a680700",
      "status": "Failed",
      "startTime": "2026-08-18T01:34:45.799991077Z",
      "endTime": "2026-08-18T01:53:54.557961621Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift

01:17:15 UTC · failed at e2e · test Update HCPOpenShiftCluster Negative creates a cluster and fails to update its name with a PATCH request · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:84]: failed to create HCP cluster for patch-name test
Unexpected error:
    <*fmt.wrapError | 0xc000b7ad20>: 
    failed to create HCP cluster patch-name-cluster: failed waiting for cluster="patch-name-cluster" in resourcegroup="patch-name-6jdjkfkzjj4q" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/84cf09bc-9ccf-4c52-ad0b-3215eecddc43
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/84cf09bc-9ccf-4c52-ad0b-3215eecddc43",
      "name": "84cf09bc-9ccf-4c52-ad0b-3215eecddc43",
      "status": "Failed",
      "startTime": "2026-08-18T01:35:14.689867232Z",
      "endTime": "2026-08-18T01:54:14.533556615Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:84]: failed to create HCP cluster for patch-name test
Unexpected error:
    <*fmt.wrapError | 0xc000b7ad20>: 
    failed to create HCP cluster patch-name-cluster: failed waiting for cluster="patch-name-cluster" in resourcegroup="patch-name-6jdjkfkzjj4q" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/84cf09bc-9ccf-4c52-ad0b-3215eecddc43
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/84cf09bc-9ccf-4c52-ad0b-3215eecddc43",
      "name": "84cf09bc-9ccf-4c52-ad0b-3215eecddc43",
      "status": "Failed",
      "startTime": "2026-08-18T01:35:14.689867232Z",
      "endTime": "2026-08-18T01:54:14.533556615Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

01:17:15 UTC · failed at e2e · test Customer should be able to create an HCP cluster and manage ImageDigestMirrors · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/idms_lifecycle.go:113]: failed to create HCP cluster with ImageDigestMirrors
Unexpected error:
    <*fmt.wrapError | 0xc000a94b00>: 
    failed to create HCP cluster idms-e2e-hcp-cluster: failed waiting for cluster="idms-e2e-hcp-cluster" in resourcegroup="idms-rl7558nqx6x5" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/6c83ecdf-321d-4c22-8c0e-0e67dfc51b95
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/6c83ecdf-321d-4c22-8c0e-0e67dfc51b95",
      "name": "6c83ecdf-321d-4c22-8c0e-0e67dfc51b95",
      "status": "Failed",
      "startTime": "2026-08-18T02:22:38.138134816Z",
      "endTime": "2026-08-18T02:41:45.799493023Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/idms_lifecycle.go:113]: failed to create HCP cluster with ImageDigestMirrors
Unexpected error:
    <*fmt.wrapError | 0xc000a94b00>: 
    failed to create HCP cluster idms-e2e-hcp-cluster: failed waiting for cluster="idms-e2e-hcp-cluster" in resourcegroup="idms-rl7558nqx6x5" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/6c83ecdf-321d-4c22-8c0e-0e67dfc51b95
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/6c83ecdf-321d-4c22-8c0e-0e67dfc51b95",
      "name": "6c83ecdf-321d-4c22-8c0e-0e67dfc51b95",
      "status": "Failed",
      "startTime": "2026-08-18T02:22:38.138134816Z",
      "endTime": "2026-08-18T02:41:45.799493023Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift

01:17:15 UTC · failed at e2e · test Customer should be able to use workload identity via the cluster OIDC issuer URL · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/oidc_issuer_workload_identity.go:217]: failed to create HCP cluster oidc-wi-cluster
Unexpected error:
    <*fmt.wrapError | 0xc001454fa0>: 
    failed to create HCP cluster oidc-wi-cluster: failed waiting for cluster="oidc-wi-cluster" in resourcegroup="oidc-wi-rlpxm5hnr4qk" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/9204c2aa-afaa-4aec-929c-673c0f6e633e
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/9204c2aa-afaa-4aec-929c-673c0f6e633e",
      "name": "9204c2aa-afaa-4aec-929c-673c0f6e633e",
      "status": "Failed",
      "startTime": "2026-08-18T03:06:29.321278577Z",
      "endTime": "2026-08-18T03:25:37.629557591Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/oidc_issuer_workload_identity.go:217]: failed to create HCP cluster oidc-wi-cluster
Unexpected error:
    <*fmt.wrapError | 0xc001454fa0>: 
    failed to create HCP cluster oidc-wi-cluster: failed waiting for cluster="oidc-wi-cluster" in resourcegroup="oidc-wi-rlpxm5hnr4qk" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/9204c2aa-afaa-4aec-929c-673c0f6e633e
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/9204c2aa-afaa-4aec-929c-673c0f6e633e",
      "name": "9204c2aa-afaa-4aec-929c-673c0f6e633e",
      "status": "Failed",
      "startTime": "2026-08-18T03:06:29.321278577Z",
      "endTime": "2026-08-18T03:25:37.629557591Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift

01:17:15 UTC · failed at e2e · test Customer should not be able to perform various invalid operations on cluster resources · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/simple_negative_cases.go:110]: failed to create HCP cluster negative-tests-cluster
Unexpected error:
    <*fmt.wrapError | 0xc000881e40>: 
    failed to create HCP cluster negative-tests-cluster: failed waiting for cluster="negative-tests-cluster" in resourcegroup="rg-negative-tests-55jnpgdk2xqq" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/d2316f39-ccb1-485c-bc80-ad87973b6715
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/d2316f39-ccb1-485c-bc80-ad87973b6715",
      "name": "d2316f39-ccb1-485c-bc80-ad87973b6715",
      "status": "Failed",
      "startTime": "2026-08-18T03:07:26.053560789Z",
      "endTime": "2026-08-18T03:26:27.648974782Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/simple_negative_cases.go:110]: failed to create HCP cluster negative-tests-cluster
Unexpected error:
    <*fmt.wrapError | 0xc000881e40>: 
    failed to create HCP cluster negative-tests-cluster: failed waiting for cluster="negative-tests-cluster" in resourcegroup="rg-negative-tests-55jnpgdk2xqq" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/d2316f39-ccb1-485c-bc80-ad87973b6715
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/d2316f39-ccb1-485c-bc80-ad87973b6715",
      "name": "d2316f39-ccb1-485c-bc80-ad87973b6715",
      "status": "Failed",
      "startTime": "2026-08-18T03:07:26.053560789Z",
      "endTime": "2026-08-18T03:26:27.648974782Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

timeout during CreateHCPClusterAndWait; context deadline exceeded

01:17:15 UTC · failed at e2e · test Customer should be able to create a no-CNI private cluster with a private key vault, a nodepool and install cilium CNI successfully · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:106]: failed to create HCP cluster "cilium-cluster" with no CNI and private etcd
Unexpected error:
    <*fmt.wrapErrors | 0xc00127f380>: 
    failed waiting for cluster="cilium-cluster" in resourcegroup="complex-cilium-kv-nz8jdkqt97lt" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223AndWait for cluster cilium-cluster in resource group complex-cilium-kv-nz8jdkqt97lt, error: context deadline exceeded
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:106]: failed to create HCP cluster "cilium-cluster" with no CNI and private etcd
Unexpected error:
    <*fmt.wrapErrors | 0xc00127f380>: 
    failed waiting for cluster="cilium-cluster" in resourcegroup="complex-cilium-kv-nz8jdkqt97lt" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223AndWait for cluster cilium-cluster in resource group complex-cilium-kv-nz8jdkqt97lt, error: context deadline exceeded
    ...
occurred

01:17:15 UTC · failed at e2e · test Create HCPOpenShiftCluster with Private KeyVault should create a cluster with private keyvault using v20251223preview API · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:98]: failed to create HCP cluster "private-kv-cluster" with private keyvault
Unexpected error:
    <*fmt.wrapError | 0xc0008c74a0>: 
    failed waiting for cluster="private-kv-cluster" in resourcegroup="private-keyvault-klf4n4fvlz26" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/d5fa142d-cb72-43e2-a5ac-bdaf2c8ab514
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/d5fa142d-cb72-43e2-a5ac-bdaf2c8ab514",
      "name": "d5fa142d-cb72-43e2-a5ac-bdaf2c8ab514",
      "status": "Failed",
      "startTime": "2026-08-18T01:35:55.986111403Z",
      "endTime": "2026-08-18T01:55:04.389818569Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:98]: failed to create HCP cluster "private-kv-cluster" with private keyvault
Unexpected error:
    <*fmt.wrapError | 0xc0008c74a0>: 
    failed waiting for cluster="private-kv-cluster" in resourcegroup="private-keyvault-klf4n4fvlz26" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/d5fa142d-cb72-43e2-a5ac-bdaf2c8ab514
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/d5fa142d-cb72-43e2-a5ac-bdaf2c8ab514",
      "name": "d5fa142d-cb72-43e2-a5ac-bdaf2c8ab514",
      "status": "Failed",
      "startTime": "2026-08-18T01:35:55.986111403Z",
      "endTime": "2026-08-18T01:55:04.389818569Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift

01:17:15 UTC · failed at e2e · test Nodepool Ephemeral OS Disk should create a nodepool with ephemeral OS disk when autoRepair is enabled · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_ephemeral_osdisk.go:98]: failed to create HCP cluster ephemeral-disk
Unexpected error:
    <*fmt.wrapError | 0xc000a90600>: 
    failed to create HCP cluster ephemeral-disk: failed waiting for cluster="ephemeral-disk" in resourcegroup="ephemeral-osdisk-dskdjvfx5j7t" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/0b9c0bcd-a92a-43db-a991-ea9328e1f923
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/0b9c0bcd-a92a-43db-a991-ea9328e1f923",
      "name": "0b9c0bcd-a92a-43db-a991-ea9328e1f923",
      "status": "Failed",
      "startTime": "2026-08-18T02:25:33.035856103Z",
      "endTime": "2026-08-18T02:44:36.584407006Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_ephemeral_osdisk.go:98]: failed to create HCP cluster ephemeral-disk
Unexpected error:
    <*fmt.wrapError | 0xc000a90600>: 
    failed to create HCP cluster ephemeral-disk: failed waiting for cluster="ephemeral-disk" in resourcegroup="ephemeral-osdisk-dskdjvfx5j7t" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/0b9c0bcd-a92a-43db-a991-ea9328e1f923
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/0b9c0bcd-a92a-43db-a991-ea9328e1f923",
      "name": "0b9c0bcd-a92a-43db-a991-ea9328e1f923",
      "status": "Failed",
      "startTime": "2026-08-18T02:25:33.035856103Z",
      "endTime": "2026-08-18T02:44:36.584407006Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift

01:17:15 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 4.20 · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:110]: failed to create HCP cluster "cluster-zstream-4-20-jrsvwm" with version 4.20.33 on candidate channel
Unexpected error:
    <*fmt.wrapError | 0xc000fe6440>: 
    failed to create HCP cluster cluster-zstream-4-20-jrsvwm: failed waiting for cluster="cluster-zstream-4-20-jrsvwm" in resourcegroup="rg-zstream-upgrade-4-20-qpmcvtcvj9dv" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/1687c7d6-bf31-410c-8390-111504ab636c
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/1687c7d6-bf31-410c-8390-111504ab636c",
      "name": "1687c7d6-bf31-410c-8390-111504ab636c",
      "status": "Failed",
      "startTime": "2026-08-18T02:22:59.495869545Z",
      "endTime": "2026-08-18T02:42:05.746097106Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:110]: failed to create HCP cluster "cluster-zstream-4-20-jrsvwm" with version 4.20.33 on candidate channel
Unexpected error:
    <*fmt.wrapError | 0xc000fe6440>: 
    failed to create HCP cluster cluster-zstream-4-20-jrsvwm: failed waiting for cluster="cluster-zstream-4-20-jrsvwm" in resourcegroup="rg-zstream-upgrade-4-20-qpmcvtcvj9dv" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/1687c7d6-bf31-410c-8390-111504ab636c
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/1687c7d6-bf31-410c-8390-111504ab636c",
      "name": "1687c7d6-bf31-410c-8390-111504ab636c",
      "status": "Failed",
      "startTime": "2026-08-18T02:22:59.495869545Z",
      "endTime": "2026-08-18T02:42:05.746097106Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift

01:17:15 UTC · failed at e2e · test Customer should not be able to reuse subnets and NSGs between clusters · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_nsg_subnet_reuse.go:107]: failed to create seed HCP cluster basic-cluster
Unexpected error:
    <*fmt.wrapError | 0xc0008b94a0>: 
    failed to create HCP cluster basic-cluster: failed waiting for cluster="basic-cluster" in resourcegroup="rg-cluster-nsg-subnet-reuse-t67ltk24vhhl" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/4b27045c-87f0-45f9-bb70-024ec2e52cbe
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/4b27045c-87f0-45f9-bb70-024ec2e52cbe",
      "name": "4b27045c-87f0-45f9-bb70-024ec2e52cbe",
      "status": "Failed",
      "startTime": "2026-08-18T01:35:16.063344861Z",
      "endTime": "2026-08-18T01:54:24.672728151Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_nsg_subnet_reuse.go:107]: failed to create seed HCP cluster basic-cluster
Unexpected error:
    <*fmt.wrapError | 0xc0008b94a0>: 
    failed to create HCP cluster basic-cluster: failed waiting for cluster="basic-cluster" in resourcegroup="rg-cluster-nsg-subnet-reuse-t67ltk24vhhl" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/4b27045c-87f0-45f9-bb70-024ec2e52cbe
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/4b27045c-87f0-45f9-bb70-024ec2e52cbe",
      "name": "4b27045c-87f0-45f9-bb70-024ec2e52cbe",
      "status": "Failed",
      "startTime": "2026-08-18T01:35:16.063344861Z",
      "endTime": "2026-08-18T01:54:24.672728151Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

01:17:15 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.20 · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:107]: HCP cluster rg-candidate-4-20-hgtq5vkjmtng/cluster-candidate-4-20-7ntjrh should provision
Unexpected error:
    <*fmt.wrapError | 0xc0005ac5a0>: 
    failed to create HCP cluster cluster-candidate-4-20-7ntjrh: failed waiting for cluster="cluster-candidate-4-20-7ntjrh" in resourcegroup="rg-candidate-4-20-hgtq5vkjmtng" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/7498cc37-bb15-4954-891a-544689c57fd9
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/7498cc37-bb15-4954-891a-544689c57fd9",
      "name": "7498cc37-bb15-4954-891a-544689c57fd9",
      "status": "Failed",
      "startTime": "2026-08-18T02:20:22.991368241Z",
      "endTime": "2026-08-18T02:39:25.761048408Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:107]: HCP cluster rg-candidate-4-20-hgtq5vkjmtng/cluster-candidate-4-20-7ntjrh should provision
Unexpected error:
    <*fmt.wrapError | 0xc0005ac5a0>: 
    failed to create HCP cluster cluster-candidate-4-20-7ntjrh: failed waiting for cluster="cluster-candidate-4-20-7ntjrh" in resourcegroup="rg-candidate-4-20-hgtq5vkjmtng" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/7498cc37-bb15-4954-891a-544689c57fd9
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/7498cc37-bb15-4954-891a-544689c57fd9",
      "name": "7498cc37-bb15-4954-891a-544689c57fd9",
      "status": "Failed",
      "startTime": "2026-08-18T02:20:22.991368241Z",
      "endTime": "2026-08-18T02:39:25.761048408Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift

01:17:15 UTC · failed at e2e · test Customer should upgrade and update a nodepool from 4.20.z to 4.21.zLatest · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:133]: failed to create HCP cluster np-version-upgrade-cluster-ktwcf4 with version 4.21.29
Unexpected error:
    <*fmt.wrapError | 0xc001088e20>: 
    failed to create HCP cluster np-version-upgrade-cluster-ktwcf4: failed waiting for cluster="np-version-upgrade-cluster-ktwcf4" in resourcegroup="rg-np-version-upgrade-ktwcf4-k458jgrddqcq" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/b4c0e4e5-8627-4490-88d5-2632a2a2f8fc
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/b4c0e4e5-8627-4490-88d5-2632a2a2f8fc",
      "name": "b4c0e4e5-8627-4490-88d5-2632a2a2f8fc",
      "status": "Failed",
      "startTime": "2026-08-18T03:05:31.516468484Z",
      "endTime": "2026-08-18T03:24:37.697618815Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:133]: failed to create HCP cluster np-version-upgrade-cluster-ktwcf4 with version 4.21.29
Unexpected error:
    <*fmt.wrapError | 0xc001088e20>: 
    failed to create HCP cluster np-version-upgrade-cluster-ktwcf4: failed waiting for cluster="np-version-upgrade-cluster-ktwcf4" in resourcegroup="rg-np-version-upgrade-ktwcf4-k458jgrddqcq" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/b4c0e4e5-8627-4490-88d5-2632a2a2f8fc
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/b4c0e4e5-8627-4490-88d5-2632a2a2f8fc",
      "name": "b4c0e4e5-8627-4490-88d5-2632a2a2f8fc",
      "status": "Failed",
      "startTime": "2026-08-18T03:05:31.516468484Z",
      "endTime": "2026-08-18T03:24:37.697618815Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

cluster entered terminal Failed state after role assignment deployment

01:17:15 UTC · failed at e2e · test ARO HCP Service should successfully provision cluster when MI role assignments are added after cluster creation · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:220]: Timed out after 2520.000s.
cluster should eventually succeed after role assignments are created
The function passed to Eventually failed at /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:216 with:
cluster entered terminal Failed state after role assignment deployment
Expected
    <armredhatopenshifthcp.ProvisioningState>: ...
not to equal
    <armredhatopenshifthcp.ProvisioningState>: ...
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:220]: Timed out after 2520.000s.
cluster should eventually succeed after role assignments are created
The function passed to Eventually failed at /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:216 with:
cluster entered terminal Failed state after role assignment deployment
Expected
    <armredhatopenshifthcp.ProvisioningState>: ...
not to equal
    <armredhatopenshifthcp.ProvisioningState>: ...

01:17:15 UTC · failed at e2e · test Customer should create a cluster with private ingress using v20260630preview and verify the ingress is internal · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_ingress.go:103]: failed to create HCP cluster "private-ingress" with private ingress
Unexpected error:
    <*fmt.wrapError | 0xc0008bdf60>: 
    failed to create HCP cluster private-ingress: failed waiting for cluster="private-ingress" in resourcegroup="private-ingress-nctgpthtz9f4" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/af1fee24-3049-47ef-b920-97189cf8fcee
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/af1fee24-3049-47ef-b920-97189cf8fcee",
      "name": "af1fee24-3049-47ef-b920-97189cf8fcee",
      "status": "Failed",
      "startTime": "2026-08-18T01:34:08.580882055Z",
      "endTime": "2026-08-18T01:53:14.490660293Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_ingress.go:103]: failed to create HCP cluster "private-ingress" with private ingress
Unexpected error:
    <*fmt.wrapError | 0xc0008bdf60>: 
    failed to create HCP cluster private-ingress: failed waiting for cluster="private-ingress" in resourcegroup="private-ingress-nctgpthtz9f4" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/af1fee24-3049-47ef-b920-97189cf8fcee
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/af1fee24-3049-47ef-b920-97189cf8fcee",
      "name": "af1fee24-3049-47ef-b920-97189cf8fcee",
      "status": "Failed",
      "startTime": "2026-08-18T01:34:08.580882055Z",
      "endTime": "2026-08-18T01:53:14.490660293Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift

01:17:15 UTC · failed at e2e · test Customer should downgrade a nodepool to a lower minor version from 4.21.zLatest to 4.19.zLatest · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:741]: failed to create HCP cluster
Unexpected error:
    <*fmt.wrapError | 0xc000facbc0>: 
    failed to create HCP cluster np-dg-minor-szpgrq: failed waiting for cluster="np-dg-minor-szpgrq" in resourcegroup="rg-np-dg-minor-szpgrq-t9m426rffsgp" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/3db25e9a-b224-465f-9d23-7922b114d810
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/3db25e9a-b224-465f-9d23-7922b114d810",
      "name": "3db25e9a-b224-465f-9d23-7922b114d810",
      "status": "Failed",
      "startTime": "2026-08-18T03:06:49.429771196Z",
      "endTime": "2026-08-18T03:25:57.711979405Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:741]: failed to create HCP cluster
Unexpected error:
    <*fmt.wrapError | 0xc000facbc0>: 
    failed to create HCP cluster np-dg-minor-szpgrq: failed waiting for cluster="np-dg-minor-szpgrq" in resourcegroup="rg-np-dg-minor-szpgrq-t9m426rffsgp" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/3db25e9a-b224-465f-9d23-7922b114d810
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/3db25e9a-b224-465f-9d23-7922b114d810",
      "name": "3db25e9a-b224-465f-9d23-7922b114d810",
      "status": "Failed",
      "startTime": "2026-08-18T03:06:49.429771196Z",
      "endTime": "2026-08-18T03:25:57.711979405Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

timeout during CreateHCPClusterAndWait; context deadline exceeded

01:17:15 UTC · failed at e2e · test Customer should be able to create an HCP cluster with back-level version 4.19 · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:151]: failed to create HCP cluster version 4.19
Unexpected error:
    <*fmt.wrapErrors | 0xc0013d8240>: 
    failed waiting for cluster="cluster-ver-4-19" in resourcegroup="rg-cluster-back-version-qfcrpn6hrh9n" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterAndWait for cluster cluster-ver-4-19 in resource group rg-cluster-back-version-qfcrpn6hrh9n, error: context deadline exceeded
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:151]: failed to create HCP cluster version 4.19
Unexpected error:
    <*fmt.wrapErrors | 0xc0013d8240>: 
    failed waiting for cluster="cluster-ver-4-19" in resourcegroup="rg-cluster-back-version-qfcrpn6hrh9n" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterAndWait for cluster cluster-ver-4-19 in resource group rg-cluster-back-version-qfcrpn6hrh9n, error: context deadline exceeded
    ...
occurred

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift

01:17:15 UTC · failed at e2e · test Customer should be able to create several HCP clusters in their customer resource group, but not in the same managed resource group · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:131]: failed to wait for first cluster "basic-hcp-cluster" to complete creation (timeout '20.000000' minutes)
Unexpected error:
    <*exported.ResponseError | 0xc00074cf00>: 
    GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/14b9d201-ed20-4b0a-9a6e-e010ba8f28af
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/14b9d201-ed20-4b0a-9a6e-e010ba8f28af",
      "name": "14b9d201-ed20-4b0a-9a6e-e010ba8f28af",
      "status": "Failed",
      "startTime": "2026-08-18T01:35:11.854338314Z",
      "endTime": "2026-08-18T01:54:14.528632911Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:131]: failed to wait for first cluster "basic-hcp-cluster" to complete creation (timeout '20.000000' minutes)
Unexpected error:
    <*exported.ResponseError | 0xc00074cf00>: 
    GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/14b9d201-ed20-4b0a-9a6e-e010ba8f28af
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/14b9d201-ed20-4b0a-9a6e-e010ba8f28af",
      "name": "14b9d201-ed20-4b0a-9a6e-e010ba8f28af",
      "status": "Failed",
      "startTime": "2026-08-18T01:35:11.854338314Z",
      "endTime": "2026-08-18T01:54:14.528632911Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift

01:17:15 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 4.21 · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:110]: failed to create HCP cluster "cluster-zstream-4-21-4rw7qw" with version 4.21.28 on candidate channel
Unexpected error:
    <*fmt.wrapError | 0xc000e09420>: 
    failed to create HCP cluster cluster-zstream-4-21-4rw7qw: failed waiting for cluster="cluster-zstream-4-21-4rw7qw" in resourcegroup="rg-zstream-upgrade-4-21-6st9sr5lfd6j" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/328eb91e-47ee-4dae-98a5-c5c57ddda57d
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/328eb91e-47ee-4dae-98a5-c5c57ddda57d",
      "name": "328eb91e-47ee-4dae-98a5-c5c57ddda57d",
      "status": "Failed",
      "startTime": "2026-08-18T02:20:42.085524691Z",
      "endTime": "2026-08-18T02:39:45.847398158Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:110]: failed to create HCP cluster "cluster-zstream-4-21-4rw7qw" with version 4.21.28 on candidate channel
Unexpected error:
    <*fmt.wrapError | 0xc000e09420>: 
    failed to create HCP cluster cluster-zstream-4-21-4rw7qw: failed waiting for cluster="cluster-zstream-4-21-4rw7qw" in resourcegroup="rg-zstream-upgrade-4-21-6st9sr5lfd6j" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/328eb91e-47ee-4dae-98a5-c5c57ddda57d
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/328eb91e-47ee-4dae-98a5-c5c57ddda57d",
      "name": "328eb91e-47ee-4dae-98a5-c5c57ddda57d",
      "status": "Failed",
      "startTime": "2026-08-18T02:20:42.085524691Z",
      "endTime": "2026-08-18T02:39:45.847398158Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift

01:17:15 UTC · failed at e2e · test Customer should be able to successfully upgrade control plane minor version from 4.21 minor to 4.22 minor · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:182]: failed to create HCP cluster "cp-ystream-upgrade-4-22-h482f9" with install version 4.21.28
Unexpected error:
    <*fmt.wrapError | 0xc0004aca80>: 
    failed to create HCP cluster cp-ystream-upgrade-4-22-h482f9: failed waiting for cluster="cp-ystream-upgrade-4-22-h482f9" in resourcegroup="rg-cp-ystream-upgrade-4-22-cns7sgjvqbjf" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/12192dc8-297c-457e-9c26-cdda4e94430f
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/12192dc8-297c-457e-9c26-cdda4e94430f",
      "name": "12192dc8-297c-457e-9c26-cdda4e94430f",
      "status": "Failed",
      "startTime": "2026-08-18T02:21:25.103882504Z",
      "endTime": "2026-08-18T02:40:25.795531616Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:182]: failed to create HCP cluster "cp-ystream-upgrade-4-22-h482f9" with install version 4.21.28
Unexpected error:
    <*fmt.wrapError | 0xc0004aca80>: 
    failed to create HCP cluster cp-ystream-upgrade-4-22-h482f9: failed waiting for cluster="cp-ystream-upgrade-4-22-h482f9" in resourcegroup="rg-cp-ystream-upgrade-4-22-cns7sgjvqbjf" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/12192dc8-297c-457e-9c26-cdda4e94430f
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/12192dc8-297c-457e-9c26-cdda4e94430f",
      "name": "12192dc8-297c-457e-9c26-cdda4e94430f",
      "status": "Failed",
      "startTime": "2026-08-18T02:21:25.103882504Z",
      "endTime": "2026-08-18T02:40:25.795531616Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift

01:17:15 UTC · failed at e2e · test Customer should upgrade and update a nodepool from 4.21.z to 4.21.zLatest · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:133]: failed to create HCP cluster np-version-upgrade-cluster-phxfbm with version 4.21.29
Unexpected error:
    <*fmt.wrapError | 0xc001092ce0>: 
    failed to create HCP cluster np-version-upgrade-cluster-phxfbm: failed waiting for cluster="np-version-upgrade-cluster-phxfbm" in resourcegroup="rg-np-version-upgrade-phxfbm-v96mg7kkv48z" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/3af2530c-5f72-43c5-8c8d-ed0a7e8448bd
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/3af2530c-5f72-43c5-8c8d-ed0a7e8448bd",
      "name": "3af2530c-5f72-43c5-8c8d-ed0a7e8448bd",
      "status": "Failed",
      "startTime": "2026-08-18T03:05:52.501093606Z",
      "endTime": "2026-08-18T03:24:57.820442571Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:133]: failed to create HCP cluster np-version-upgrade-cluster-phxfbm with version 4.21.29
Unexpected error:
    <*fmt.wrapError | 0xc001092ce0>: 
    failed to create HCP cluster np-version-upgrade-cluster-phxfbm: failed waiting for cluster="np-version-upgrade-cluster-phxfbm" in resourcegroup="rg-np-version-upgrade-phxfbm-v96mg7kkv48z" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/3af2530c-5f72-43c5-8c8d-ed0a7e8448bd
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/3af2530c-5f72-43c5-8c8d-ed0a7e8448bd",
      "name": "3af2530c-5f72-43c5-8c8d-ed0a7e8448bd",
      "status": "Failed",
      "startTime": "2026-08-18T03:05:52.501093606Z",
      "endTime": "2026-08-18T03:24:57.820442571Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift

01:17:15 UTC · failed at e2e · test Customer should upgrade and update a nodepool from 4.20.z to 4.20.zLatest · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:133]: failed to create HCP cluster np-version-upgrade-cluster-5stxsv with version 4.20.34
Unexpected error:
    <*fmt.wrapError | 0xc000902a00>: 
    failed to create HCP cluster np-version-upgrade-cluster-5stxsv: failed waiting for cluster="np-version-upgrade-cluster-5stxsv" in resourcegroup="rg-np-version-upgrade-5stxsv-pd9lkcxvmsqc" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/23bca7d1-d34e-45cb-bcf2-aa6a76f71651
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/23bca7d1-d34e-45cb-bcf2-aa6a76f71651",
      "name": "23bca7d1-d34e-45cb-bcf2-aa6a76f71651",
      "status": "Failed",
      "startTime": "2026-08-18T03:08:57.14601964Z",
      "endTime": "2026-08-18T03:27:57.63550179Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:133]: failed to create HCP cluster np-version-upgrade-cluster-5stxsv with version 4.20.34
Unexpected error:
    <*fmt.wrapError | 0xc000902a00>: 
    failed to create HCP cluster np-version-upgrade-cluster-5stxsv: failed waiting for cluster="np-version-upgrade-cluster-5stxsv" in resourcegroup="rg-np-version-upgrade-5stxsv-pd9lkcxvmsqc" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/23bca7d1-d34e-45cb-bcf2-aa6a76f71651
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/23bca7d1-d34e-45cb-bcf2-aa6a76f71651",
      "name": "23bca7d1-d34e-45cb-bcf2-aa6a76f71651",
      "status": "Failed",
      "startTime": "2026-08-18T03:08:57.14601964Z",
      "endTime": "2026-08-18T03:27:57.63550179Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift

01:17:15 UTC · failed at e2e · test Customer should respect cluster-wide node limits with nodepool autoscaling · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:291]: failed to create HCP cluster node-limit-cluster with MaxNodesTotal limit
Unexpected error:
    <*fmt.wrapError | 0xc00150c400>: 
    failed to create HCP cluster node-limit-cluster: failed waiting for cluster="node-limit-cluster" in resourcegroup="node-limit-test-xsc659bqwsjv" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/ee89d445-6931-48de-beed-2caf5b9d9058
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/ee89d445-6931-48de-beed-2caf5b9d9058",
      "name": "ee89d445-6931-48de-beed-2caf5b9d9058",
      "status": "Failed",
      "startTime": "2026-08-18T02:26:39.318377569Z",
      "endTime": "2026-08-18T02:45:46.637467319Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:291]: failed to create HCP cluster node-limit-cluster with MaxNodesTotal limit
Unexpected error:
    <*fmt.wrapError | 0xc00150c400>: 
    failed to create HCP cluster node-limit-cluster: failed waiting for cluster="node-limit-cluster" in resourcegroup="node-limit-test-xsc659bqwsjv" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/ee89d445-6931-48de-beed-2caf5b9d9058
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/ee89d445-6931-48de-beed-2caf5b9d9058",
      "name": "ee89d445-6931-48de-beed-2caf5b9d9058",
      "status": "Failed",
      "startTime": "2026-08-18T02:26:39.318377569Z",
      "endTime": "2026-08-18T02:45:46.637467319Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

timeout during CreateHCPClusterAndWait; context deadline exceeded

01:17:15 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.22 · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:107]: HCP cluster rg-candidate-4-22-9rh5s6cz8tvc/cluster-candidate-4-22-cxtdzk should provision
Unexpected error:
    <*fmt.wrapErrors | 0xc00180c810>: 
    failed to create HCP cluster cluster-candidate-4-22-cxtdzk, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster cluster-candidate-4-22-cxtdzk in resource group rg-candidate-4-22-9rh5s6cz8tvc, error: failed waiting for cluster="cluster-candidate-4-22-cxtdzk" in resourcegroup="rg-candidate-4-22-9rh5s6cz8tvc" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster cluster-candidate-4-22-cxtdzk in resource group rg-candidate-4-22-9rh5s6cz8tvc, error: context deadline exceeded
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:107]: HCP cluster rg-candidate-4-22-9rh5s6cz8tvc/cluster-candidate-4-22-cxtdzk should provision
Unexpected error:
    <*fmt.wrapErrors | 0xc00180c810>: 
    failed to create HCP cluster cluster-candidate-4-22-cxtdzk, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster cluster-candidate-4-22-cxtdzk in resource group rg-candidate-4-22-9rh5s6cz8tvc, error: failed waiting for cluster="cluster-candidate-4-22-cxtdzk" in resourcegroup="rg-candidate-4-22-9rh5s6cz8tvc" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster cluster-candidate-4-22-cxtdzk in resource group rg-candidate-4-22-9rh5s6cz8tvc, error: context deadline exceeded
    ...
occurred

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift

01:17:15 UTC · failed at e2e · test Update HCPOpenShiftCluster Positive creates a cluster and updates tags with a PATCH request · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:161]: failed to create HCP cluster for patch-tags test
Unexpected error:
    <*fmt.wrapError | 0xc00141ef60>: 
    failed to create HCP cluster patch-tags-cluster: failed waiting for cluster="patch-tags-cluster" in resourcegroup="patch-tags-hx8xlpfzk9mg" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/1009cfae-4b95-4d7c-84aa-7bd528b272f2
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/1009cfae-4b95-4d7c-84aa-7bd528b272f2",
      "name": "1009cfae-4b95-4d7c-84aa-7bd528b272f2",
      "status": "Failed",
      "startTime": "2026-08-18T01:39:31.881688747Z",
      "endTime": "2026-08-18T01:58:34.438297863Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:161]: failed to create HCP cluster for patch-tags test
Unexpected error:
    <*fmt.wrapError | 0xc00141ef60>: 
    failed to create HCP cluster patch-tags-cluster: failed waiting for cluster="patch-tags-cluster" in resourcegroup="patch-tags-hx8xlpfzk9mg" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/1009cfae-4b95-4d7c-84aa-7bd528b272f2
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/1009cfae-4b95-4d7c-84aa-7bd528b272f2",
      "name": "1009cfae-4b95-4d7c-84aa-7bd528b272f2",
      "status": "Failed",
      "startTime": "2026-08-18T01:39:31.881688747Z",
      "endTime": "2026-08-18T01:58:34.438297863Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

01:17:15 UTC · failed at e2e · test FIPS Mode Support with v2026-06-30 API should create an HCP cluster with FIPS mode enabled via cryptoRestrictions property · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_fips_mode.go:106]: failed to create HCP cluster "fips-enabled-cluster" with cryptoRestrictions set to FIPS
Unexpected error:
    <*fmt.wrapError | 0xc000eec640>: 
    failed waiting for cluster="fips-enabled-cluster" in resourcegroup="fips-enabled-js6s9lhv9qhv" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/82a24a4c-d210-4a62-8673-c0f5a51b1448
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/82a24a4c-d210-4a62-8673-c0f5a51b1448",
      "name": "82a24a4c-d210-4a62-8673-c0f5a51b1448",
      "status": "Failed",
      "startTime": "2026-08-18T01:34:47.523636047Z",
      "endTime": "2026-08-18T01:53:54.569768124Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_fips_mode.go:106]: failed to create HCP cluster "fips-enabled-cluster" with cryptoRestrictions set to FIPS
Unexpected error:
    <*fmt.wrapError | 0xc000eec640>: 
    failed waiting for cluster="fips-enabled-cluster" in resourcegroup="fips-enabled-js6s9lhv9qhv" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/82a24a4c-d210-4a62-8673-c0f5a51b1448
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/82a24a4c-d210-4a62-8673-c0f5a51b1448",
      "name": "82a24a4c-d210-4a62-8673-c0f5a51b1448",
      "status": "Failed",
      "startTime": "2026-08-18T01:34:47.523636047Z",
      "endTime": "2026-08-18T01:53:54.569768124Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift

01:17:15 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 4.22 · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:110]: failed to create HCP cluster "cluster-zstream-4-22-xztxk8" with version 4.22.8 on candidate channel
Unexpected error:
    <*fmt.wrapError | 0xc000e90a00>: 
    failed to create HCP cluster cluster-zstream-4-22-xztxk8: failed waiting for cluster="cluster-zstream-4-22-xztxk8" in resourcegroup="rg-zstream-upgrade-4-22-9n47t7ndzgk9" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/386cfeca-2d47-4438-8194-e04e52779c85
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/386cfeca-2d47-4438-8194-e04e52779c85",
      "name": "386cfeca-2d47-4438-8194-e04e52779c85",
      "status": "Failed",
      "startTime": "2026-08-18T02:22:48.738999859Z",
      "endTime": "2026-08-18T02:41:55.833760991Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:110]: failed to create HCP cluster "cluster-zstream-4-22-xztxk8" with version 4.22.8 on candidate channel
Unexpected error:
    <*fmt.wrapError | 0xc000e90a00>: 
    failed to create HCP cluster cluster-zstream-4-22-xztxk8: failed waiting for cluster="cluster-zstream-4-22-xztxk8" in resourcegroup="rg-zstream-upgrade-4-22-9n47t7ndzgk9" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/386cfeca-2d47-4438-8194-e04e52779c85
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/386cfeca-2d47-4438-8194-e04e52779c85",
      "name": "386cfeca-2d47-4438-8194-e04e52779c85",
      "status": "Failed",
      "startTime": "2026-08-18T02:22:48.738999859Z",
      "endTime": "2026-08-18T02:41:55.833760991Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift

01:17:15 UTC · failed at e2e · test Customer should be able to create an HCP cluster with Image Registry not present · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/image_registry_cluster_create.go:89]: failed to create HCP cluster with image registry disabled
Unexpected error:
    <*fmt.wrapError | 0xc000dd70c0>: 
    failed to create HCP cluster disabled-image-registry-hcp-cluster: failed waiting for cluster="disabled-image-registry-hcp-cluster" in resourcegroup="disabled-image-registry-mxnr5h7mxxnk" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/29126f72-891d-444f-87f5-fdcc7d248d85
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/29126f72-891d-444f-87f5-fdcc7d248d85",
      "name": "29126f72-891d-444f-87f5-fdcc7d248d85",
      "status": "Failed",
      "startTime": "2026-08-18T02:22:43.089171061Z",
      "endTime": "2026-08-18T02:41:45.735845882Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/image_registry_cluster_create.go:89]: failed to create HCP cluster with image registry disabled
Unexpected error:
    <*fmt.wrapError | 0xc000dd70c0>: 
    failed to create HCP cluster disabled-image-registry-hcp-cluster: failed waiting for cluster="disabled-image-registry-hcp-cluster" in resourcegroup="disabled-image-registry-mxnr5h7mxxnk" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/29126f72-891d-444f-87f5-fdcc7d248d85
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/29126f72-891d-444f-87f5-fdcc7d248d85",
      "name": "29126f72-891d-444f-87f5-fdcc7d248d85",
      "status": "Failed",
      "startTime": "2026-08-18T02:22:43.089171061Z",
      "endTime": "2026-08-18T02:41:45.735845882Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift

01:17:15 UTC · failed at e2e · test Customer should be able to create a cluster with default autoscaling and a nodepool with autoscaling enabled up to replica limits · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:102]: failed to create HCP cluster np-autoscale-cluster
Unexpected error:
    <*fmt.wrapError | 0xc000efa660>: 
    failed to create HCP cluster np-autoscale-cluster: failed waiting for cluster="np-autoscale-cluster" in resourcegroup="np-autoscaling-j45nlzgmgwsl" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/1522b984-8738-4ce3-a94c-ff57b760074b
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/1522b984-8738-4ce3-a94c-ff57b760074b",
      "name": "1522b984-8738-4ce3-a94c-ff57b760074b",
      "status": "Failed",
      "startTime": "2026-08-18T02:23:15.108288438Z",
      "endTime": "2026-08-18T02:42:15.72613301Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:102]: failed to create HCP cluster np-autoscale-cluster
Unexpected error:
    <*fmt.wrapError | 0xc000efa660>: 
    failed to create HCP cluster np-autoscale-cluster: failed waiting for cluster="np-autoscale-cluster" in resourcegroup="np-autoscaling-j45nlzgmgwsl" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/1522b984-8738-4ce3-a94c-ff57b760074b
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/1522b984-8738-4ce3-a94c-ff57b760074b",
      "name": "1522b984-8738-4ce3-a94c-ff57b760074b",
      "status": "Failed",
      "startTime": "2026-08-18T02:23:15.108288438Z",
      "endTime": "2026-08-18T02:42:15.72613301Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred