CIHealth Run Log

↗ Open Failure patterns for this day

Environment: PROD

Time (UTC)	Job	Failed at	Result	PR	Failed tests	Details
23:00:07	periodic-ci-Azure-ARO-HCP-main-periodic-prod-e2e-parallel-ocp-nightly post-goodmerged PR No additional run metadata captured	e2e	failed	n/a	1	e2e (1) ERROR CODE: InternalServerError; detail message [clusterServiceNodePoolStatus] <no_message>; provider Microsoft.RedHatOpenShift 23:00:07 UTC · failed at e2e · test Create HCPOpenShiftCluster with Private KeyVault should create a cluster with private keyvault using v20251223preview API · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:137]: failed to create node pool "np-1" for private keyvault cluster "private-kv-cluster" Unexpected error: <fmt.wrapError \| 0xc000d24a00>: failed to create NodePool np-1: failed waiting for nodepool="np-1" for cluster "private-kv-cluster" in resourcegroup="private-keyvault-9ngxf9zlhblm" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/3463e632-f100-45af-9f71-fc69cdf2dd4c -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/3463e632-f100-45af-9f71-fc69cdf2dd4c", "name": "3463e632-f100-45af-9f71-fc69cdf2dd4c", "status": "Failed", "startTime": "2026-08-02T23:16:20.627905758Z", "endTime": "2026-08-02T23:35:28.37866899Z", "error": { "code": "InternalServerError", "message": "[clusterServiceNodePoolStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:137]: failed to create node pool "np-1" for private keyvault cluster "private-kv-cluster" Unexpected error: <fmt.wrapError \| 0xc000d24a00>: failed to create NodePool np-1: failed waiting for nodepool="np-1" for cluster "private-kv-cluster" in resourcegroup="private-keyvault-9ngxf9zlhblm" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/3463e632-f100-45af-9f71-fc69cdf2dd4c -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/3463e632-f100-45af-9f71-fc69cdf2dd4c", "name": "3463e632-f100-45af-9f71-fc69cdf2dd4c", "status": "Failed", "startTime": "2026-08-02T23:16:20.627905758Z", "endTime": "2026-08-02T23:35:28.37866899Z", "error": { "code": "InternalServerError", "message": "[clusterServiceNodePoolStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred
09:15:40	branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel post-goodmerged PR No additional run metadata captured	e2e	failed	n/a	38	e2e (38) ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift 09:15:40 UTC · failed at e2e · test Customer should be able to create an HCP cluster then delete it by deleting the customer resource group · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delete_cx_rg.go:90]: failed to create HCP cluster "cx-rg-hcp-cluster" Unexpected error: <fmt.wrapError \| 0xc000d90900>: failed to create HCP cluster cx-rg-hcp-cluster: failed waiting for cluster="cx-rg-hcp-cluster" in resourcegroup="cx-rg-cluster-7tzmlfvcvjz2" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/8e7f6e9d-54e9-43d6-9057-7304d01b9584 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/8e7f6e9d-54e9-43d6-9057-7304d01b9584", "name": "8e7f6e9d-54e9-43d6-9057-7304d01b9584", "status": "Failed", "startTime": "2026-08-02T09:34:18.700863215Z", "endTime": "2026-08-02T09:53:19.940689234Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delete_cx_rg.go:90]: failed to create HCP cluster "cx-rg-hcp-cluster" Unexpected error: <fmt.wrapError \| 0xc000d90900>: failed to create HCP cluster cx-rg-hcp-cluster: failed waiting for cluster="cx-rg-hcp-cluster" in resourcegroup="cx-rg-cluster-7tzmlfvcvjz2" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/8e7f6e9d-54e9-43d6-9057-7304d01b9584 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/8e7f6e9d-54e9-43d6-9057-7304d01b9584", "name": "8e7f6e9d-54e9-43d6-9057-7304d01b9584", "status": "Failed", "startTime": "2026-08-02T09:34:18.700863215Z", "endTime": "2026-08-02T09:53:19.940689234Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift 09:15:40 UTC · failed at e2e · test Customer should be able to create node pools with ARM64-based VMs · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/arm64_nodepool.go:88]: failed to create HCP cluster "arm64-vm-hcp-cluster" Unexpected error: <fmt.wrapError \| 0xc0009404a0>: failed to create HCP cluster arm64-vm-hcp-cluster: failed waiting for cluster="arm64-vm-hcp-cluster" in resourcegroup="arm64-vm-cluster-4k4dbkr4s254" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/4c3c5ea4-015b-4faa-988b-ca92a5589f78 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/4c3c5ea4-015b-4faa-988b-ca92a5589f78", "name": "4c3c5ea4-015b-4faa-988b-ca92a5589f78", "status": "Failed", "startTime": "2026-08-02T09:34:01.642064318Z", "endTime": "2026-08-02T09:53:20.129747727Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/arm64_nodepool.go:88]: failed to create HCP cluster "arm64-vm-hcp-cluster" Unexpected error: <fmt.wrapError \| 0xc0009404a0>: failed to create HCP cluster arm64-vm-hcp-cluster: failed waiting for cluster="arm64-vm-hcp-cluster" in resourcegroup="arm64-vm-cluster-4k4dbkr4s254" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/4c3c5ea4-015b-4faa-988b-ca92a5589f78 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/4c3c5ea4-015b-4faa-988b-ca92a5589f78", "name": "4c3c5ea4-015b-4faa-988b-ca92a5589f78", "status": "Failed", "startTime": "2026-08-02T09:34:01.642064318Z", "endTime": "2026-08-02T09:53:20.129747727Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift 09:15:40 UTC · failed at e2e · test Customer should upgrade a nodepool skipping one minor version (+2) from 4.20.z to 4.22.zLatest · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:462]: failed to create HCP cluster np-skip-minor-hj85vs at version 4.22.8 Unexpected error: <fmt.wrapError \| 0xc000cd93c0>: failed to create HCP cluster np-skip-minor-hj85vs: failed waiting for cluster="np-skip-minor-hj85vs" in resourcegroup="rg-np-skip-minor-hj85vs-5sslvjsdhbbb" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/79102d9d-d8f7-4787-8796-65c7af655254 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/79102d9d-d8f7-4787-8796-65c7af655254", "name": "79102d9d-d8f7-4787-8796-65c7af655254", "status": "Failed", "startTime": "2026-08-02T10:46:52.48500524Z", "endTime": "2026-08-02T11:05:53.701127939Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:462]: failed to create HCP cluster np-skip-minor-hj85vs at version 4.22.8 Unexpected error: <fmt.wrapError \| 0xc000cd93c0>: failed to create HCP cluster np-skip-minor-hj85vs: failed waiting for cluster="np-skip-minor-hj85vs" in resourcegroup="rg-np-skip-minor-hj85vs-5sslvjsdhbbb" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/79102d9d-d8f7-4787-8796-65c7af655254 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/79102d9d-d8f7-4787-8796-65c7af655254", "name": "79102d9d-d8f7-4787-8796-65c7af655254", "status": "Failed", "startTime": "2026-08-02T10:46:52.48500524Z", "endTime": "2026-08-02T11:05:53.701127939Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift 09:15:40 UTC · failed at e2e · test Customer should be able to create an HCP cluster and manage pull secrets · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_pullsecret.go:114]: failed to create HCP cluster for pull secret test Unexpected error: <fmt.wrapError \| 0xc00099e6a0>: failed to create HCP cluster pullsecret-hcp-cluster: failed waiting for cluster="pullsecret-hcp-cluster" in resourcegroup="pullsecret-test-h88dk8ndppfm" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/6dbae615-f8d0-4154-ac6c-998d81f7b84d -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/6dbae615-f8d0-4154-ac6c-998d81f7b84d", "name": "6dbae615-f8d0-4154-ac6c-998d81f7b84d", "status": "Failed", "startTime": "2026-08-02T09:33:48.731562196Z", "endTime": "2026-08-02T09:52:59.947253927Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_pullsecret.go:114]: failed to create HCP cluster for pull secret test Unexpected error: <fmt.wrapError \| 0xc00099e6a0>: failed to create HCP cluster pullsecret-hcp-cluster: failed waiting for cluster="pullsecret-hcp-cluster" in resourcegroup="pullsecret-test-h88dk8ndppfm" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/6dbae615-f8d0-4154-ac6c-998d81f7b84d -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/6dbae615-f8d0-4154-ac6c-998d81f7b84d", "name": "6dbae615-f8d0-4154-ac6c-998d81f7b84d", "status": "Failed", "startTime": "2026-08-02T09:33:48.731562196Z", "endTime": "2026-08-02T09:52:59.947253927Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift 09:15:40 UTC · failed at e2e · test Customer should upgrade a nodepool to a version without Cincinnati upgrade edge z-stream upgrade without Cincinnati edge in 4.20 · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:314]: failed to create HCP cluster np-noedge-b59ll2 Unexpected error: <fmt.wrapError \| 0xc000ddd1e0>: failed to create HCP cluster np-noedge-b59ll2: failed waiting for cluster="np-noedge-b59ll2" in resourcegroup="rg-np-noedge-b59ll2-p7jkwbdc422s" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/1654762e-f7e5-4618-a61f-b73443f5f202 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/1654762e-f7e5-4618-a61f-b73443f5f202", "name": "1654762e-f7e5-4618-a61f-b73443f5f202", "status": "Failed", "startTime": "2026-08-02T10:26:18.049472465Z", "endTime": "2026-08-02T10:45:22.746370197Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:314]: failed to create HCP cluster np-noedge-b59ll2 Unexpected error: <fmt.wrapError \| 0xc000ddd1e0>: failed to create HCP cluster np-noedge-b59ll2: failed waiting for cluster="np-noedge-b59ll2" in resourcegroup="rg-np-noedge-b59ll2-p7jkwbdc422s" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/1654762e-f7e5-4618-a61f-b73443f5f202 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/1654762e-f7e5-4618-a61f-b73443f5f202", "name": "1654762e-f7e5-4618-a61f-b73443f5f202", "status": "Failed", "startTime": "2026-08-02T10:26:18.049472465Z", "endTime": "2026-08-02T10:45:22.746370197Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred cluster entered terminal Failed state after role assignment deployment 09:15:40 UTC · failed at e2e · test ARO HCP Service should successfully provision cluster when MI role assignments are added after cluster creation · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:220]: Timed out after 2520.001s. cluster should eventually succeed after role assignments are created The function passed to Eventually failed at /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:216 with: cluster entered terminal Failed state after role assignment deployment Expected <armredhatopenshifthcp.ProvisioningState>: ... not to equal <armredhatopenshifthcp.ProvisioningState>: ... fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:220]: Timed out after 2520.001s. cluster should eventually succeed after role assignments are created The function passed to Eventually failed at /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:216 with: cluster entered terminal Failed state after role assignment deployment Expected <armredhatopenshifthcp.ProvisioningState>: ... not to equal <armredhatopenshifthcp.ProvisioningState>: ... ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift 09:15:40 UTC · failed at e2e · test Authorized CIDRs Connectivity should allow API access only from authorized VM IP · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:151]: failed to create HCP cluster "cidr-connectivity-test" with authorized CIDRs Unexpected error: <fmt.wrapError \| 0xc0019052c0>: failed to create HCP cluster cidr-connectivity-test: failed waiting for cluster="cidr-connectivity-test" in resourcegroup="e2e-cidr-connectivity-wmrgqh8gzn96" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/0fbe47c0-96cd-4379-9f15-8327e095b80e -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/0fbe47c0-96cd-4379-9f15-8327e095b80e", "name": "0fbe47c0-96cd-4379-9f15-8327e095b80e", "status": "Failed", "startTime": "2026-08-02T09:34:41.172279489Z", "endTime": "2026-08-02T09:53:50.007487871Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:151]: failed to create HCP cluster "cidr-connectivity-test" with authorized CIDRs Unexpected error: <fmt.wrapError \| 0xc0019052c0>: failed to create HCP cluster cidr-connectivity-test: failed waiting for cluster="cidr-connectivity-test" in resourcegroup="e2e-cidr-connectivity-wmrgqh8gzn96" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/0fbe47c0-96cd-4379-9f15-8327e095b80e -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/0fbe47c0-96cd-4379-9f15-8327e095b80e", "name": "0fbe47c0-96cd-4379-9f15-8327e095b80e", "status": "Failed", "startTime": "2026-08-02T09:34:41.172279489Z", "endTime": "2026-08-02T09:53:50.007487871Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift 09:15:40 UTC · failed at e2e · test Customer should be able to create several HCP clusters in their customer resource group, but not in the same managed resource group · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:131]: failed to wait for first cluster "basic-hcp-cluster" to complete creation (timeout '20.000000' minutes) Unexpected error: <exported.ResponseError \| 0xc001475b00>: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/a3dde528-d0eb-458a-a5eb-c741c73cfea4 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/a3dde528-d0eb-458a-a5eb-c741c73cfea4", "name": "a3dde528-d0eb-458a-a5eb-c741c73cfea4", "status": "Failed", "startTime": "2026-08-02T09:35:34.710183543Z", "endTime": "2026-08-02T09:54:39.901296729Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:131]: failed to wait for first cluster "basic-hcp-cluster" to complete creation (timeout '20.000000' minutes) Unexpected error: <exported.ResponseError \| 0xc001475b00>: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/a3dde528-d0eb-458a-a5eb-c741c73cfea4 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/a3dde528-d0eb-458a-a5eb-c741c73cfea4", "name": "a3dde528-d0eb-458a-a5eb-c741c73cfea4", "status": "Failed", "startTime": "2026-08-02T09:35:34.710183543Z", "endTime": "2026-08-02T09:54:39.901296729Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift 09:15:40 UTC · failed at e2e · test Customer should be able to create an HCP cluster with custom autoscaling · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:98]: failed to create HCP cluster "autoscaling-hcp-cluster" with custom autoscaling Unexpected error: <fmt.wrapError \| 0xc000439100>: failed to create HCP cluster autoscaling-hcp-cluster: failed waiting for cluster="autoscaling-hcp-cluster" in resourcegroup="autoscaling-cluster-schbwv7s4vqj" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/433a98a2-2a48-4b1a-ad1a-2eb9672f77a2 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/433a98a2-2a48-4b1a-ad1a-2eb9672f77a2", "name": "433a98a2-2a48-4b1a-ad1a-2eb9672f77a2", "status": "Failed", "startTime": "2026-08-02T09:33:58.846876088Z", "endTime": "2026-08-02T09:53:00.063648031Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:98]: failed to create HCP cluster "autoscaling-hcp-cluster" with custom autoscaling Unexpected error: <fmt.wrapError \| 0xc000439100>: failed to create HCP cluster autoscaling-hcp-cluster: failed waiting for cluster="autoscaling-hcp-cluster" in resourcegroup="autoscaling-cluster-schbwv7s4vqj" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/433a98a2-2a48-4b1a-ad1a-2eb9672f77a2 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/433a98a2-2a48-4b1a-ad1a-2eb9672f77a2", "name": "433a98a2-2a48-4b1a-ad1a-2eb9672f77a2", "status": "Failed", "startTime": "2026-08-02T09:33:58.846876088Z", "endTime": "2026-08-02T09:53:00.063648031Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift 09:15:40 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 4.20 · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:110]: failed to create HCP cluster "cluster-zstream-4-20-x5275t" with version 4.20.27 on candidate channel Unexpected error: <fmt.wrapError \| 0xc0010888e0>: failed to create HCP cluster cluster-zstream-4-20-x5275t: failed waiting for cluster="cluster-zstream-4-20-x5275t" in resourcegroup="rg-zstream-upgrade-4-20-fs7v9z22qchl" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/6f2e46fd-96fc-4ba6-b840-436a57da57e9 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/6f2e46fd-96fc-4ba6-b840-436a57da57e9", "name": "6f2e46fd-96fc-4ba6-b840-436a57da57e9", "status": "Failed", "startTime": "2026-08-02T10:20:39.561995336Z", "endTime": "2026-08-02T10:39:42.899977522Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:110]: failed to create HCP cluster "cluster-zstream-4-20-x5275t" with version 4.20.27 on candidate channel Unexpected error: <fmt.wrapError \| 0xc0010888e0>: failed to create HCP cluster cluster-zstream-4-20-x5275t: failed waiting for cluster="cluster-zstream-4-20-x5275t" in resourcegroup="rg-zstream-upgrade-4-20-fs7v9z22qchl" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/6f2e46fd-96fc-4ba6-b840-436a57da57e9 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/6f2e46fd-96fc-4ba6-b840-436a57da57e9", "name": "6f2e46fd-96fc-4ba6-b840-436a57da57e9", "status": "Failed", "startTime": "2026-08-02T10:20:39.561995336Z", "endTime": "2026-08-02T10:39:42.899977522Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift 09:15:40 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 4.19 · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:110]: failed to create HCP cluster "cluster-zstream-4-19-nhx2wp" with version 4.19.39 on candidate channel Unexpected error: <fmt.wrapError \| 0xc00097aa00>: failed to create HCP cluster cluster-zstream-4-19-nhx2wp: failed waiting for cluster="cluster-zstream-4-19-nhx2wp" in resourcegroup="rg-zstream-upgrade-4-19-rn966m5vc2n7" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/3263c87f-1b2a-4019-8a07-2574b3e4a978 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/3263c87f-1b2a-4019-8a07-2574b3e4a978", "name": "3263c87f-1b2a-4019-8a07-2574b3e4a978", "status": "Failed", "startTime": "2026-08-02T10:20:38.264467464Z", "endTime": "2026-08-02T10:39:42.477082291Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:110]: failed to create HCP cluster "cluster-zstream-4-19-nhx2wp" with version 4.19.39 on candidate channel Unexpected error: <fmt.wrapError \| 0xc00097aa00>: failed to create HCP cluster cluster-zstream-4-19-nhx2wp: failed waiting for cluster="cluster-zstream-4-19-nhx2wp" in resourcegroup="rg-zstream-upgrade-4-19-rn966m5vc2n7" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/3263c87f-1b2a-4019-8a07-2574b3e4a978 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/3263c87f-1b2a-4019-8a07-2574b3e4a978", "name": "3263c87f-1b2a-4019-8a07-2574b3e4a978", "status": "Failed", "startTime": "2026-08-02T10:20:38.264467464Z", "endTime": "2026-08-02T10:39:42.477082291Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred timeout during CreateHCPClusterAndWait; context deadline exceeded 09:15:40 UTC · failed at e2e · test Customer should be able to create an HCP cluster with back-level version 4.19 · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:149]: failed to create HCP cluster version 4.19 Unexpected error: <fmt.wrapErrors \| 0xc001412750>: failed waiting for cluster="cluster-ver-4-19" in resourcegroup="rg-cluster-back-version-sf8fnq7ckvqz" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterAndWait for cluster cluster-ver-4-19 in resource group rg-cluster-back-version-sf8fnq7ckvqz, error: context deadline exceeded ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:149]: failed to create HCP cluster version 4.19 Unexpected error: <fmt.wrapErrors \| 0xc001412750>: failed waiting for cluster="cluster-ver-4-19" in resourcegroup="rg-cluster-back-version-sf8fnq7ckvqz" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterAndWait for cluster cluster-ver-4-19 in resource group rg-cluster-back-version-sf8fnq7ckvqz, error: context deadline exceeded ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift 09:15:40 UTC · failed at e2e · test Customer should create an HCP cluster and validate TLS certificates · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:95]: failed to create HCP cluster for TLS endpoint test Unexpected error: <fmt.wrapError \| 0xc000afc3c0>: failed to create HCP cluster tls-endpoint-hcp-cluster: failed waiting for cluster="tls-endpoint-hcp-cluster" in resourcegroup="tls-endpoint-cluster-4klq7d2pktl6" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/99457e0f-0b14-423b-bb59-8608f2ea25b8 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/99457e0f-0b14-423b-bb59-8608f2ea25b8", "name": "99457e0f-0b14-423b-bb59-8608f2ea25b8", "status": "Failed", "startTime": "2026-08-02T09:34:03.731452984Z", "endTime": "2026-08-02T09:53:20.117946772Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:95]: failed to create HCP cluster for TLS endpoint test Unexpected error: <fmt.wrapError \| 0xc000afc3c0>: failed to create HCP cluster tls-endpoint-hcp-cluster: failed waiting for cluster="tls-endpoint-hcp-cluster" in resourcegroup="tls-endpoint-cluster-4klq7d2pktl6" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/99457e0f-0b14-423b-bb59-8608f2ea25b8 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/99457e0f-0b14-423b-bb59-8608f2ea25b8", "name": "99457e0f-0b14-423b-bb59-8608f2ea25b8", "status": "Failed", "startTime": "2026-08-02T09:34:03.731452984Z", "endTime": "2026-08-02T09:53:20.117946772Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift 09:15:40 UTC · failed at e2e · test Customer should be able to create an HCP cluster and custom node pool osDisk size · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_nodepool_osdisk.go:80]: failed to create HCP cluster "hcp-cluster-np-128" Unexpected error: <fmt.wrapError \| 0xc000dd1360>: failed to create HCP cluster hcp-cluster-np-128: failed waiting for cluster="hcp-cluster-np-128" in resourcegroup="clusternp128-zrbkqqp97x2s" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/89700ddf-4dea-44e4-88e8-5b32393432ad -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/89700ddf-4dea-44e4-88e8-5b32393432ad", "name": "89700ddf-4dea-44e4-88e8-5b32393432ad", "status": "Failed", "startTime": "2026-08-02T09:34:57.46338231Z", "endTime": "2026-08-02T09:54:10.022162917Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_nodepool_osdisk.go:80]: failed to create HCP cluster "hcp-cluster-np-128" Unexpected error: <fmt.wrapError \| 0xc000dd1360>: failed to create HCP cluster hcp-cluster-np-128: failed waiting for cluster="hcp-cluster-np-128" in resourcegroup="clusternp128-zrbkqqp97x2s" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/89700ddf-4dea-44e4-88e8-5b32393432ad -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/89700ddf-4dea-44e4-88e8-5b32393432ad", "name": "89700ddf-4dea-44e4-88e8-5b32393432ad", "status": "Failed", "startTime": "2026-08-02T09:34:57.46338231Z", "endTime": "2026-08-02T09:54:10.022162917Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift 09:15:40 UTC · failed at e2e · test Update HCPOpenShiftCluster Negative creates a cluster and fails to update its name with a PATCH request · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:84]: failed to create HCP cluster for patch-name test Unexpected error: <fmt.wrapError \| 0xc000946480>: failed to create HCP cluster patch-name-cluster: failed waiting for cluster="patch-name-cluster" in resourcegroup="patch-name-b8tzdmst7fvv" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/d9310a88-789a-413f-94fe-bbd77de07f32 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/d9310a88-789a-413f-94fe-bbd77de07f32", "name": "d9310a88-789a-413f-94fe-bbd77de07f32", "status": "Failed", "startTime": "2026-08-02T09:34:21.5378839Z", "endTime": "2026-08-02T09:53:39.994858773Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:84]: failed to create HCP cluster for patch-name test Unexpected error: <fmt.wrapError \| 0xc000946480>: failed to create HCP cluster patch-name-cluster: failed waiting for cluster="patch-name-cluster" in resourcegroup="patch-name-b8tzdmst7fvv" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/d9310a88-789a-413f-94fe-bbd77de07f32 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/d9310a88-789a-413f-94fe-bbd77de07f32", "name": "d9310a88-789a-413f-94fe-bbd77de07f32", "status": "Failed", "startTime": "2026-08-02T09:34:21.5378839Z", "endTime": "2026-08-02T09:53:39.994858773Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated; provider Microsoft.RedHatOpenShift 09:15:40 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.22 · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:121]: HCP cluster rg-candidate-4-22-2xrctt6gqvhc/cluster-candidate-4-22-jzgl67 should provision Unexpected error: <fmt.wrapError \| 0xc0005109e0>: failed to create HCP cluster cluster-candidate-4-22-jzgl67: failed waiting for cluster="cluster-candidate-4-22-jzgl67" in resourcegroup="rg-candidate-4-22-2xrctt6gqvhc" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/b0d09561-daaf-4fc0-ada1-2d89894ad25d -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/b0d09561-daaf-4fc0-ada1-2d89894ad25d", "name": "b0d09561-daaf-4fc0-ada1-2d89894ad25d", "status": "Failed", "startTime": "2026-08-02T10:19:58.698754411Z", "endTime": "2026-08-02T10:39:02.899668713Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:121]: HCP cluster rg-candidate-4-22-2xrctt6gqvhc/cluster-candidate-4-22-jzgl67 should provision Unexpected error: <fmt.wrapError \| 0xc0005109e0>: failed to create HCP cluster cluster-candidate-4-22-jzgl67: failed waiting for cluster="cluster-candidate-4-22-jzgl67" in resourcegroup="rg-candidate-4-22-2xrctt6gqvhc" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/b0d09561-daaf-4fc0-ada1-2d89894ad25d -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/b0d09561-daaf-4fc0-ada1-2d89894ad25d", "name": "b0d09561-daaf-4fc0-ada1-2d89894ad25d", "status": "Failed", "startTime": "2026-08-02T10:19:58.698754411Z", "endTime": "2026-08-02T10:39:02.899668713Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift 09:15:40 UTC · failed at e2e · test Customer should be able to lifecycle and confirm external auth on a cluster · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_list_and_verify.go:87]: failed to create HCP cluster for external auth list test Unexpected error: <fmt.wrapError \| 0xc000f88a80>: failed to create HCP cluster ea-listg8p2lr: failed waiting for cluster="ea-listg8p2lr" in resourcegroup="ea-list-zpwzzw4hr4s4" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/f8f6fba3-8dc3-43e1-a8de-b639f8946093 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/f8f6fba3-8dc3-43e1-a8de-b639f8946093", "name": "f8f6fba3-8dc3-43e1-a8de-b639f8946093", "status": "Failed", "startTime": "2026-08-02T10:22:17.745429228Z", "endTime": "2026-08-02T10:41:22.679642027Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_list_and_verify.go:87]: failed to create HCP cluster for external auth list test Unexpected error: <fmt.wrapError \| 0xc000f88a80>: failed to create HCP cluster ea-listg8p2lr: failed waiting for cluster="ea-listg8p2lr" in resourcegroup="ea-list-zpwzzw4hr4s4" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/f8f6fba3-8dc3-43e1-a8de-b639f8946093 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/f8f6fba3-8dc3-43e1-a8de-b639f8946093", "name": "f8f6fba3-8dc3-43e1-a8de-b639f8946093", "status": "Failed", "startTime": "2026-08-02T10:22:17.745429228Z", "endTime": "2026-08-02T10:41:22.679642027Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift 09:15:40 UTC · failed at e2e · test Customer should be able to create a cluster with default autoscaling and a nodepool with autoscaling enabled up to replica limits · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:102]: failed to create HCP cluster np-autoscale-cluster Unexpected error: <fmt.wrapError \| 0xc000f58fa0>: failed to create HCP cluster np-autoscale-cluster: failed waiting for cluster="np-autoscale-cluster" in resourcegroup="np-autoscaling-nqr8kk5t28mq" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/9a6942f7-8b76-41f6-9814-53dcafaf62c2 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/9a6942f7-8b76-41f6-9814-53dcafaf62c2", "name": "9a6942f7-8b76-41f6-9814-53dcafaf62c2", "status": "Failed", "startTime": "2026-08-02T10:21:01.532783895Z", "endTime": "2026-08-02T10:40:02.630115585Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:102]: failed to create HCP cluster np-autoscale-cluster Unexpected error: <fmt.wrapError \| 0xc000f58fa0>: failed to create HCP cluster np-autoscale-cluster: failed waiting for cluster="np-autoscale-cluster" in resourcegroup="np-autoscaling-nqr8kk5t28mq" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/9a6942f7-8b76-41f6-9814-53dcafaf62c2 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/9a6942f7-8b76-41f6-9814-53dcafaf62c2", "name": "9a6942f7-8b76-41f6-9814-53dcafaf62c2", "status": "Failed", "startTime": "2026-08-02T10:21:01.532783895Z", "endTime": "2026-08-02T10:40:02.630115585Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift 09:15:40 UTC · failed at e2e · test Customer should be able to create a HCP cluster and use cilium CNI plugin · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_cni_cilium.go:88]: failed to create HCP cluster "cilium-cl" without CNI Unexpected error: <fmt.wrapError \| 0xc00118f360>: failed to create HCP cluster cilium-cl: failed waiting for cluster="cilium-cl" in resourcegroup="cni-cilium-5shxkc7h96kd" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/f4e10e32-a5a2-48c8-bb8f-add91194d45d -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/f4e10e32-a5a2-48c8-bb8f-add91194d45d", "name": "f4e10e32-a5a2-48c8-bb8f-add91194d45d", "status": "Failed", "startTime": "2026-08-02T09:34:57.74262599Z", "endTime": "2026-08-02T09:54:00.011864472Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_cni_cilium.go:88]: failed to create HCP cluster "cilium-cl" without CNI Unexpected error: <fmt.wrapError \| 0xc00118f360>: failed to create HCP cluster cilium-cl: failed waiting for cluster="cilium-cl" in resourcegroup="cni-cilium-5shxkc7h96kd" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/f4e10e32-a5a2-48c8-bb8f-add91194d45d -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/f4e10e32-a5a2-48c8-bb8f-add91194d45d", "name": "f4e10e32-a5a2-48c8-bb8f-add91194d45d", "status": "Failed", "startTime": "2026-08-02T09:34:57.74262599Z", "endTime": "2026-08-02T09:54:00.011864472Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift 09:15:40 UTC · failed at e2e · test Customer should be able to successfully upgrade control plane minor version from 4.20 minor to 4.21 minor · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:182]: failed to create HCP cluster "cp-ystream-upgrade-4-21-c4zqpd" with install version 4.20.28 Unexpected error: <fmt.wrapError \| 0xc00069ce00>: failed to create HCP cluster cp-ystream-upgrade-4-21-c4zqpd: failed waiting for cluster="cp-ystream-upgrade-4-21-c4zqpd" in resourcegroup="rg-cp-ystream-upgrade-4-21-b4dtm4754scw" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/ebb17e58-f37d-456d-8dda-e04ad5a89ffd -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/ebb17e58-f37d-456d-8dda-e04ad5a89ffd", "name": "ebb17e58-f37d-456d-8dda-e04ad5a89ffd", "status": "Failed", "startTime": "2026-08-02T10:21:55.844645935Z", "endTime": "2026-08-02T10:41:02.499727821Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:182]: failed to create HCP cluster "cp-ystream-upgrade-4-21-c4zqpd" with install version 4.20.28 Unexpected error: <fmt.wrapError \| 0xc00069ce00>: failed to create HCP cluster cp-ystream-upgrade-4-21-c4zqpd: failed waiting for cluster="cp-ystream-upgrade-4-21-c4zqpd" in resourcegroup="rg-cp-ystream-upgrade-4-21-b4dtm4754scw" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/ebb17e58-f37d-456d-8dda-e04ad5a89ffd -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/ebb17e58-f37d-456d-8dda-e04ad5a89ffd", "name": "ebb17e58-f37d-456d-8dda-e04ad5a89ffd", "status": "Failed", "startTime": "2026-08-02T10:21:55.844645935Z", "endTime": "2026-08-02T10:41:02.499727821Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated; provider Microsoft.RedHatOpenShift 09:15:40 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.21 · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:121]: HCP cluster rg-candidate-4-21-thgbrpsnmsdd/cluster-candidate-4-21-vvlhtd should provision Unexpected error: <fmt.wrapError \| 0xc000b94c00>: failed to create HCP cluster cluster-candidate-4-21-vvlhtd: failed waiting for cluster="cluster-candidate-4-21-vvlhtd" in resourcegroup="rg-candidate-4-21-thgbrpsnmsdd" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/cfe40609-dfaa-4309-a817-b449ab573e51 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/cfe40609-dfaa-4309-a817-b449ab573e51", "name": "cfe40609-dfaa-4309-a817-b449ab573e51", "status": "Failed", "startTime": "2026-08-02T09:40:52.659578391Z", "endTime": "2026-08-02T10:00:01.276872508Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:121]: HCP cluster rg-candidate-4-21-thgbrpsnmsdd/cluster-candidate-4-21-vvlhtd should provision Unexpected error: <fmt.wrapError \| 0xc000b94c00>: failed to create HCP cluster cluster-candidate-4-21-vvlhtd: failed waiting for cluster="cluster-candidate-4-21-vvlhtd" in resourcegroup="rg-candidate-4-21-thgbrpsnmsdd" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/cfe40609-dfaa-4309-a817-b449ab573e51 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/cfe40609-dfaa-4309-a817-b449ab573e51", "name": "cfe40609-dfaa-4309-a817-b449ab573e51", "status": "Failed", "startTime": "2026-08-02T09:40:52.659578391Z", "endTime": "2026-08-02T10:00:01.276872508Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift 09:15:40 UTC · failed at e2e · test Customer should upgrade and update a nodepool from 4.20.z to 4.20.zLatest · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:133]: failed to create HCP cluster np-version-upgrade-cluster-7tjrwf with version 4.20.32 Unexpected error: <fmt.wrapError \| 0xc0009fc860>: failed to create HCP cluster np-version-upgrade-cluster-7tjrwf: failed waiting for cluster="np-version-upgrade-cluster-7tjrwf" in resourcegroup="rg-np-version-upgrade-7tjrwf-gb7vxqn6kj8r" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/5f96cdc6-19a6-49b1-97a9-d42ab5c04593 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/5f96cdc6-19a6-49b1-97a9-d42ab5c04593", "name": "5f96cdc6-19a6-49b1-97a9-d42ab5c04593", "status": "Failed", "startTime": "2026-08-02T10:25:02.326268128Z", "endTime": "2026-08-02T10:44:02.561752064Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:133]: failed to create HCP cluster np-version-upgrade-cluster-7tjrwf with version 4.20.32 Unexpected error: <fmt.wrapError \| 0xc0009fc860>: failed to create HCP cluster np-version-upgrade-cluster-7tjrwf: failed waiting for cluster="np-version-upgrade-cluster-7tjrwf" in resourcegroup="rg-np-version-upgrade-7tjrwf-gb7vxqn6kj8r" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/5f96cdc6-19a6-49b1-97a9-d42ab5c04593 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/5f96cdc6-19a6-49b1-97a9-d42ab5c04593", "name": "5f96cdc6-19a6-49b1-97a9-d42ab5c04593", "status": "Failed", "startTime": "2026-08-02T10:25:02.326268128Z", "endTime": "2026-08-02T10:44:02.561752064Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift 09:15:40 UTC · failed at e2e · test Update HCPOpenShiftCluster Positive creates a cluster and updates tags with a PATCH request · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:161]: failed to create HCP cluster for patch-tags test Unexpected error: <fmt.wrapError \| 0xc0001bc3c0>: failed to create HCP cluster patch-tags-cluster: failed waiting for cluster="patch-tags-cluster" in resourcegroup="patch-tags-z7vrqxfs7z6k" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/96eaab45-e218-47ab-8385-511d2b8bea05 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/96eaab45-e218-47ab-8385-511d2b8bea05", "name": "96eaab45-e218-47ab-8385-511d2b8bea05", "status": "Failed", "startTime": "2026-08-02T09:34:01.062473396Z", "endTime": "2026-08-02T09:53:20.146592724Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:161]: failed to create HCP cluster for patch-tags test Unexpected error: <fmt.wrapError \| 0xc0001bc3c0>: failed to create HCP cluster patch-tags-cluster: failed waiting for cluster="patch-tags-cluster" in resourcegroup="patch-tags-z7vrqxfs7z6k" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/96eaab45-e218-47ab-8385-511d2b8bea05 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/96eaab45-e218-47ab-8385-511d2b8bea05", "name": "96eaab45-e218-47ab-8385-511d2b8bea05", "status": "Failed", "startTime": "2026-08-02T09:34:01.062473396Z", "endTime": "2026-08-02T09:53:20.146592724Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift 09:15:40 UTC · failed at e2e · test Customer should be able to update node pool labels and taints · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_labels_taints.go:92]: failed to create HCP cluster cluster-np-labels-taints Unexpected error: <fmt.wrapError \| 0xc000e08ee0>: failed to create HCP cluster cluster-np-labels-taints: failed waiting for cluster="cluster-np-labels-taints" in resourcegroup="rg-np-labels-taints-m6z62b4jc2tx" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/37f499cd-df85-402b-81c7-871df8ababcf -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/37f499cd-df85-402b-81c7-871df8ababcf", "name": "37f499cd-df85-402b-81c7-871df8ababcf", "status": "Failed", "startTime": "2026-08-02T10:21:11.158027917Z", "endTime": "2026-08-02T10:40:12.70290935Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_labels_taints.go:92]: failed to create HCP cluster cluster-np-labels-taints Unexpected error: <fmt.wrapError \| 0xc000e08ee0>: failed to create HCP cluster cluster-np-labels-taints: failed waiting for cluster="cluster-np-labels-taints" in resourcegroup="rg-np-labels-taints-m6z62b4jc2tx" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/37f499cd-df85-402b-81c7-871df8ababcf -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/37f499cd-df85-402b-81c7-871df8ababcf", "name": "37f499cd-df85-402b-81c7-871df8ababcf", "status": "Failed", "startTime": "2026-08-02T10:21:11.158027917Z", "endTime": "2026-08-02T10:40:12.70290935Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift 09:15:40 UTC · failed at e2e · test Customer should upgrade and update a nodepool from 4.20.z to 4.21.zLatest · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:133]: failed to create HCP cluster np-version-upgrade-cluster-4c9krr with version 4.21.23 Unexpected error: <fmt.wrapError \| 0xc000d80ec0>: failed to create HCP cluster np-version-upgrade-cluster-4c9krr: failed waiting for cluster="np-version-upgrade-cluster-4c9krr" in resourcegroup="rg-np-version-upgrade-4c9krr-wrfbrlj4lbbr" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/146bbd4a-743f-4661-9c79-1a8d6a5e97cd -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/146bbd4a-743f-4661-9c79-1a8d6a5e97cd", "name": "146bbd4a-743f-4661-9c79-1a8d6a5e97cd", "status": "Failed", "startTime": "2026-08-02T10:22:48.128797596Z", "endTime": "2026-08-02T10:41:52.721915786Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:133]: failed to create HCP cluster np-version-upgrade-cluster-4c9krr with version 4.21.23 Unexpected error: <fmt.wrapError \| 0xc000d80ec0>: failed to create HCP cluster np-version-upgrade-cluster-4c9krr: failed waiting for cluster="np-version-upgrade-cluster-4c9krr" in resourcegroup="rg-np-version-upgrade-4c9krr-wrfbrlj4lbbr" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/146bbd4a-743f-4661-9c79-1a8d6a5e97cd -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/146bbd4a-743f-4661-9c79-1a8d6a5e97cd", "name": "146bbd4a-743f-4661-9c79-1a8d6a5e97cd", "status": "Failed", "startTime": "2026-08-02T10:22:48.128797596Z", "endTime": "2026-08-02T10:41:52.721915786Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated; provider Microsoft.RedHatOpenShift 09:15:40 UTC · failed at e2e · test Customer should be able to create an HCP cluster and manage ImageDigestMirrors · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/idms_lifecycle.go:113]: failed to create HCP cluster with ImageDigestMirrors Unexpected error: <fmt.wrapError \| 0xc000954860>: failed to create HCP cluster idms-e2e-hcp-cluster: failed waiting for cluster="idms-e2e-hcp-cluster" in resourcegroup="idms-qzhvx5bbqbk9" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/016f96d1-550a-41f2-9fc1-b4db6bb899c9 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/016f96d1-550a-41f2-9fc1-b4db6bb899c9", "name": "016f96d1-550a-41f2-9fc1-b4db6bb899c9", "status": "Failed", "startTime": "2026-08-02T10:21:25.833585484Z", "endTime": "2026-08-02T10:40:42.522944424Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/idms_lifecycle.go:113]: failed to create HCP cluster with ImageDigestMirrors Unexpected error: <fmt.wrapError \| 0xc000954860>: failed to create HCP cluster idms-e2e-hcp-cluster: failed waiting for cluster="idms-e2e-hcp-cluster" in resourcegroup="idms-qzhvx5bbqbk9" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/016f96d1-550a-41f2-9fc1-b4db6bb899c9 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/016f96d1-550a-41f2-9fc1-b4db6bb899c9", "name": "016f96d1-550a-41f2-9fc1-b4db6bb899c9", "status": "Failed", "startTime": "2026-08-02T10:21:25.833585484Z", "endTime": "2026-08-02T10:40:42.522944424Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift 09:15:40 UTC · failed at e2e · test Customer should be able to create a cluster with an external auth config and get the external auth config · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:94]: failed to create HCP cluster for external auth test Unexpected error: <fmt.wrapError \| 0xc0012d48e0>: failed to create HCP cluster ea-cluster: failed waiting for cluster="ea-cluster" in resourcegroup="external-auth-cluster-fqw8f7c4wwz6" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/24c83ba8-3e7d-4e9d-acd7-818541d03e5b -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/24c83ba8-3e7d-4e9d-acd7-818541d03e5b", "name": "24c83ba8-3e7d-4e9d-acd7-818541d03e5b", "status": "Failed", "startTime": "2026-08-02T10:22:16.371732004Z", "endTime": "2026-08-02T10:41:22.659684191Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:94]: failed to create HCP cluster for external auth test Unexpected error: <fmt.wrapError \| 0xc0012d48e0>: failed to create HCP cluster ea-cluster: failed waiting for cluster="ea-cluster" in resourcegroup="external-auth-cluster-fqw8f7c4wwz6" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/24c83ba8-3e7d-4e9d-acd7-818541d03e5b -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/24c83ba8-3e7d-4e9d-acd7-818541d03e5b", "name": "24c83ba8-3e7d-4e9d-acd7-818541d03e5b", "status": "Failed", "startTime": "2026-08-02T10:22:16.371732004Z", "endTime": "2026-08-02T10:41:22.659684191Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift 09:15:40 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 4.22 · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:110]: failed to create HCP cluster "cluster-zstream-4-22-qm2jhm" with version 4.22.6 on candidate channel Unexpected error: <fmt.wrapError \| 0xc000f14c40>: failed to create HCP cluster cluster-zstream-4-22-qm2jhm: failed waiting for cluster="cluster-zstream-4-22-qm2jhm" in resourcegroup="rg-zstream-upgrade-4-22-7prtcnrz9p7l" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/604d2e18-9f93-40ae-8c59-0ae91f105f7a -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/604d2e18-9f93-40ae-8c59-0ae91f105f7a", "name": "604d2e18-9f93-40ae-8c59-0ae91f105f7a", "status": "Failed", "startTime": "2026-08-02T10:20:29.396777978Z", "endTime": "2026-08-02T10:39:42.921782454Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:110]: failed to create HCP cluster "cluster-zstream-4-22-qm2jhm" with version 4.22.6 on candidate channel Unexpected error: <fmt.wrapError \| 0xc000f14c40>: failed to create HCP cluster cluster-zstream-4-22-qm2jhm: failed waiting for cluster="cluster-zstream-4-22-qm2jhm" in resourcegroup="rg-zstream-upgrade-4-22-7prtcnrz9p7l" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/604d2e18-9f93-40ae-8c59-0ae91f105f7a -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/604d2e18-9f93-40ae-8c59-0ae91f105f7a", "name": "604d2e18-9f93-40ae-8c59-0ae91f105f7a", "status": "Failed", "startTime": "2026-08-02T10:20:29.396777978Z", "endTime": "2026-08-02T10:39:42.921782454Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated; provider Microsoft.RedHatOpenShift 09:15:40 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.20 · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:121]: HCP cluster rg-candidate-4-20-p79b4rkhtstx/cluster-candidate-4-20-qrm8jx should provision Unexpected error: <fmt.wrapError \| 0xc000a92d80>: failed to create HCP cluster cluster-candidate-4-20-qrm8jx: failed waiting for cluster="cluster-candidate-4-20-qrm8jx" in resourcegroup="rg-candidate-4-20-p79b4rkhtstx" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/8bade74f-a4fb-450a-a9c8-31a90fb5df6a -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/8bade74f-a4fb-450a-a9c8-31a90fb5df6a", "name": "8bade74f-a4fb-450a-a9c8-31a90fb5df6a", "status": "Failed", "startTime": "2026-08-02T09:35:10.119552619Z", "endTime": "2026-08-02T09:54:10.020290658Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:121]: HCP cluster rg-candidate-4-20-p79b4rkhtstx/cluster-candidate-4-20-qrm8jx should provision Unexpected error: <fmt.wrapError \| 0xc000a92d80>: failed to create HCP cluster cluster-candidate-4-20-qrm8jx: failed waiting for cluster="cluster-candidate-4-20-qrm8jx" in resourcegroup="rg-candidate-4-20-p79b4rkhtstx" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/8bade74f-a4fb-450a-a9c8-31a90fb5df6a -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/8bade74f-a4fb-450a-a9c8-31a90fb5df6a", "name": "8bade74f-a4fb-450a-a9c8-31a90fb5df6a", "status": "Failed", "startTime": "2026-08-02T09:35:10.119552619Z", "endTime": "2026-08-02T09:54:10.020290658Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift 09:15:40 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 4.21 · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:110]: failed to create HCP cluster "cluster-zstream-4-21-wpfbzb" with version 4.21.22 on candidate channel Unexpected error: <fmt.wrapError \| 0xc000eb0240>: failed to create HCP cluster cluster-zstream-4-21-wpfbzb: failed waiting for cluster="cluster-zstream-4-21-wpfbzb" in resourcegroup="rg-zstream-upgrade-4-21-hqwpx2rp9hwp" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/5a86e2da-8342-4e05-9a7b-646fec5d1f57 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/5a86e2da-8342-4e05-9a7b-646fec5d1f57", "name": "5a86e2da-8342-4e05-9a7b-646fec5d1f57", "status": "Failed", "startTime": "2026-08-02T10:20:45.199672861Z", "endTime": "2026-08-02T10:39:52.85500071Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:110]: failed to create HCP cluster "cluster-zstream-4-21-wpfbzb" with version 4.21.22 on candidate channel Unexpected error: <fmt.wrapError \| 0xc000eb0240>: failed to create HCP cluster cluster-zstream-4-21-wpfbzb: failed waiting for cluster="cluster-zstream-4-21-wpfbzb" in resourcegroup="rg-zstream-upgrade-4-21-hqwpx2rp9hwp" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/5a86e2da-8342-4e05-9a7b-646fec5d1f57 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/5a86e2da-8342-4e05-9a7b-646fec5d1f57", "name": "5a86e2da-8342-4e05-9a7b-646fec5d1f57", "status": "Failed", "startTime": "2026-08-02T10:20:45.199672861Z", "endTime": "2026-08-02T10:39:52.85500071Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift 09:15:40 UTC · failed at e2e · test Customer should be able to create an HCP cluster with Image Registry not present · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/image_registry_cluster_create.go:89]: failed to create HCP cluster with image registry disabled Unexpected error: <fmt.wrapError \| 0xc0001a5360>: failed to create HCP cluster disabled-image-registry-hcp-cluster: failed waiting for cluster="disabled-image-registry-hcp-cluster" in resourcegroup="disabled-image-registry-xjxxj2swjnz9" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/df395857-64f2-4cba-a384-dba5dbda1151 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/df395857-64f2-4cba-a384-dba5dbda1151", "name": "df395857-64f2-4cba-a384-dba5dbda1151", "status": "Failed", "startTime": "2026-08-02T10:21:04.574656397Z", "endTime": "2026-08-02T10:40:12.775068231Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/image_registry_cluster_create.go:89]: failed to create HCP cluster with image registry disabled Unexpected error: <fmt.wrapError \| 0xc0001a5360>: failed to create HCP cluster disabled-image-registry-hcp-cluster: failed waiting for cluster="disabled-image-registry-hcp-cluster" in resourcegroup="disabled-image-registry-xjxxj2swjnz9" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/df395857-64f2-4cba-a384-dba5dbda1151 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/df395857-64f2-4cba-a384-dba5dbda1151", "name": "df395857-64f2-4cba-a384-dba5dbda1151", "status": "Failed", "startTime": "2026-08-02T10:21:04.574656397Z", "endTime": "2026-08-02T10:40:12.775068231Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift 09:15:40 UTC · failed at e2e · test Customer should respect cluster-wide node limits with nodepool autoscaling · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:291]: failed to create HCP cluster node-limit-cluster with MaxNodesTotal limit Unexpected error: <fmt.wrapError \| 0xc0004ae680>: failed to create HCP cluster node-limit-cluster: failed waiting for cluster="node-limit-cluster" in resourcegroup="node-limit-test-774f7qvh8zdv" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/6f2628e2-cfb7-4630-92d6-e86eb6db9b46 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/6f2628e2-cfb7-4630-92d6-e86eb6db9b46", "name": "6f2628e2-cfb7-4630-92d6-e86eb6db9b46", "status": "Failed", "startTime": "2026-08-02T10:21:35.592173312Z", "endTime": "2026-08-02T10:40:42.509887887Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:291]: failed to create HCP cluster node-limit-cluster with MaxNodesTotal limit Unexpected error: <fmt.wrapError \| 0xc0004ae680>: failed to create HCP cluster node-limit-cluster: failed waiting for cluster="node-limit-cluster" in resourcegroup="node-limit-test-774f7qvh8zdv" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/6f2628e2-cfb7-4630-92d6-e86eb6db9b46 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/6f2628e2-cfb7-4630-92d6-e86eb6db9b46", "name": "6f2628e2-cfb7-4630-92d6-e86eb6db9b46", "status": "Failed", "startTime": "2026-08-02T10:21:35.592173312Z", "endTime": "2026-08-02T10:40:42.509887887Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred Cluster provisioning failed 09:15:40 UTC · failed at e2e · test Customer should be able to test admin credentials before cluster ready, then full admin credential lifecycle · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:158]: Cluster provisioning failed fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:158]: Cluster provisioning failed ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift 09:15:40 UTC · failed at e2e · test Customer should upgrade and update a nodepool from 4.21.z to 4.21.zLatest · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:133]: failed to create HCP cluster np-version-upgrade-cluster-t5gbrr with version 4.21.23 Unexpected error: <fmt.wrapError \| 0xc00062e4c0>: failed to create HCP cluster np-version-upgrade-cluster-t5gbrr: failed waiting for cluster="np-version-upgrade-cluster-t5gbrr" in resourcegroup="rg-np-version-upgrade-t5gbrr-9m92wdf9smth" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/a893a929-5de6-4c11-9274-877b11423a45 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/a893a929-5de6-4c11-9274-877b11423a45", "name": "a893a929-5de6-4c11-9274-877b11423a45", "status": "Failed", "startTime": "2026-08-02T10:22:58.969255601Z", "endTime": "2026-08-02T10:42:02.681869225Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:133]: failed to create HCP cluster np-version-upgrade-cluster-t5gbrr with version 4.21.23 Unexpected error: <fmt.wrapError \| 0xc00062e4c0>: failed to create HCP cluster np-version-upgrade-cluster-t5gbrr: failed waiting for cluster="np-version-upgrade-cluster-t5gbrr" in resourcegroup="rg-np-version-upgrade-t5gbrr-9m92wdf9smth" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/a893a929-5de6-4c11-9274-877b11423a45 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/a893a929-5de6-4c11-9274-877b11423a45", "name": "a893a929-5de6-4c11-9274-877b11423a45", "status": "Failed", "startTime": "2026-08-02T10:22:58.969255601Z", "endTime": "2026-08-02T10:42:02.681869225Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift 09:15:40 UTC · failed at e2e · test Customer should not be able to reuse subnets and NSGs between clusters · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_nsg_subnet_reuse.go:107]: failed to create seed HCP cluster basic-cluster Unexpected error: <fmt.wrapError \| 0xc000906a20>: failed to create HCP cluster basic-cluster: failed waiting for cluster="basic-cluster" in resourcegroup="rg-cluster-nsg-subnet-reuse-n9jxzw2mh8nf" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/b2328641-a92c-449e-ae11-b4e1b37ed5a2 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/b2328641-a92c-449e-ae11-b4e1b37ed5a2", "name": "b2328641-a92c-449e-ae11-b4e1b37ed5a2", "status": "Failed", "startTime": "2026-08-02T09:34:07.535948206Z", "endTime": "2026-08-02T09:53:20.13427646Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_nsg_subnet_reuse.go:107]: failed to create seed HCP cluster basic-cluster Unexpected error: <fmt.wrapError \| 0xc000906a20>: failed to create HCP cluster basic-cluster: failed waiting for cluster="basic-cluster" in resourcegroup="rg-cluster-nsg-subnet-reuse-n9jxzw2mh8nf" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/b2328641-a92c-449e-ae11-b4e1b37ed5a2 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/b2328641-a92c-449e-ae11-b4e1b37ed5a2", "name": "b2328641-a92c-449e-ae11-b4e1b37ed5a2", "status": "Failed", "startTime": "2026-08-02T09:34:07.535948206Z", "endTime": "2026-08-02T09:53:20.13427646Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated; provider Microsoft.RedHatOpenShift 09:15:40 UTC · failed at e2e · test Create HCPOpenShiftCluster with Private KeyVault should create a cluster with private keyvault using v20251223preview API · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:98]: failed to create HCP cluster "private-kv-cluster" with private keyvault Unexpected error: <fmt.wrapError \| 0xc000aa8d00>: failed waiting for cluster="private-kv-cluster" in resourcegroup="private-keyvault-xxd89b5gsd7p" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/6d9000a4-f892-48f3-beba-a01dcf970841 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/6d9000a4-f892-48f3-beba-a01dcf970841", "name": "6d9000a4-f892-48f3-beba-a01dcf970841", "status": "Failed", "startTime": "2026-08-02T09:38:30.54644304Z", "endTime": "2026-08-02T09:57:39.932107313Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:98]: failed to create HCP cluster "private-kv-cluster" with private keyvault Unexpected error: <fmt.wrapError \| 0xc000aa8d00>: failed waiting for cluster="private-kv-cluster" in resourcegroup="private-keyvault-xxd89b5gsd7p" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/6d9000a4-f892-48f3-beba-a01dcf970841 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/6d9000a4-f892-48f3-beba-a01dcf970841", "name": "6d9000a4-f892-48f3-beba-a01dcf970841", "status": "Failed", "startTime": "2026-08-02T09:38:30.54644304Z", "endTime": "2026-08-02T09:57:39.932107313Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift 09:15:40 UTC · failed at e2e · test Nodepool Ephemeral OS Disk should create a nodepool with ephemeral OS disk when autoRepair is enabled · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_ephemeral_osdisk.go:112]: failed to create HCP cluster ephemeral-disk Unexpected error: <fmt.wrapError \| 0xc000f569c0>: failed to create HCP cluster ephemeral-disk: failed waiting for cluster="ephemeral-disk" in resourcegroup="ephemeral-osdisk-tpg49l4jbx5d" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/565715c5-d553-4e22-9e42-0fc997a2a44c -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/565715c5-d553-4e22-9e42-0fc997a2a44c", "name": "565715c5-d553-4e22-9e42-0fc997a2a44c", "status": "Failed", "startTime": "2026-08-02T10:21:22.100482516Z", "endTime": "2026-08-02T10:40:22.835720238Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_ephemeral_osdisk.go:112]: failed to create HCP cluster ephemeral-disk Unexpected error: <fmt.wrapError \| 0xc000f569c0>: failed to create HCP cluster ephemeral-disk: failed waiting for cluster="ephemeral-disk" in resourcegroup="ephemeral-osdisk-tpg49l4jbx5d" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/565715c5-d553-4e22-9e42-0fc997a2a44c -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/565715c5-d553-4e22-9e42-0fc997a2a44c", "name": "565715c5-d553-4e22-9e42-0fc997a2a44c", "status": "Failed", "startTime": "2026-08-02T10:21:22.100482516Z", "endTime": "2026-08-02T10:40:22.835720238Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated; provider Microsoft.RedHatOpenShift 09:15:40 UTC · failed at e2e · test Customer should be able to create a no-CNI private cluster with a private key vault, a nodepool and install cilium CNI successfully · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:105]: failed to create HCP cluster "cilium-cluster" with no CNI and private etcd Unexpected error: <fmt.wrapError \| 0xc0003fe7c0>: failed waiting for cluster="cilium-cluster" in resourcegroup="complex-cilium-kv-q7k5fm2cgm92" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/66774fff-682b-4c22-af08-5f9dea95118b -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/66774fff-682b-4c22-af08-5f9dea95118b", "name": "66774fff-682b-4c22-af08-5f9dea95118b", "status": "Failed", "startTime": "2026-08-02T09:36:47.470711734Z", "endTime": "2026-08-02T09:55:49.913729058Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:105]: failed to create HCP cluster "cilium-cluster" with no CNI and private etcd Unexpected error: <fmt.wrapError \| 0xc0003fe7c0>: failed waiting for cluster="cilium-cluster" in resourcegroup="complex-cilium-kv-q7k5fm2cgm92" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/66774fff-682b-4c22-af08-5f9dea95118b -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/66774fff-682b-4c22-af08-5f9dea95118b", "name": "66774fff-682b-4c22-af08-5f9dea95118b", "status": "Failed", "startTime": "2026-08-02T09:36:47.470711734Z", "endTime": "2026-08-02T09:55:49.913729058Z", "error": { "code": "InternalServerError", "message": "[clusterServiceClusterStatus] \u003cno_message\u003e; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated" } } -------------------------------------------------------------------------------- ... occurred
02:00:06	periodic-ci-Azure-ARO-HCP-main-periodic-prod-e2e-parallel post-goodmerged PR No additional run metadata captured	e2e	failed	n/a	1	e2e (1) ERROR CODE: InternalServerError; detail message [clusterServiceNodePoolStatus] <no_message>; provider Microsoft.RedHatOpenShift 02:00:06 UTC · failed at e2e · test Customer should be able to create a cluster and node pools with aggregated advanced features · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:328]: failed to verify cilium is running and nodes are Ready for cluster "agg-cluster" Unexpected error: <errors.joinError \| 0xc00214e018>: VerifyNodesReady failed: no nodes found in the cluster VerifyCiliumOperational failed: not all pods in kube-system namespace are running: no cilium pods found in kube-system namespace failed to create NodePool agg-np-a: failed waiting for nodepool="agg-np-a" for cluster "agg-cluster" in resourcegroup="feature-aggregation-l54jll2tvrwp" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0bdec85c-5f48-447c-b8b0-4dedc7990652 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0bdec85c-5f48-447c-b8b0-4dedc7990652", "name": "0bdec85c-5f48-447c-b8b0-4dedc7990652", "status": "Failed", "startTime": "2026-08-02T02:20:43.022860317Z", "endTime": "2026-08-02T02:39:48.638662525Z", "error": { "code": "InternalServerError", "message": "[clusterServiceNodePoolStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:328]: failed to verify cilium is running and nodes are Ready for cluster "agg-cluster" Unexpected error: <errors.joinError \| 0xc00214e018>: VerifyNodesReady failed: no nodes found in the cluster VerifyCiliumOperational failed: not all pods in kube-system namespace are running: no cilium pods found in kube-system namespace failed to create NodePool agg-np-a: failed waiting for nodepool="agg-np-a" for cluster "agg-cluster" in resourcegroup="feature-aggregation-l54jll2tvrwp" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0bdec85c-5f48-447c-b8b0-4dedc7990652 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0bdec85c-5f48-447c-b8b0-4dedc7990652", "name": "0bdec85c-5f48-447c-b8b0-4dedc7990652", "status": "Failed", "startTime": "2026-08-02T02:20:43.022860317Z", "endTime": "2026-08-02T02:39:48.638662525Z", "error": { "code": "InternalServerError", "message": "[clusterServiceNodePoolStatus] \u003cno_message\u003e" } } -------------------------------------------------------------------------------- ... occurred

Time (UTC)

Job

Failed at

Result

Failed tests

Details

23:00:07

periodic-ci-Azure-ARO-HCP-main-periodic-prod-e2e-parallel-ocp-nightly

post-goodmerged PR

No additional run metadata captured

e2e

failed

n/a

e2e (1)

ERROR CODE: InternalServerError; detail message [clusterServiceNodePoolStatus] <no_message>; provider Microsoft.RedHatOpenShift

23:00:07 UTC · failed at e2e · test Create HCPOpenShiftCluster with Private KeyVault should create a cluster with private keyvault using v20251223preview API · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:137]: failed to create node pool "np-1" for private keyvault cluster "private-kv-cluster"
Unexpected error:
    <*fmt.wrapError | 0xc000d24a00>: 
    failed to create NodePool np-1: failed waiting for nodepool="np-1" for cluster "private-kv-cluster" in resourcegroup="private-keyvault-9ngxf9zlhblm" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/3463e632-f100-45af-9f71-fc69cdf2dd4c
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/3463e632-f100-45af-9f71-fc69cdf2dd4c",
      "name": "3463e632-f100-45af-9f71-fc69cdf2dd4c",
      "status": "Failed",
      "startTime": "2026-08-02T23:16:20.627905758Z",
      "endTime": "2026-08-02T23:35:28.37866899Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceNodePoolStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:137]: failed to create node pool "np-1" for private keyvault cluster "private-kv-cluster"
Unexpected error:
    <*fmt.wrapError | 0xc000d24a00>: 
    failed to create NodePool np-1: failed waiting for nodepool="np-1" for cluster "private-kv-cluster" in resourcegroup="private-keyvault-9ngxf9zlhblm" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/3463e632-f100-45af-9f71-fc69cdf2dd4c
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/3463e632-f100-45af-9f71-fc69cdf2dd4c",
      "name": "3463e632-f100-45af-9f71-fc69cdf2dd4c",
      "status": "Failed",
      "startTime": "2026-08-02T23:16:20.627905758Z",
      "endTime": "2026-08-02T23:35:28.37866899Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceNodePoolStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

09:15:40

branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel

post-goodmerged PR

No additional run metadata captured

e2e

failed

n/a

e2e (38)

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift

09:15:40 UTC · failed at e2e · test Customer should be able to create an HCP cluster then delete it by deleting the customer resource group · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delete_cx_rg.go:90]: failed to create HCP cluster "cx-rg-hcp-cluster"
Unexpected error:
    <*fmt.wrapError | 0xc000d90900>: 
    failed to create HCP cluster cx-rg-hcp-cluster: failed waiting for cluster="cx-rg-hcp-cluster" in resourcegroup="cx-rg-cluster-7tzmlfvcvjz2" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/8e7f6e9d-54e9-43d6-9057-7304d01b9584
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/8e7f6e9d-54e9-43d6-9057-7304d01b9584",
      "name": "8e7f6e9d-54e9-43d6-9057-7304d01b9584",
      "status": "Failed",
      "startTime": "2026-08-02T09:34:18.700863215Z",
      "endTime": "2026-08-02T09:53:19.940689234Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delete_cx_rg.go:90]: failed to create HCP cluster "cx-rg-hcp-cluster"
Unexpected error:
    <*fmt.wrapError | 0xc000d90900>: 
    failed to create HCP cluster cx-rg-hcp-cluster: failed waiting for cluster="cx-rg-hcp-cluster" in resourcegroup="cx-rg-cluster-7tzmlfvcvjz2" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/8e7f6e9d-54e9-43d6-9057-7304d01b9584
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/8e7f6e9d-54e9-43d6-9057-7304d01b9584",
      "name": "8e7f6e9d-54e9-43d6-9057-7304d01b9584",
      "status": "Failed",
      "startTime": "2026-08-02T09:34:18.700863215Z",
      "endTime": "2026-08-02T09:53:19.940689234Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift

09:15:40 UTC · failed at e2e · test Customer should be able to create node pools with ARM64-based VMs · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/arm64_nodepool.go:88]: failed to create HCP cluster "arm64-vm-hcp-cluster"
Unexpected error:
    <*fmt.wrapError | 0xc0009404a0>: 
    failed to create HCP cluster arm64-vm-hcp-cluster: failed waiting for cluster="arm64-vm-hcp-cluster" in resourcegroup="arm64-vm-cluster-4k4dbkr4s254" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/4c3c5ea4-015b-4faa-988b-ca92a5589f78
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/4c3c5ea4-015b-4faa-988b-ca92a5589f78",
      "name": "4c3c5ea4-015b-4faa-988b-ca92a5589f78",
      "status": "Failed",
      "startTime": "2026-08-02T09:34:01.642064318Z",
      "endTime": "2026-08-02T09:53:20.129747727Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/arm64_nodepool.go:88]: failed to create HCP cluster "arm64-vm-hcp-cluster"
Unexpected error:
    <*fmt.wrapError | 0xc0009404a0>: 
    failed to create HCP cluster arm64-vm-hcp-cluster: failed waiting for cluster="arm64-vm-hcp-cluster" in resourcegroup="arm64-vm-cluster-4k4dbkr4s254" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/4c3c5ea4-015b-4faa-988b-ca92a5589f78
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/4c3c5ea4-015b-4faa-988b-ca92a5589f78",
      "name": "4c3c5ea4-015b-4faa-988b-ca92a5589f78",
      "status": "Failed",
      "startTime": "2026-08-02T09:34:01.642064318Z",
      "endTime": "2026-08-02T09:53:20.129747727Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift

09:15:40 UTC · failed at e2e · test Customer should upgrade a nodepool skipping one minor version (+2) from 4.20.z to 4.22.zLatest · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:462]: failed to create HCP cluster np-skip-minor-hj85vs at version 4.22.8
Unexpected error:
    <*fmt.wrapError | 0xc000cd93c0>: 
    failed to create HCP cluster np-skip-minor-hj85vs: failed waiting for cluster="np-skip-minor-hj85vs" in resourcegroup="rg-np-skip-minor-hj85vs-5sslvjsdhbbb" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/79102d9d-d8f7-4787-8796-65c7af655254
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/79102d9d-d8f7-4787-8796-65c7af655254",
      "name": "79102d9d-d8f7-4787-8796-65c7af655254",
      "status": "Failed",
      "startTime": "2026-08-02T10:46:52.48500524Z",
      "endTime": "2026-08-02T11:05:53.701127939Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:462]: failed to create HCP cluster np-skip-minor-hj85vs at version 4.22.8
Unexpected error:
    <*fmt.wrapError | 0xc000cd93c0>: 
    failed to create HCP cluster np-skip-minor-hj85vs: failed waiting for cluster="np-skip-minor-hj85vs" in resourcegroup="rg-np-skip-minor-hj85vs-5sslvjsdhbbb" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/79102d9d-d8f7-4787-8796-65c7af655254
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/79102d9d-d8f7-4787-8796-65c7af655254",
      "name": "79102d9d-d8f7-4787-8796-65c7af655254",
      "status": "Failed",
      "startTime": "2026-08-02T10:46:52.48500524Z",
      "endTime": "2026-08-02T11:05:53.701127939Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift

09:15:40 UTC · failed at e2e · test Customer should be able to create an HCP cluster and manage pull secrets · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_pullsecret.go:114]: failed to create HCP cluster for pull secret test
Unexpected error:
    <*fmt.wrapError | 0xc00099e6a0>: 
    failed to create HCP cluster pullsecret-hcp-cluster: failed waiting for cluster="pullsecret-hcp-cluster" in resourcegroup="pullsecret-test-h88dk8ndppfm" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/6dbae615-f8d0-4154-ac6c-998d81f7b84d
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/6dbae615-f8d0-4154-ac6c-998d81f7b84d",
      "name": "6dbae615-f8d0-4154-ac6c-998d81f7b84d",
      "status": "Failed",
      "startTime": "2026-08-02T09:33:48.731562196Z",
      "endTime": "2026-08-02T09:52:59.947253927Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_pullsecret.go:114]: failed to create HCP cluster for pull secret test
Unexpected error:
    <*fmt.wrapError | 0xc00099e6a0>: 
    failed to create HCP cluster pullsecret-hcp-cluster: failed waiting for cluster="pullsecret-hcp-cluster" in resourcegroup="pullsecret-test-h88dk8ndppfm" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/6dbae615-f8d0-4154-ac6c-998d81f7b84d
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/6dbae615-f8d0-4154-ac6c-998d81f7b84d",
      "name": "6dbae615-f8d0-4154-ac6c-998d81f7b84d",
      "status": "Failed",
      "startTime": "2026-08-02T09:33:48.731562196Z",
      "endTime": "2026-08-02T09:52:59.947253927Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift

09:15:40 UTC · failed at e2e · test Customer should upgrade a nodepool to a version without Cincinnati upgrade edge z-stream upgrade without Cincinnati edge in 4.20 · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:314]: failed to create HCP cluster np-noedge-b59ll2
Unexpected error:
    <*fmt.wrapError | 0xc000ddd1e0>: 
    failed to create HCP cluster np-noedge-b59ll2: failed waiting for cluster="np-noedge-b59ll2" in resourcegroup="rg-np-noedge-b59ll2-p7jkwbdc422s" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/1654762e-f7e5-4618-a61f-b73443f5f202
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/1654762e-f7e5-4618-a61f-b73443f5f202",
      "name": "1654762e-f7e5-4618-a61f-b73443f5f202",
      "status": "Failed",
      "startTime": "2026-08-02T10:26:18.049472465Z",
      "endTime": "2026-08-02T10:45:22.746370197Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:314]: failed to create HCP cluster np-noedge-b59ll2
Unexpected error:
    <*fmt.wrapError | 0xc000ddd1e0>: 
    failed to create HCP cluster np-noedge-b59ll2: failed waiting for cluster="np-noedge-b59ll2" in resourcegroup="rg-np-noedge-b59ll2-p7jkwbdc422s" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/1654762e-f7e5-4618-a61f-b73443f5f202
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/1654762e-f7e5-4618-a61f-b73443f5f202",
      "name": "1654762e-f7e5-4618-a61f-b73443f5f202",
      "status": "Failed",
      "startTime": "2026-08-02T10:26:18.049472465Z",
      "endTime": "2026-08-02T10:45:22.746370197Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

cluster entered terminal Failed state after role assignment deployment

09:15:40 UTC · failed at e2e · test ARO HCP Service should successfully provision cluster when MI role assignments are added after cluster creation · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:220]: Timed out after 2520.001s.
cluster should eventually succeed after role assignments are created
The function passed to Eventually failed at /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:216 with:
cluster entered terminal Failed state after role assignment deployment
Expected
    <armredhatopenshifthcp.ProvisioningState>: ...
not to equal
    <armredhatopenshifthcp.ProvisioningState>: ...
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:220]: Timed out after 2520.001s.
cluster should eventually succeed after role assignments are created
The function passed to Eventually failed at /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:216 with:
cluster entered terminal Failed state after role assignment deployment
Expected
    <armredhatopenshifthcp.ProvisioningState>: ...
not to equal
    <armredhatopenshifthcp.ProvisioningState>: ...

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift

09:15:40 UTC · failed at e2e · test Authorized CIDRs Connectivity should allow API access only from authorized VM IP · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:151]: failed to create HCP cluster "cidr-connectivity-test" with authorized CIDRs
Unexpected error:
    <*fmt.wrapError | 0xc0019052c0>: 
    failed to create HCP cluster cidr-connectivity-test: failed waiting for cluster="cidr-connectivity-test" in resourcegroup="e2e-cidr-connectivity-wmrgqh8gzn96" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/0fbe47c0-96cd-4379-9f15-8327e095b80e
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/0fbe47c0-96cd-4379-9f15-8327e095b80e",
      "name": "0fbe47c0-96cd-4379-9f15-8327e095b80e",
      "status": "Failed",
      "startTime": "2026-08-02T09:34:41.172279489Z",
      "endTime": "2026-08-02T09:53:50.007487871Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:151]: failed to create HCP cluster "cidr-connectivity-test" with authorized CIDRs
Unexpected error:
    <*fmt.wrapError | 0xc0019052c0>: 
    failed to create HCP cluster cidr-connectivity-test: failed waiting for cluster="cidr-connectivity-test" in resourcegroup="e2e-cidr-connectivity-wmrgqh8gzn96" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/0fbe47c0-96cd-4379-9f15-8327e095b80e
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/0fbe47c0-96cd-4379-9f15-8327e095b80e",
      "name": "0fbe47c0-96cd-4379-9f15-8327e095b80e",
      "status": "Failed",
      "startTime": "2026-08-02T09:34:41.172279489Z",
      "endTime": "2026-08-02T09:53:50.007487871Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift

09:15:40 UTC · failed at e2e · test Customer should be able to create several HCP clusters in their customer resource group, but not in the same managed resource group · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:131]: failed to wait for first cluster "basic-hcp-cluster" to complete creation (timeout '20.000000' minutes)
Unexpected error:
    <*exported.ResponseError | 0xc001475b00>: 
    GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/a3dde528-d0eb-458a-a5eb-c741c73cfea4
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/a3dde528-d0eb-458a-a5eb-c741c73cfea4",
      "name": "a3dde528-d0eb-458a-a5eb-c741c73cfea4",
      "status": "Failed",
      "startTime": "2026-08-02T09:35:34.710183543Z",
      "endTime": "2026-08-02T09:54:39.901296729Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:131]: failed to wait for first cluster "basic-hcp-cluster" to complete creation (timeout '20.000000' minutes)
Unexpected error:
    <*exported.ResponseError | 0xc001475b00>: 
    GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/a3dde528-d0eb-458a-a5eb-c741c73cfea4
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/a3dde528-d0eb-458a-a5eb-c741c73cfea4",
      "name": "a3dde528-d0eb-458a-a5eb-c741c73cfea4",
      "status": "Failed",
      "startTime": "2026-08-02T09:35:34.710183543Z",
      "endTime": "2026-08-02T09:54:39.901296729Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift

09:15:40 UTC · failed at e2e · test Customer should be able to create an HCP cluster with custom autoscaling · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:98]: failed to create HCP cluster "autoscaling-hcp-cluster" with custom autoscaling
Unexpected error:
    <*fmt.wrapError | 0xc000439100>: 
    failed to create HCP cluster autoscaling-hcp-cluster: failed waiting for cluster="autoscaling-hcp-cluster" in resourcegroup="autoscaling-cluster-schbwv7s4vqj" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/433a98a2-2a48-4b1a-ad1a-2eb9672f77a2
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/433a98a2-2a48-4b1a-ad1a-2eb9672f77a2",
      "name": "433a98a2-2a48-4b1a-ad1a-2eb9672f77a2",
      "status": "Failed",
      "startTime": "2026-08-02T09:33:58.846876088Z",
      "endTime": "2026-08-02T09:53:00.063648031Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:98]: failed to create HCP cluster "autoscaling-hcp-cluster" with custom autoscaling
Unexpected error:
    <*fmt.wrapError | 0xc000439100>: 
    failed to create HCP cluster autoscaling-hcp-cluster: failed waiting for cluster="autoscaling-hcp-cluster" in resourcegroup="autoscaling-cluster-schbwv7s4vqj" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/433a98a2-2a48-4b1a-ad1a-2eb9672f77a2
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/433a98a2-2a48-4b1a-ad1a-2eb9672f77a2",
      "name": "433a98a2-2a48-4b1a-ad1a-2eb9672f77a2",
      "status": "Failed",
      "startTime": "2026-08-02T09:33:58.846876088Z",
      "endTime": "2026-08-02T09:53:00.063648031Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift

09:15:40 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 4.20 · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:110]: failed to create HCP cluster "cluster-zstream-4-20-x5275t" with version 4.20.27 on candidate channel
Unexpected error:
    <*fmt.wrapError | 0xc0010888e0>: 
    failed to create HCP cluster cluster-zstream-4-20-x5275t: failed waiting for cluster="cluster-zstream-4-20-x5275t" in resourcegroup="rg-zstream-upgrade-4-20-fs7v9z22qchl" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/6f2e46fd-96fc-4ba6-b840-436a57da57e9
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/6f2e46fd-96fc-4ba6-b840-436a57da57e9",
      "name": "6f2e46fd-96fc-4ba6-b840-436a57da57e9",
      "status": "Failed",
      "startTime": "2026-08-02T10:20:39.561995336Z",
      "endTime": "2026-08-02T10:39:42.899977522Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:110]: failed to create HCP cluster "cluster-zstream-4-20-x5275t" with version 4.20.27 on candidate channel
Unexpected error:
    <*fmt.wrapError | 0xc0010888e0>: 
    failed to create HCP cluster cluster-zstream-4-20-x5275t: failed waiting for cluster="cluster-zstream-4-20-x5275t" in resourcegroup="rg-zstream-upgrade-4-20-fs7v9z22qchl" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/6f2e46fd-96fc-4ba6-b840-436a57da57e9
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/6f2e46fd-96fc-4ba6-b840-436a57da57e9",
      "name": "6f2e46fd-96fc-4ba6-b840-436a57da57e9",
      "status": "Failed",
      "startTime": "2026-08-02T10:20:39.561995336Z",
      "endTime": "2026-08-02T10:39:42.899977522Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift

09:15:40 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 4.19 · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:110]: failed to create HCP cluster "cluster-zstream-4-19-nhx2wp" with version 4.19.39 on candidate channel
Unexpected error:
    <*fmt.wrapError | 0xc00097aa00>: 
    failed to create HCP cluster cluster-zstream-4-19-nhx2wp: failed waiting for cluster="cluster-zstream-4-19-nhx2wp" in resourcegroup="rg-zstream-upgrade-4-19-rn966m5vc2n7" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/3263c87f-1b2a-4019-8a07-2574b3e4a978
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/3263c87f-1b2a-4019-8a07-2574b3e4a978",
      "name": "3263c87f-1b2a-4019-8a07-2574b3e4a978",
      "status": "Failed",
      "startTime": "2026-08-02T10:20:38.264467464Z",
      "endTime": "2026-08-02T10:39:42.477082291Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:110]: failed to create HCP cluster "cluster-zstream-4-19-nhx2wp" with version 4.19.39 on candidate channel
Unexpected error:
    <*fmt.wrapError | 0xc00097aa00>: 
    failed to create HCP cluster cluster-zstream-4-19-nhx2wp: failed waiting for cluster="cluster-zstream-4-19-nhx2wp" in resourcegroup="rg-zstream-upgrade-4-19-rn966m5vc2n7" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/3263c87f-1b2a-4019-8a07-2574b3e4a978
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/3263c87f-1b2a-4019-8a07-2574b3e4a978",
      "name": "3263c87f-1b2a-4019-8a07-2574b3e4a978",
      "status": "Failed",
      "startTime": "2026-08-02T10:20:38.264467464Z",
      "endTime": "2026-08-02T10:39:42.477082291Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

timeout during CreateHCPClusterAndWait; context deadline exceeded

09:15:40 UTC · failed at e2e · test Customer should be able to create an HCP cluster with back-level version 4.19 · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:149]: failed to create HCP cluster version 4.19
Unexpected error:
    <*fmt.wrapErrors | 0xc001412750>: 
    failed waiting for cluster="cluster-ver-4-19" in resourcegroup="rg-cluster-back-version-sf8fnq7ckvqz" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterAndWait for cluster cluster-ver-4-19 in resource group rg-cluster-back-version-sf8fnq7ckvqz, error: context deadline exceeded
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:149]: failed to create HCP cluster version 4.19
Unexpected error:
    <*fmt.wrapErrors | 0xc001412750>: 
    failed waiting for cluster="cluster-ver-4-19" in resourcegroup="rg-cluster-back-version-sf8fnq7ckvqz" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterAndWait for cluster cluster-ver-4-19 in resource group rg-cluster-back-version-sf8fnq7ckvqz, error: context deadline exceeded
    ...
occurred

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift

09:15:40 UTC · failed at e2e · test Customer should create an HCP cluster and validate TLS certificates · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:95]: failed to create HCP cluster for TLS endpoint test
Unexpected error:
    <*fmt.wrapError | 0xc000afc3c0>: 
    failed to create HCP cluster tls-endpoint-hcp-cluster: failed waiting for cluster="tls-endpoint-hcp-cluster" in resourcegroup="tls-endpoint-cluster-4klq7d2pktl6" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/99457e0f-0b14-423b-bb59-8608f2ea25b8
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/99457e0f-0b14-423b-bb59-8608f2ea25b8",
      "name": "99457e0f-0b14-423b-bb59-8608f2ea25b8",
      "status": "Failed",
      "startTime": "2026-08-02T09:34:03.731452984Z",
      "endTime": "2026-08-02T09:53:20.117946772Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:95]: failed to create HCP cluster for TLS endpoint test
Unexpected error:
    <*fmt.wrapError | 0xc000afc3c0>: 
    failed to create HCP cluster tls-endpoint-hcp-cluster: failed waiting for cluster="tls-endpoint-hcp-cluster" in resourcegroup="tls-endpoint-cluster-4klq7d2pktl6" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/99457e0f-0b14-423b-bb59-8608f2ea25b8
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/99457e0f-0b14-423b-bb59-8608f2ea25b8",
      "name": "99457e0f-0b14-423b-bb59-8608f2ea25b8",
      "status": "Failed",
      "startTime": "2026-08-02T09:34:03.731452984Z",
      "endTime": "2026-08-02T09:53:20.117946772Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift

09:15:40 UTC · failed at e2e · test Customer should be able to create an HCP cluster and custom node pool osDisk size · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_nodepool_osdisk.go:80]: failed to create HCP cluster "hcp-cluster-np-128"
Unexpected error:
    <*fmt.wrapError | 0xc000dd1360>: 
    failed to create HCP cluster hcp-cluster-np-128: failed waiting for cluster="hcp-cluster-np-128" in resourcegroup="clusternp128-zrbkqqp97x2s" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/89700ddf-4dea-44e4-88e8-5b32393432ad
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/89700ddf-4dea-44e4-88e8-5b32393432ad",
      "name": "89700ddf-4dea-44e4-88e8-5b32393432ad",
      "status": "Failed",
      "startTime": "2026-08-02T09:34:57.46338231Z",
      "endTime": "2026-08-02T09:54:10.022162917Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_nodepool_osdisk.go:80]: failed to create HCP cluster "hcp-cluster-np-128"
Unexpected error:
    <*fmt.wrapError | 0xc000dd1360>: 
    failed to create HCP cluster hcp-cluster-np-128: failed waiting for cluster="hcp-cluster-np-128" in resourcegroup="clusternp128-zrbkqqp97x2s" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/89700ddf-4dea-44e4-88e8-5b32393432ad
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/89700ddf-4dea-44e4-88e8-5b32393432ad",
      "name": "89700ddf-4dea-44e4-88e8-5b32393432ad",
      "status": "Failed",
      "startTime": "2026-08-02T09:34:57.46338231Z",
      "endTime": "2026-08-02T09:54:10.022162917Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift

09:15:40 UTC · failed at e2e · test Update HCPOpenShiftCluster Negative creates a cluster and fails to update its name with a PATCH request · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:84]: failed to create HCP cluster for patch-name test
Unexpected error:
    <*fmt.wrapError | 0xc000946480>: 
    failed to create HCP cluster patch-name-cluster: failed waiting for cluster="patch-name-cluster" in resourcegroup="patch-name-b8tzdmst7fvv" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/d9310a88-789a-413f-94fe-bbd77de07f32
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/d9310a88-789a-413f-94fe-bbd77de07f32",
      "name": "d9310a88-789a-413f-94fe-bbd77de07f32",
      "status": "Failed",
      "startTime": "2026-08-02T09:34:21.5378839Z",
      "endTime": "2026-08-02T09:53:39.994858773Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:84]: failed to create HCP cluster for patch-name test
Unexpected error:
    <*fmt.wrapError | 0xc000946480>: 
    failed to create HCP cluster patch-name-cluster: failed waiting for cluster="patch-name-cluster" in resourcegroup="patch-name-b8tzdmst7fvv" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/d9310a88-789a-413f-94fe-bbd77de07f32
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/d9310a88-789a-413f-94fe-bbd77de07f32",
      "name": "d9310a88-789a-413f-94fe-bbd77de07f32",
      "status": "Failed",
      "startTime": "2026-08-02T09:34:21.5378839Z",
      "endTime": "2026-08-02T09:53:39.994858773Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated; provider Microsoft.RedHatOpenShift

09:15:40 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.22 · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:121]: HCP cluster rg-candidate-4-22-2xrctt6gqvhc/cluster-candidate-4-22-jzgl67 should provision
Unexpected error:
    <*fmt.wrapError | 0xc0005109e0>: 
    failed to create HCP cluster cluster-candidate-4-22-jzgl67: failed waiting for cluster="cluster-candidate-4-22-jzgl67" in resourcegroup="rg-candidate-4-22-2xrctt6gqvhc" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/b0d09561-daaf-4fc0-ada1-2d89894ad25d
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/b0d09561-daaf-4fc0-ada1-2d89894ad25d",
      "name": "b0d09561-daaf-4fc0-ada1-2d89894ad25d",
      "status": "Failed",
      "startTime": "2026-08-02T10:19:58.698754411Z",
      "endTime": "2026-08-02T10:39:02.899668713Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:121]: HCP cluster rg-candidate-4-22-2xrctt6gqvhc/cluster-candidate-4-22-jzgl67 should provision
Unexpected error:
    <*fmt.wrapError | 0xc0005109e0>: 
    failed to create HCP cluster cluster-candidate-4-22-jzgl67: failed waiting for cluster="cluster-candidate-4-22-jzgl67" in resourcegroup="rg-candidate-4-22-2xrctt6gqvhc" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/b0d09561-daaf-4fc0-ada1-2d89894ad25d
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/b0d09561-daaf-4fc0-ada1-2d89894ad25d",
      "name": "b0d09561-daaf-4fc0-ada1-2d89894ad25d",
      "status": "Failed",
      "startTime": "2026-08-02T10:19:58.698754411Z",
      "endTime": "2026-08-02T10:39:02.899668713Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift

09:15:40 UTC · failed at e2e · test Customer should be able to lifecycle and confirm external auth on a cluster · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_list_and_verify.go:87]: failed to create HCP cluster for external auth list test
Unexpected error:
    <*fmt.wrapError | 0xc000f88a80>: 
    failed to create HCP cluster ea-listg8p2lr: failed waiting for cluster="ea-listg8p2lr" in resourcegroup="ea-list-zpwzzw4hr4s4" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/f8f6fba3-8dc3-43e1-a8de-b639f8946093
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/f8f6fba3-8dc3-43e1-a8de-b639f8946093",
      "name": "f8f6fba3-8dc3-43e1-a8de-b639f8946093",
      "status": "Failed",
      "startTime": "2026-08-02T10:22:17.745429228Z",
      "endTime": "2026-08-02T10:41:22.679642027Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_list_and_verify.go:87]: failed to create HCP cluster for external auth list test
Unexpected error:
    <*fmt.wrapError | 0xc000f88a80>: 
    failed to create HCP cluster ea-listg8p2lr: failed waiting for cluster="ea-listg8p2lr" in resourcegroup="ea-list-zpwzzw4hr4s4" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/f8f6fba3-8dc3-43e1-a8de-b639f8946093
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/f8f6fba3-8dc3-43e1-a8de-b639f8946093",
      "name": "f8f6fba3-8dc3-43e1-a8de-b639f8946093",
      "status": "Failed",
      "startTime": "2026-08-02T10:22:17.745429228Z",
      "endTime": "2026-08-02T10:41:22.679642027Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift

09:15:40 UTC · failed at e2e · test Customer should be able to create a cluster with default autoscaling and a nodepool with autoscaling enabled up to replica limits · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:102]: failed to create HCP cluster np-autoscale-cluster
Unexpected error:
    <*fmt.wrapError | 0xc000f58fa0>: 
    failed to create HCP cluster np-autoscale-cluster: failed waiting for cluster="np-autoscale-cluster" in resourcegroup="np-autoscaling-nqr8kk5t28mq" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/9a6942f7-8b76-41f6-9814-53dcafaf62c2
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/9a6942f7-8b76-41f6-9814-53dcafaf62c2",
      "name": "9a6942f7-8b76-41f6-9814-53dcafaf62c2",
      "status": "Failed",
      "startTime": "2026-08-02T10:21:01.532783895Z",
      "endTime": "2026-08-02T10:40:02.630115585Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:102]: failed to create HCP cluster np-autoscale-cluster
Unexpected error:
    <*fmt.wrapError | 0xc000f58fa0>: 
    failed to create HCP cluster np-autoscale-cluster: failed waiting for cluster="np-autoscale-cluster" in resourcegroup="np-autoscaling-nqr8kk5t28mq" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/9a6942f7-8b76-41f6-9814-53dcafaf62c2
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/9a6942f7-8b76-41f6-9814-53dcafaf62c2",
      "name": "9a6942f7-8b76-41f6-9814-53dcafaf62c2",
      "status": "Failed",
      "startTime": "2026-08-02T10:21:01.532783895Z",
      "endTime": "2026-08-02T10:40:02.630115585Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift

09:15:40 UTC · failed at e2e · test Customer should be able to create a HCP cluster and use cilium CNI plugin · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_cni_cilium.go:88]: failed to create HCP cluster "cilium-cl" without CNI
Unexpected error:
    <*fmt.wrapError | 0xc00118f360>: 
    failed to create HCP cluster cilium-cl: failed waiting for cluster="cilium-cl" in resourcegroup="cni-cilium-5shxkc7h96kd" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/f4e10e32-a5a2-48c8-bb8f-add91194d45d
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/f4e10e32-a5a2-48c8-bb8f-add91194d45d",
      "name": "f4e10e32-a5a2-48c8-bb8f-add91194d45d",
      "status": "Failed",
      "startTime": "2026-08-02T09:34:57.74262599Z",
      "endTime": "2026-08-02T09:54:00.011864472Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_cni_cilium.go:88]: failed to create HCP cluster "cilium-cl" without CNI
Unexpected error:
    <*fmt.wrapError | 0xc00118f360>: 
    failed to create HCP cluster cilium-cl: failed waiting for cluster="cilium-cl" in resourcegroup="cni-cilium-5shxkc7h96kd" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/f4e10e32-a5a2-48c8-bb8f-add91194d45d
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/f4e10e32-a5a2-48c8-bb8f-add91194d45d",
      "name": "f4e10e32-a5a2-48c8-bb8f-add91194d45d",
      "status": "Failed",
      "startTime": "2026-08-02T09:34:57.74262599Z",
      "endTime": "2026-08-02T09:54:00.011864472Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift

09:15:40 UTC · failed at e2e · test Customer should be able to successfully upgrade control plane minor version from 4.20 minor to 4.21 minor · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:182]: failed to create HCP cluster "cp-ystream-upgrade-4-21-c4zqpd" with install version 4.20.28
Unexpected error:
    <*fmt.wrapError | 0xc00069ce00>: 
    failed to create HCP cluster cp-ystream-upgrade-4-21-c4zqpd: failed waiting for cluster="cp-ystream-upgrade-4-21-c4zqpd" in resourcegroup="rg-cp-ystream-upgrade-4-21-b4dtm4754scw" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/ebb17e58-f37d-456d-8dda-e04ad5a89ffd
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/ebb17e58-f37d-456d-8dda-e04ad5a89ffd",
      "name": "ebb17e58-f37d-456d-8dda-e04ad5a89ffd",
      "status": "Failed",
      "startTime": "2026-08-02T10:21:55.844645935Z",
      "endTime": "2026-08-02T10:41:02.499727821Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:182]: failed to create HCP cluster "cp-ystream-upgrade-4-21-c4zqpd" with install version 4.20.28
Unexpected error:
    <*fmt.wrapError | 0xc00069ce00>: 
    failed to create HCP cluster cp-ystream-upgrade-4-21-c4zqpd: failed waiting for cluster="cp-ystream-upgrade-4-21-c4zqpd" in resourcegroup="rg-cp-ystream-upgrade-4-21-b4dtm4754scw" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/ebb17e58-f37d-456d-8dda-e04ad5a89ffd
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/ebb17e58-f37d-456d-8dda-e04ad5a89ffd",
      "name": "ebb17e58-f37d-456d-8dda-e04ad5a89ffd",
      "status": "Failed",
      "startTime": "2026-08-02T10:21:55.844645935Z",
      "endTime": "2026-08-02T10:41:02.499727821Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

09:15:40 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.21 · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:121]: HCP cluster rg-candidate-4-21-thgbrpsnmsdd/cluster-candidate-4-21-vvlhtd should provision
Unexpected error:
    <*fmt.wrapError | 0xc000b94c00>: 
    failed to create HCP cluster cluster-candidate-4-21-vvlhtd: failed waiting for cluster="cluster-candidate-4-21-vvlhtd" in resourcegroup="rg-candidate-4-21-thgbrpsnmsdd" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/cfe40609-dfaa-4309-a817-b449ab573e51
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/cfe40609-dfaa-4309-a817-b449ab573e51",
      "name": "cfe40609-dfaa-4309-a817-b449ab573e51",
      "status": "Failed",
      "startTime": "2026-08-02T09:40:52.659578391Z",
      "endTime": "2026-08-02T10:00:01.276872508Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:121]: HCP cluster rg-candidate-4-21-thgbrpsnmsdd/cluster-candidate-4-21-vvlhtd should provision
Unexpected error:
    <*fmt.wrapError | 0xc000b94c00>: 
    failed to create HCP cluster cluster-candidate-4-21-vvlhtd: failed waiting for cluster="cluster-candidate-4-21-vvlhtd" in resourcegroup="rg-candidate-4-21-thgbrpsnmsdd" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/cfe40609-dfaa-4309-a817-b449ab573e51
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/cfe40609-dfaa-4309-a817-b449ab573e51",
      "name": "cfe40609-dfaa-4309-a817-b449ab573e51",
      "status": "Failed",
      "startTime": "2026-08-02T09:40:52.659578391Z",
      "endTime": "2026-08-02T10:00:01.276872508Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift

09:15:40 UTC · failed at e2e · test Customer should upgrade and update a nodepool from 4.20.z to 4.20.zLatest · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:133]: failed to create HCP cluster np-version-upgrade-cluster-7tjrwf with version 4.20.32
Unexpected error:
    <*fmt.wrapError | 0xc0009fc860>: 
    failed to create HCP cluster np-version-upgrade-cluster-7tjrwf: failed waiting for cluster="np-version-upgrade-cluster-7tjrwf" in resourcegroup="rg-np-version-upgrade-7tjrwf-gb7vxqn6kj8r" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/5f96cdc6-19a6-49b1-97a9-d42ab5c04593
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/5f96cdc6-19a6-49b1-97a9-d42ab5c04593",
      "name": "5f96cdc6-19a6-49b1-97a9-d42ab5c04593",
      "status": "Failed",
      "startTime": "2026-08-02T10:25:02.326268128Z",
      "endTime": "2026-08-02T10:44:02.561752064Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:133]: failed to create HCP cluster np-version-upgrade-cluster-7tjrwf with version 4.20.32
Unexpected error:
    <*fmt.wrapError | 0xc0009fc860>: 
    failed to create HCP cluster np-version-upgrade-cluster-7tjrwf: failed waiting for cluster="np-version-upgrade-cluster-7tjrwf" in resourcegroup="rg-np-version-upgrade-7tjrwf-gb7vxqn6kj8r" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/5f96cdc6-19a6-49b1-97a9-d42ab5c04593
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/5f96cdc6-19a6-49b1-97a9-d42ab5c04593",
      "name": "5f96cdc6-19a6-49b1-97a9-d42ab5c04593",
      "status": "Failed",
      "startTime": "2026-08-02T10:25:02.326268128Z",
      "endTime": "2026-08-02T10:44:02.561752064Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift

09:15:40 UTC · failed at e2e · test Update HCPOpenShiftCluster Positive creates a cluster and updates tags with a PATCH request · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:161]: failed to create HCP cluster for patch-tags test
Unexpected error:
    <*fmt.wrapError | 0xc0001bc3c0>: 
    failed to create HCP cluster patch-tags-cluster: failed waiting for cluster="patch-tags-cluster" in resourcegroup="patch-tags-z7vrqxfs7z6k" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/96eaab45-e218-47ab-8385-511d2b8bea05
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/96eaab45-e218-47ab-8385-511d2b8bea05",
      "name": "96eaab45-e218-47ab-8385-511d2b8bea05",
      "status": "Failed",
      "startTime": "2026-08-02T09:34:01.062473396Z",
      "endTime": "2026-08-02T09:53:20.146592724Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:161]: failed to create HCP cluster for patch-tags test
Unexpected error:
    <*fmt.wrapError | 0xc0001bc3c0>: 
    failed to create HCP cluster patch-tags-cluster: failed waiting for cluster="patch-tags-cluster" in resourcegroup="patch-tags-z7vrqxfs7z6k" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/96eaab45-e218-47ab-8385-511d2b8bea05
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/96eaab45-e218-47ab-8385-511d2b8bea05",
      "name": "96eaab45-e218-47ab-8385-511d2b8bea05",
      "status": "Failed",
      "startTime": "2026-08-02T09:34:01.062473396Z",
      "endTime": "2026-08-02T09:53:20.146592724Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift

09:15:40 UTC · failed at e2e · test Customer should be able to update node pool labels and taints · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_labels_taints.go:92]: failed to create HCP cluster cluster-np-labels-taints
Unexpected error:
    <*fmt.wrapError | 0xc000e08ee0>: 
    failed to create HCP cluster cluster-np-labels-taints: failed waiting for cluster="cluster-np-labels-taints" in resourcegroup="rg-np-labels-taints-m6z62b4jc2tx" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/37f499cd-df85-402b-81c7-871df8ababcf
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/37f499cd-df85-402b-81c7-871df8ababcf",
      "name": "37f499cd-df85-402b-81c7-871df8ababcf",
      "status": "Failed",
      "startTime": "2026-08-02T10:21:11.158027917Z",
      "endTime": "2026-08-02T10:40:12.70290935Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_labels_taints.go:92]: failed to create HCP cluster cluster-np-labels-taints
Unexpected error:
    <*fmt.wrapError | 0xc000e08ee0>: 
    failed to create HCP cluster cluster-np-labels-taints: failed waiting for cluster="cluster-np-labels-taints" in resourcegroup="rg-np-labels-taints-m6z62b4jc2tx" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/37f499cd-df85-402b-81c7-871df8ababcf
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/37f499cd-df85-402b-81c7-871df8ababcf",
      "name": "37f499cd-df85-402b-81c7-871df8ababcf",
      "status": "Failed",
      "startTime": "2026-08-02T10:21:11.158027917Z",
      "endTime": "2026-08-02T10:40:12.70290935Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift

09:15:40 UTC · failed at e2e · test Customer should upgrade and update a nodepool from 4.20.z to 4.21.zLatest · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:133]: failed to create HCP cluster np-version-upgrade-cluster-4c9krr with version 4.21.23
Unexpected error:
    <*fmt.wrapError | 0xc000d80ec0>: 
    failed to create HCP cluster np-version-upgrade-cluster-4c9krr: failed waiting for cluster="np-version-upgrade-cluster-4c9krr" in resourcegroup="rg-np-version-upgrade-4c9krr-wrfbrlj4lbbr" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/146bbd4a-743f-4661-9c79-1a8d6a5e97cd
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/146bbd4a-743f-4661-9c79-1a8d6a5e97cd",
      "name": "146bbd4a-743f-4661-9c79-1a8d6a5e97cd",
      "status": "Failed",
      "startTime": "2026-08-02T10:22:48.128797596Z",
      "endTime": "2026-08-02T10:41:52.721915786Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:133]: failed to create HCP cluster np-version-upgrade-cluster-4c9krr with version 4.21.23
Unexpected error:
    <*fmt.wrapError | 0xc000d80ec0>: 
    failed to create HCP cluster np-version-upgrade-cluster-4c9krr: failed waiting for cluster="np-version-upgrade-cluster-4c9krr" in resourcegroup="rg-np-version-upgrade-4c9krr-wrfbrlj4lbbr" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/146bbd4a-743f-4661-9c79-1a8d6a5e97cd
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/146bbd4a-743f-4661-9c79-1a8d6a5e97cd",
      "name": "146bbd4a-743f-4661-9c79-1a8d6a5e97cd",
      "status": "Failed",
      "startTime": "2026-08-02T10:22:48.128797596Z",
      "endTime": "2026-08-02T10:41:52.721915786Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

09:15:40 UTC · failed at e2e · test Customer should be able to create an HCP cluster and manage ImageDigestMirrors · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/idms_lifecycle.go:113]: failed to create HCP cluster with ImageDigestMirrors
Unexpected error:
    <*fmt.wrapError | 0xc000954860>: 
    failed to create HCP cluster idms-e2e-hcp-cluster: failed waiting for cluster="idms-e2e-hcp-cluster" in resourcegroup="idms-qzhvx5bbqbk9" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/016f96d1-550a-41f2-9fc1-b4db6bb899c9
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/016f96d1-550a-41f2-9fc1-b4db6bb899c9",
      "name": "016f96d1-550a-41f2-9fc1-b4db6bb899c9",
      "status": "Failed",
      "startTime": "2026-08-02T10:21:25.833585484Z",
      "endTime": "2026-08-02T10:40:42.522944424Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/idms_lifecycle.go:113]: failed to create HCP cluster with ImageDigestMirrors
Unexpected error:
    <*fmt.wrapError | 0xc000954860>: 
    failed to create HCP cluster idms-e2e-hcp-cluster: failed waiting for cluster="idms-e2e-hcp-cluster" in resourcegroup="idms-qzhvx5bbqbk9" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/016f96d1-550a-41f2-9fc1-b4db6bb899c9
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/016f96d1-550a-41f2-9fc1-b4db6bb899c9",
      "name": "016f96d1-550a-41f2-9fc1-b4db6bb899c9",
      "status": "Failed",
      "startTime": "2026-08-02T10:21:25.833585484Z",
      "endTime": "2026-08-02T10:40:42.522944424Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift

09:15:40 UTC · failed at e2e · test Customer should be able to create a cluster with an external auth config and get the external auth config · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:94]: failed to create HCP cluster for external auth test
Unexpected error:
    <*fmt.wrapError | 0xc0012d48e0>: 
    failed to create HCP cluster ea-cluster: failed waiting for cluster="ea-cluster" in resourcegroup="external-auth-cluster-fqw8f7c4wwz6" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/24c83ba8-3e7d-4e9d-acd7-818541d03e5b
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/24c83ba8-3e7d-4e9d-acd7-818541d03e5b",
      "name": "24c83ba8-3e7d-4e9d-acd7-818541d03e5b",
      "status": "Failed",
      "startTime": "2026-08-02T10:22:16.371732004Z",
      "endTime": "2026-08-02T10:41:22.659684191Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:94]: failed to create HCP cluster for external auth test
Unexpected error:
    <*fmt.wrapError | 0xc0012d48e0>: 
    failed to create HCP cluster ea-cluster: failed waiting for cluster="ea-cluster" in resourcegroup="external-auth-cluster-fqw8f7c4wwz6" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/24c83ba8-3e7d-4e9d-acd7-818541d03e5b
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/24c83ba8-3e7d-4e9d-acd7-818541d03e5b",
      "name": "24c83ba8-3e7d-4e9d-acd7-818541d03e5b",
      "status": "Failed",
      "startTime": "2026-08-02T10:22:16.371732004Z",
      "endTime": "2026-08-02T10:41:22.659684191Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift

09:15:40 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 4.22 · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:110]: failed to create HCP cluster "cluster-zstream-4-22-qm2jhm" with version 4.22.6 on candidate channel
Unexpected error:
    <*fmt.wrapError | 0xc000f14c40>: 
    failed to create HCP cluster cluster-zstream-4-22-qm2jhm: failed waiting for cluster="cluster-zstream-4-22-qm2jhm" in resourcegroup="rg-zstream-upgrade-4-22-7prtcnrz9p7l" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/604d2e18-9f93-40ae-8c59-0ae91f105f7a
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/604d2e18-9f93-40ae-8c59-0ae91f105f7a",
      "name": "604d2e18-9f93-40ae-8c59-0ae91f105f7a",
      "status": "Failed",
      "startTime": "2026-08-02T10:20:29.396777978Z",
      "endTime": "2026-08-02T10:39:42.921782454Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:110]: failed to create HCP cluster "cluster-zstream-4-22-qm2jhm" with version 4.22.6 on candidate channel
Unexpected error:
    <*fmt.wrapError | 0xc000f14c40>: 
    failed to create HCP cluster cluster-zstream-4-22-qm2jhm: failed waiting for cluster="cluster-zstream-4-22-qm2jhm" in resourcegroup="rg-zstream-upgrade-4-22-7prtcnrz9p7l" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/604d2e18-9f93-40ae-8c59-0ae91f105f7a
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/604d2e18-9f93-40ae-8c59-0ae91f105f7a",
      "name": "604d2e18-9f93-40ae-8c59-0ae91f105f7a",
      "status": "Failed",
      "startTime": "2026-08-02T10:20:29.396777978Z",
      "endTime": "2026-08-02T10:39:42.921782454Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

09:15:40 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.20 · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:121]: HCP cluster rg-candidate-4-20-p79b4rkhtstx/cluster-candidate-4-20-qrm8jx should provision
Unexpected error:
    <*fmt.wrapError | 0xc000a92d80>: 
    failed to create HCP cluster cluster-candidate-4-20-qrm8jx: failed waiting for cluster="cluster-candidate-4-20-qrm8jx" in resourcegroup="rg-candidate-4-20-p79b4rkhtstx" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/8bade74f-a4fb-450a-a9c8-31a90fb5df6a
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/8bade74f-a4fb-450a-a9c8-31a90fb5df6a",
      "name": "8bade74f-a4fb-450a-a9c8-31a90fb5df6a",
      "status": "Failed",
      "startTime": "2026-08-02T09:35:10.119552619Z",
      "endTime": "2026-08-02T09:54:10.020290658Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:121]: HCP cluster rg-candidate-4-20-p79b4rkhtstx/cluster-candidate-4-20-qrm8jx should provision
Unexpected error:
    <*fmt.wrapError | 0xc000a92d80>: 
    failed to create HCP cluster cluster-candidate-4-20-qrm8jx: failed waiting for cluster="cluster-candidate-4-20-qrm8jx" in resourcegroup="rg-candidate-4-20-p79b4rkhtstx" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/8bade74f-a4fb-450a-a9c8-31a90fb5df6a
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/8bade74f-a4fb-450a-a9c8-31a90fb5df6a",
      "name": "8bade74f-a4fb-450a-a9c8-31a90fb5df6a",
      "status": "Failed",
      "startTime": "2026-08-02T09:35:10.119552619Z",
      "endTime": "2026-08-02T09:54:10.020290658Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift

09:15:40 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 4.21 · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:110]: failed to create HCP cluster "cluster-zstream-4-21-wpfbzb" with version 4.21.22 on candidate channel
Unexpected error:
    <*fmt.wrapError | 0xc000eb0240>: 
    failed to create HCP cluster cluster-zstream-4-21-wpfbzb: failed waiting for cluster="cluster-zstream-4-21-wpfbzb" in resourcegroup="rg-zstream-upgrade-4-21-hqwpx2rp9hwp" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/5a86e2da-8342-4e05-9a7b-646fec5d1f57
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/5a86e2da-8342-4e05-9a7b-646fec5d1f57",
      "name": "5a86e2da-8342-4e05-9a7b-646fec5d1f57",
      "status": "Failed",
      "startTime": "2026-08-02T10:20:45.199672861Z",
      "endTime": "2026-08-02T10:39:52.85500071Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:110]: failed to create HCP cluster "cluster-zstream-4-21-wpfbzb" with version 4.21.22 on candidate channel
Unexpected error:
    <*fmt.wrapError | 0xc000eb0240>: 
    failed to create HCP cluster cluster-zstream-4-21-wpfbzb: failed waiting for cluster="cluster-zstream-4-21-wpfbzb" in resourcegroup="rg-zstream-upgrade-4-21-hqwpx2rp9hwp" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/5a86e2da-8342-4e05-9a7b-646fec5d1f57
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/5a86e2da-8342-4e05-9a7b-646fec5d1f57",
      "name": "5a86e2da-8342-4e05-9a7b-646fec5d1f57",
      "status": "Failed",
      "startTime": "2026-08-02T10:20:45.199672861Z",
      "endTime": "2026-08-02T10:39:52.85500071Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift

09:15:40 UTC · failed at e2e · test Customer should be able to create an HCP cluster with Image Registry not present · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/image_registry_cluster_create.go:89]: failed to create HCP cluster with image registry disabled
Unexpected error:
    <*fmt.wrapError | 0xc0001a5360>: 
    failed to create HCP cluster disabled-image-registry-hcp-cluster: failed waiting for cluster="disabled-image-registry-hcp-cluster" in resourcegroup="disabled-image-registry-xjxxj2swjnz9" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/df395857-64f2-4cba-a384-dba5dbda1151
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/df395857-64f2-4cba-a384-dba5dbda1151",
      "name": "df395857-64f2-4cba-a384-dba5dbda1151",
      "status": "Failed",
      "startTime": "2026-08-02T10:21:04.574656397Z",
      "endTime": "2026-08-02T10:40:12.775068231Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/image_registry_cluster_create.go:89]: failed to create HCP cluster with image registry disabled
Unexpected error:
    <*fmt.wrapError | 0xc0001a5360>: 
    failed to create HCP cluster disabled-image-registry-hcp-cluster: failed waiting for cluster="disabled-image-registry-hcp-cluster" in resourcegroup="disabled-image-registry-xjxxj2swjnz9" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/df395857-64f2-4cba-a384-dba5dbda1151
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/df395857-64f2-4cba-a384-dba5dbda1151",
      "name": "df395857-64f2-4cba-a384-dba5dbda1151",
      "status": "Failed",
      "startTime": "2026-08-02T10:21:04.574656397Z",
      "endTime": "2026-08-02T10:40:12.775068231Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift

09:15:40 UTC · failed at e2e · test Customer should respect cluster-wide node limits with nodepool autoscaling · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:291]: failed to create HCP cluster node-limit-cluster with MaxNodesTotal limit
Unexpected error:
    <*fmt.wrapError | 0xc0004ae680>: 
    failed to create HCP cluster node-limit-cluster: failed waiting for cluster="node-limit-cluster" in resourcegroup="node-limit-test-774f7qvh8zdv" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/6f2628e2-cfb7-4630-92d6-e86eb6db9b46
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/6f2628e2-cfb7-4630-92d6-e86eb6db9b46",
      "name": "6f2628e2-cfb7-4630-92d6-e86eb6db9b46",
      "status": "Failed",
      "startTime": "2026-08-02T10:21:35.592173312Z",
      "endTime": "2026-08-02T10:40:42.509887887Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:291]: failed to create HCP cluster node-limit-cluster with MaxNodesTotal limit
Unexpected error:
    <*fmt.wrapError | 0xc0004ae680>: 
    failed to create HCP cluster node-limit-cluster: failed waiting for cluster="node-limit-cluster" in resourcegroup="node-limit-test-774f7qvh8zdv" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/6f2628e2-cfb7-4630-92d6-e86eb6db9b46
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/6f2628e2-cfb7-4630-92d6-e86eb6db9b46",
      "name": "6f2628e2-cfb7-4630-92d6-e86eb6db9b46",
      "status": "Failed",
      "startTime": "2026-08-02T10:21:35.592173312Z",
      "endTime": "2026-08-02T10:40:42.509887887Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

Cluster provisioning failed

09:15:40 UTC · failed at e2e · test Customer should be able to test admin credentials before cluster ready, then full admin credential lifecycle · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:158]: Cluster provisioning failed
fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:158]: Cluster provisioning failed

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift

09:15:40 UTC · failed at e2e · test Customer should upgrade and update a nodepool from 4.21.z to 4.21.zLatest · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:133]: failed to create HCP cluster np-version-upgrade-cluster-t5gbrr with version 4.21.23
Unexpected error:
    <*fmt.wrapError | 0xc00062e4c0>: 
    failed to create HCP cluster np-version-upgrade-cluster-t5gbrr: failed waiting for cluster="np-version-upgrade-cluster-t5gbrr" in resourcegroup="rg-np-version-upgrade-t5gbrr-9m92wdf9smth" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/a893a929-5de6-4c11-9274-877b11423a45
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/a893a929-5de6-4c11-9274-877b11423a45",
      "name": "a893a929-5de6-4c11-9274-877b11423a45",
      "status": "Failed",
      "startTime": "2026-08-02T10:22:58.969255601Z",
      "endTime": "2026-08-02T10:42:02.681869225Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:133]: failed to create HCP cluster np-version-upgrade-cluster-t5gbrr with version 4.21.23
Unexpected error:
    <*fmt.wrapError | 0xc00062e4c0>: 
    failed to create HCP cluster np-version-upgrade-cluster-t5gbrr: failed waiting for cluster="np-version-upgrade-cluster-t5gbrr" in resourcegroup="rg-np-version-upgrade-t5gbrr-9m92wdf9smth" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/a893a929-5de6-4c11-9274-877b11423a45
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/a893a929-5de6-4c11-9274-877b11423a45",
      "name": "a893a929-5de6-4c11-9274-877b11423a45",
      "status": "Failed",
      "startTime": "2026-08-02T10:22:58.969255601Z",
      "endTime": "2026-08-02T10:42:02.681869225Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift

09:15:40 UTC · failed at e2e · test Customer should not be able to reuse subnets and NSGs between clusters · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_nsg_subnet_reuse.go:107]: failed to create seed HCP cluster basic-cluster
Unexpected error:
    <*fmt.wrapError | 0xc000906a20>: 
    failed to create HCP cluster basic-cluster: failed waiting for cluster="basic-cluster" in resourcegroup="rg-cluster-nsg-subnet-reuse-n9jxzw2mh8nf" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/b2328641-a92c-449e-ae11-b4e1b37ed5a2
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/b2328641-a92c-449e-ae11-b4e1b37ed5a2",
      "name": "b2328641-a92c-449e-ae11-b4e1b37ed5a2",
      "status": "Failed",
      "startTime": "2026-08-02T09:34:07.535948206Z",
      "endTime": "2026-08-02T09:53:20.13427646Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_nsg_subnet_reuse.go:107]: failed to create seed HCP cluster basic-cluster
Unexpected error:
    <*fmt.wrapError | 0xc000906a20>: 
    failed to create HCP cluster basic-cluster: failed waiting for cluster="basic-cluster" in resourcegroup="rg-cluster-nsg-subnet-reuse-n9jxzw2mh8nf" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/b2328641-a92c-449e-ae11-b4e1b37ed5a2
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/b2328641-a92c-449e-ae11-b4e1b37ed5a2",
      "name": "b2328641-a92c-449e-ae11-b4e1b37ed5a2",
      "status": "Failed",
      "startTime": "2026-08-02T09:34:07.535948206Z",
      "endTime": "2026-08-02T09:53:20.13427646Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

09:15:40 UTC · failed at e2e · test Create HCPOpenShiftCluster with Private KeyVault should create a cluster with private keyvault using v20251223preview API · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:98]: failed to create HCP cluster "private-kv-cluster" with private keyvault
Unexpected error:
    <*fmt.wrapError | 0xc000aa8d00>: 
    failed waiting for cluster="private-kv-cluster" in resourcegroup="private-keyvault-xxd89b5gsd7p" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/6d9000a4-f892-48f3-beba-a01dcf970841
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/6d9000a4-f892-48f3-beba-a01dcf970841",
      "name": "6d9000a4-f892-48f3-beba-a01dcf970841",
      "status": "Failed",
      "startTime": "2026-08-02T09:38:30.54644304Z",
      "endTime": "2026-08-02T09:57:39.932107313Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:98]: failed to create HCP cluster "private-kv-cluster" with private keyvault
Unexpected error:
    <*fmt.wrapError | 0xc000aa8d00>: 
    failed waiting for cluster="private-kv-cluster" in resourcegroup="private-keyvault-xxd89b5gsd7p" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/6d9000a4-f892-48f3-beba-a01dcf970841
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/6d9000a4-f892-48f3-beba-a01dcf970841",
      "name": "6d9000a4-f892-48f3-beba-a01dcf970841",
      "status": "Failed",
      "startTime": "2026-08-02T09:38:30.54644304Z",
      "endTime": "2026-08-02T09:57:39.932107313Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift

09:15:40 UTC · failed at e2e · test Nodepool Ephemeral OS Disk should create a nodepool with ephemeral OS disk when autoRepair is enabled · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_ephemeral_osdisk.go:112]: failed to create HCP cluster ephemeral-disk
Unexpected error:
    <*fmt.wrapError | 0xc000f569c0>: 
    failed to create HCP cluster ephemeral-disk: failed waiting for cluster="ephemeral-disk" in resourcegroup="ephemeral-osdisk-tpg49l4jbx5d" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/565715c5-d553-4e22-9e42-0fc997a2a44c
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/565715c5-d553-4e22-9e42-0fc997a2a44c",
      "name": "565715c5-d553-4e22-9e42-0fc997a2a44c",
      "status": "Failed",
      "startTime": "2026-08-02T10:21:22.100482516Z",
      "endTime": "2026-08-02T10:40:22.835720238Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_ephemeral_osdisk.go:112]: failed to create HCP cluster ephemeral-disk
Unexpected error:
    <*fmt.wrapError | 0xc000f569c0>: 
    failed to create HCP cluster ephemeral-disk: failed waiting for cluster="ephemeral-disk" in resourcegroup="ephemeral-osdisk-tpg49l4jbx5d" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/565715c5-d553-4e22-9e42-0fc997a2a44c
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/565715c5-d553-4e22-9e42-0fc997a2a44c",
      "name": "565715c5-d553-4e22-9e42-0fc997a2a44c",
      "status": "Failed",
      "startTime": "2026-08-02T10:21:22.100482516Z",
      "endTime": "2026-08-02T10:40:22.835720238Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

09:15:40 UTC · failed at e2e · test Customer should be able to create a no-CNI private cluster with a private key vault, a nodepool and install cilium CNI successfully · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:105]: failed to create HCP cluster "cilium-cluster" with no CNI and private etcd
Unexpected error:
    <*fmt.wrapError | 0xc0003fe7c0>: 
    failed waiting for cluster="cilium-cluster" in resourcegroup="complex-cilium-kv-q7k5fm2cgm92" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/66774fff-682b-4c22-af08-5f9dea95118b
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/66774fff-682b-4c22-af08-5f9dea95118b",
      "name": "66774fff-682b-4c22-af08-5f9dea95118b",
      "status": "Failed",
      "startTime": "2026-08-02T09:36:47.470711734Z",
      "endTime": "2026-08-02T09:55:49.913729058Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:105]: failed to create HCP cluster "cilium-cluster" with no CNI and private etcd
Unexpected error:
    <*fmt.wrapError | 0xc0003fe7c0>: 
    failed waiting for cluster="cilium-cluster" in resourcegroup="complex-cilium-kv-q7k5fm2cgm92" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/66774fff-682b-4c22-af08-5f9dea95118b
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2euap/hcpOperationStatuses/66774fff-682b-4c22-af08-5f9dea95118b",
      "name": "66774fff-682b-4c22-af08-5f9dea95118b",
      "status": "Failed",
      "startTime": "2026-08-02T09:36:47.470711734Z",
      "endTime": "2026-08-02T09:55:49.913729058Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e; [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred

02:00:06

periodic-ci-Azure-ARO-HCP-main-periodic-prod-e2e-parallel

post-goodmerged PR

No additional run metadata captured

e2e

failed

n/a

e2e (1)

ERROR CODE: InternalServerError; detail message [clusterServiceNodePoolStatus] <no_message>; provider Microsoft.RedHatOpenShift

02:00:06 UTC · failed at e2e · test Customer should be able to create a cluster and node pools with aggregated advanced features · suite prod/parallel

Show raw failure

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:328]: failed to verify cilium is running and nodes are Ready for cluster "agg-cluster"
Unexpected error:
    <*errors.joinError | 0xc00214e018>: 
    VerifyNodesReady failed: no nodes found in the cluster
    VerifyCiliumOperational failed: not all pods in kube-system namespace are running: no cilium pods found in kube-system namespace
    failed to create NodePool agg-np-a: failed waiting for nodepool="agg-np-a" for cluster "agg-cluster" in resourcegroup="feature-aggregation-l54jll2tvrwp" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0bdec85c-5f48-447c-b8b0-4dedc7990652
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0bdec85c-5f48-447c-b8b0-4dedc7990652",
      "name": "0bdec85c-5f48-447c-b8b0-4dedc7990652",
      "status": "Failed",
      "startTime": "2026-08-02T02:20:43.022860317Z",
      "endTime": "2026-08-02T02:39:48.638662525Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceNodePoolStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:328]: failed to verify cilium is running and nodes are Ready for cluster "agg-cluster"
Unexpected error:
    <*errors.joinError | 0xc00214e018>: 
    VerifyNodesReady failed: no nodes found in the cluster
    VerifyCiliumOperational failed: not all pods in kube-system namespace are running: no cilium pods found in kube-system namespace
    failed to create NodePool agg-np-a: failed waiting for nodepool="agg-np-a" for cluster "agg-cluster" in resourcegroup="feature-aggregation-l54jll2tvrwp" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0bdec85c-5f48-447c-b8b0-4dedc7990652
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0bdec85c-5f48-447c-b8b0-4dedc7990652",
      "name": "0bdec85c-5f48-447c-b8b0-4dedc7990652",
      "status": "Failed",
      "startTime": "2026-08-02T02:20:43.022860317Z",
      "endTime": "2026-08-02T02:39:48.638662525Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceNodePoolStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred