CIHealth
<
Single Day: Aug 4▾
Pick a day
>
View JSON API
↗ Open Failure patterns for this day

Environment: STG

Time (UTC)JobFailed atResultPRFailed testsDetails
branch-ci-Azure-ARO-HCP-main-e2e-stage-e2e-parallel
post-goodmerged PR
No additional run metadata captured
 n/a passed n/a 0 n/a
periodic-ci-Azure-ARO-HCP-main-periodic-stage-e2e-parallel-ocp-nightly
post-goodmerged PR
No additional run metadata captured
 e2e failed n/a 1
e2e (1)
timeout during CreateHCPClusterAndWait; context deadline exceeded
23:00:02 UTC · failed at e2e · test Customer should create a cluster with private KAS and verify API server is only reachable from VNet while ingress remains public · suite stage/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:90]: failed to create HCP cluster "private-kas" with private KAS
Unexpected error:
    <*fmt.wrapErrors | 0xc0019c0390>: 
    failed to create HCP cluster private-kas, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster private-kas in resource group private-kas-cz72j6zw6vdm, error: failed waiting for cluster="private-kas" in resourcegroup="private-kas-cz72j6zw6vdm" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster private-kas in resource group private-kas-cz72j6zw6vdm, error: context deadline exceeded
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:90]: failed to create HCP cluster "private-kas" with private KAS
Unexpected error:
    <*fmt.wrapErrors | 0xc0019c0390>: 
    failed to create HCP cluster private-kas, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster private-kas in resource group private-kas-cz72j6zw6vdm, error: failed waiting for cluster="private-kas" in resourcegroup="private-kas-cz72j6zw6vdm" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster private-kas in resource group private-kas-cz72j6zw6vdm, error: context deadline exceeded
    ...
occurred
branch-ci-Azure-ARO-HCP-main-e2e-stage-e2e-parallel
post-goodmerged PR
No additional run metadata captured
 e2e failed n/a 2
e2e (2)
ERROR CODE: InternalServerError; detail message [clusterServiceNodePoolStatus] <no_message>; provider Microsoft.RedHatOpenShift
17:43:44 UTC · failed at e2e · test Create HCPOpenShiftCluster with Private KeyVault should create a cluster with private keyvault using v20251223preview API · suite stage/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:137]: failed to create node pool "np-1" for private keyvault cluster "private-kv-cluster"
Unexpected error:
    <*fmt.wrapError | 0xc000dda860>: 
    failed to create NodePool np-1: failed waiting for nodepool="np-1" for cluster "private-kv-cluster" in resourcegroup="private-keyvault-v48bhd59nlrn" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/72d45537-015f-4192-9dff-7d46dc3292b9
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/72d45537-015f-4192-9dff-7d46dc3292b9",
      "name": "72d45537-015f-4192-9dff-7d46dc3292b9",
      "status": "Failed",
      "startTime": "2026-08-04T18:17:08.75493953Z",
      "endTime": "2026-08-04T18:36:10.634625702Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceNodePoolStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:137]: failed to create node pool "np-1" for private keyvault cluster "private-kv-cluster"
Unexpected error:
    <*fmt.wrapError | 0xc000dda860>: 
    failed to create NodePool np-1: failed waiting for nodepool="np-1" for cluster "private-kv-cluster" in resourcegroup="private-keyvault-v48bhd59nlrn" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/72d45537-015f-4192-9dff-7d46dc3292b9
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/72d45537-015f-4192-9dff-7d46dc3292b9",
      "name": "72d45537-015f-4192-9dff-7d46dc3292b9",
      "status": "Failed",
      "startTime": "2026-08-04T18:17:08.75493953Z",
      "endTime": "2026-08-04T18:36:10.634625702Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceNodePoolStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InternalServerError; detail message cluster deletion did not complete before the deadline; [clusterServiceDeletion] ClusterService cluster still exists; [clusterServiceStatus] state is "uninstalling"; [descendantResources] remaining resources: serviceProviderClusters; provider Microsoft.RedHatOpenShift
17:43:44 UTC · failed at e2e · test ARO HCP Service should successfully provision cluster when MI role assignments are added after cluster creation · suite stage/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:293]: Unexpected error:
    <*errors.joinError | 0xc000882e28>: 
    failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster="delayed-rbac-cluster" in resourcegroup="delayed-rbac-b8pf9ghjcmwc" to finish deleting: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/7cc4d54c-e3f3-4cbf-912a-50ff57d5e112
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/7cc4d54c-e3f3-4cbf-912a-50ff57d5e112",
      "name": "7cc4d54c-e3f3-4cbf-912a-50ff57d5e112",
      "status": "Failed",
      "startTime": "2026-08-04T18:20:09.234200558Z",
      "endTime": "2026-08-04T18:44:21.610975109Z",
      "error": {
        "code": "InternalServerError",
        "message": "cluster deletion did not complete before the deadline; [clusterServiceDeletion] ClusterService cluster /api/aro_hcp/v1alpha1/clusters/2rvtm5adstsof56oe2dvgrblglv1cc6i still exists (deletion dispatched at 2026-08-04T18:20:10Z); [clusterServiceStatus] ClusterService state is \"uninstalling\"; [descendantResources] remaining resources: 1 microsoft.redhatopenshift/hcpopenshiftclusters/serviceProviderClusters"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:293]: Unexpected error:
    <*errors.joinError | 0xc000882e28>: 
    failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster="delayed-rbac-cluster" in resourcegroup="delayed-rbac-b8pf9ghjcmwc" to finish deleting: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/7cc4d54c-e3f3-4cbf-912a-50ff57d5e112
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/7cc4d54c-e3f3-4cbf-912a-50ff57d5e112",
      "name": "7cc4d54c-e3f3-4cbf-912a-50ff57d5e112",
      "status": "Failed",
      "startTime": "2026-08-04T18:20:09.234200558Z",
      "endTime": "2026-08-04T18:44:21.610975109Z",
      "error": {
        "code": "InternalServerError",
        "message": "cluster deletion did not complete before the deadline; [clusterServiceDeletion] ClusterService cluster /api/aro_hcp/v1alpha1/clusters/2rvtm5adstsof56oe2dvgrblglv1cc6i still exists (deletion dispatched at 2026-08-04T18:20:10Z); [clusterServiceStatus] ClusterService state is \"uninstalling\"; [descendantResources] remaining resources: 1 microsoft.redhatopenshift/hcpopenshiftclusters/serviceProviderClusters"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
branch-ci-Azure-ARO-HCP-main-e2e-stage-e2e-parallel
post-goodmerged PR
No additional run metadata captured
 e2e failed n/a 1
e2e (1)
ERROR CODE: InternalServerError; detail message [hypershiftHostedCluster] hosted cluster has no installed version; hosted cluster degraded: UnavailableReplicas: [router]; provider Microsoft.RedHatOpenShift
14:48:04 UTC · failed at e2e · test Customer should create a cluster with private KAS and verify API server is only reachable from VNet while ingress remains public · suite stage/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:90]: failed to create HCP cluster "private-kas" with private KAS
Unexpected error:
    <*fmt.wrapError | 0xc0014d11c0>: 
    failed to create HCP cluster private-kas: failed waiting for cluster="private-kas" in resourcegroup="private-kas-q8r96dg25gkl" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4018f813-07d6-479d-8c54-378c34ed46c3
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4018f813-07d6-479d-8c54-378c34ed46c3",
      "name": "4018f813-07d6-479d-8c54-378c34ed46c3",
      "status": "Failed",
      "startTime": "2026-08-04T15:09:35.150840021Z",
      "endTime": "2026-08-04T15:28:52.741249494Z",
      "error": {
        "code": "InternalServerError",
        "message": "[hypershiftHostedCluster] hosted cluster has no installed version; hosted cluster degraded: UnavailableReplicas: router deployment has 1 unavailable replicas"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:90]: failed to create HCP cluster "private-kas" with private KAS
Unexpected error:
    <*fmt.wrapError | 0xc0014d11c0>: 
    failed to create HCP cluster private-kas: failed waiting for cluster="private-kas" in resourcegroup="private-kas-q8r96dg25gkl" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4018f813-07d6-479d-8c54-378c34ed46c3
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4018f813-07d6-479d-8c54-378c34ed46c3",
      "name": "4018f813-07d6-479d-8c54-378c34ed46c3",
      "status": "Failed",
      "startTime": "2026-08-04T15:09:35.150840021Z",
      "endTime": "2026-08-04T15:28:52.741249494Z",
      "error": {
        "code": "InternalServerError",
        "message": "[hypershiftHostedCluster] hosted cluster has no installed version; hosted cluster degraded: UnavailableReplicas: router deployment has 1 unavailable replicas"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
branch-ci-Azure-ARO-HCP-main-e2e-stage-e2e-parallel
post-goodmerged PR
No additional run metadata captured
 unknown failed n/a 1 non-artifact-backed failure (no junit artifacts)
branch-ci-Azure-ARO-HCP-main-e2e-stage-e2e-parallel
post-goodmerged PR
No additional run metadata captured
 e2e failed n/a 1
e2e (1)
ERROR CODE: InternalServerError; detail message [clusterServiceNodePoolStatus] <no_message>; provider Microsoft.RedHatOpenShift
04:38:13 UTC · failed at e2e · test Customer should be able to create a no-CNI private cluster with a private key vault, a nodepool and install cilium CNI successfully · suite stage/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:191]: failed to verify nodes are Ready with Cilium CNI for cluster "cilium-cluster"
Unexpected error:
    <*errors.joinError | 0xc001f2e048>: 
    VerifyNodesReady failed: no nodes found in the cluster
    VerifyCiliumOperational failed: not all pods in kube-system namespace are running: failed to list pods in kube-system namespace: client rate limiter Wait returned an error: context deadline exceeded
    failed to create NodePool cilium-np: failed waiting for nodepool="cilium-np" for cluster "cilium-cluster" in resourcegroup="complex-cilium-kv-tqm8zjwmbxvx" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/e1b17109-deff-4598-bd83-3ef741e315b0
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/e1b17109-deff-4598-bd83-3ef741e315b0",
      "name": "e1b17109-deff-4598-bd83-3ef741e315b0",
      "status": "Failed",
      "startTime": "2026-08-04T05:09:46.184125491Z",
      "endTime": "2026-08-04T05:29:03.102580357Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceNodePoolStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:191]: failed to verify nodes are Ready with Cilium CNI for cluster "cilium-cluster"
Unexpected error:
    <*errors.joinError | 0xc001f2e048>: 
    VerifyNodesReady failed: no nodes found in the cluster
    VerifyCiliumOperational failed: not all pods in kube-system namespace are running: failed to list pods in kube-system namespace: client rate limiter Wait returned an error: context deadline exceeded
    failed to create NodePool cilium-np: failed waiting for nodepool="cilium-np" for cluster "cilium-cluster" in resourcegroup="complex-cilium-kv-tqm8zjwmbxvx" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/e1b17109-deff-4598-bd83-3ef741e315b0
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/e1b17109-deff-4598-bd83-3ef741e315b0",
      "name": "e1b17109-deff-4598-bd83-3ef741e315b0",
      "status": "Failed",
      "startTime": "2026-08-04T05:09:46.184125491Z",
      "endTime": "2026-08-04T05:29:03.102580357Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceNodePoolStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
periodic-ci-Azure-ARO-HCP-main-periodic-stage-e2e-parallel
post-goodmerged PR
No additional run metadata captured
 e2e failed n/a 44
e2e (44)
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Customer should be able to test admin credentials before cluster ready, then full admin credential lifecycle · suite stage/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:105]: failed to begin creating HCP cluster "admin-cred-lifecycle-p9c7vb"
Unexpected error:
    <*fmt.wrapError | 0xc00031a320>: 
    failed starting cluster creation "admin-cred-lifecycle-p9c7vb" in resourcegroup="admin-credential-lifecycle-test-ddddqw5bclcx": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/admin-credential-lifecycle-test-ddddqw5bclcx/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/admin-cred-lifecycle-p9c7vb
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:105]: failed to begin creating HCP cluster "admin-cred-lifecycle-p9c7vb"
Unexpected error:
    <*fmt.wrapError | 0xc00031a320>: 
    failed starting cluster creation "admin-cred-lifecycle-p9c7vb" in resourcegroup="admin-credential-lifecycle-test-ddddqw5bclcx": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/admin-credential-lifecycle-test-ddddqw5bclcx/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/admin-cred-lifecycle-p9c7vb
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.21 · suite stage/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:121]: HCP cluster rg-candidate-4-21-mjzw426l4nbl/cluster-candidate-4-21-vrmnw7 should provision
Unexpected error:
    <*fmt.wrapError | 0xc000893d80>: 
    failed to create HCP cluster cluster-candidate-4-21-vrmnw7: failed starting cluster creation "cluster-candidate-4-21-vrmnw7" in resourcegroup="rg-candidate-4-21-mjzw426l4nbl": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-4-21-mjzw426l4nbl/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-candidate-4-21-vrmnw7
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:121]: HCP cluster rg-candidate-4-21-mjzw426l4nbl/cluster-candidate-4-21-vrmnw7 should provision
Unexpected error:
    <*fmt.wrapError | 0xc000893d80>: 
    failed to create HCP cluster cluster-candidate-4-21-vrmnw7: failed starting cluster creation "cluster-candidate-4-21-vrmnw7" in resourcegroup="rg-candidate-4-21-mjzw426l4nbl": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-4-21-mjzw426l4nbl/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-candidate-4-21-vrmnw7
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Customer should be able to create an HCP cluster then delete it by deleting the customer resource group · suite stage/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delete_cx_rg.go:90]: failed to create HCP cluster "cx-rg-hcp-cluster"
Unexpected error:
    <*fmt.wrapError | 0xc000f90f00>: 
    failed to create HCP cluster cx-rg-hcp-cluster: failed starting cluster creation "cx-rg-hcp-cluster" in resourcegroup="cx-rg-cluster-nkd2m4c42psr": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/cx-rg-cluster-nkd2m4c42psr/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cx-rg-hcp-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delete_cx_rg.go:90]: failed to create HCP cluster "cx-rg-hcp-cluster"
Unexpected error:
    <*fmt.wrapError | 0xc000f90f00>: 
    failed to create HCP cluster cx-rg-hcp-cluster: failed starting cluster creation "cx-rg-hcp-cluster" in resourcegroup="cx-rg-cluster-nkd2m4c42psr": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/cx-rg-cluster-nkd2m4c42psr/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cx-rg-hcp-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 4.19 · suite stage/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:110]: failed to create HCP cluster "cluster-zstream-4-19-b66pj7" with version 4.19.39 on candidate channel
Unexpected error:
    <*fmt.wrapError | 0xc000b20e00>: 
    failed to create HCP cluster cluster-zstream-4-19-b66pj7: failed starting cluster creation "cluster-zstream-4-19-b66pj7" in resourcegroup="rg-zstream-upgrade-4-19-b52k2nwsdx98": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-19-b52k2nwsdx98/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-zstream-4-19-b66pj7
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:110]: failed to create HCP cluster "cluster-zstream-4-19-b66pj7" with version 4.19.39 on candidate channel
Unexpected error:
    <*fmt.wrapError | 0xc000b20e00>: 
    failed to create HCP cluster cluster-zstream-4-19-b66pj7: failed starting cluster creation "cluster-zstream-4-19-b66pj7" in resourcegroup="rg-zstream-upgrade-4-19-b52k2nwsdx98": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-19-b52k2nwsdx98/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-zstream-4-19-b66pj7
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Customer should create an HCP cluster and validate TLS certificates · suite stage/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:95]: failed to create HCP cluster for TLS endpoint test
Unexpected error:
    <*fmt.wrapError | 0xc000ddc120>: 
    failed to create HCP cluster tls-endpoint-hcp-cluster: failed starting cluster creation "tls-endpoint-hcp-cluster" in resourcegroup="tls-endpoint-cluster-bwjns78fkdlj": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/tls-endpoint-cluster-bwjns78fkdlj/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/tls-endpoint-hcp-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:95]: failed to create HCP cluster for TLS endpoint test
Unexpected error:
    <*fmt.wrapError | 0xc000ddc120>: 
    failed to create HCP cluster tls-endpoint-hcp-cluster: failed starting cluster creation "tls-endpoint-hcp-cluster" in resourcegroup="tls-endpoint-cluster-bwjns78fkdlj": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/tls-endpoint-cluster-bwjns78fkdlj/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/tls-endpoint-hcp-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Customer should be able to create a HCP cluster and use cilium CNI plugin · suite stage/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_cni_cilium.go:88]: failed to create HCP cluster "cilium-cl" without CNI
Unexpected error:
    <*fmt.wrapError | 0xc000c1c420>: 
    failed to create HCP cluster cilium-cl: failed starting cluster creation "cilium-cl" in resourcegroup="cni-cilium-nhlp7mf7bhrb": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/cni-cilium-nhlp7mf7bhrb/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cilium-cl
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_cni_cilium.go:88]: failed to create HCP cluster "cilium-cl" without CNI
Unexpected error:
    <*fmt.wrapError | 0xc000c1c420>: 
    failed to create HCP cluster cilium-cl: failed starting cluster creation "cilium-cl" in resourcegroup="cni-cilium-nhlp7mf7bhrb": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/cni-cilium-nhlp7mf7bhrb/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cilium-cl
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Customer should be able to create an HCP cluster and manage ImageDigestMirrors · suite stage/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/idms_lifecycle.go:113]: failed to create HCP cluster with ImageDigestMirrors
Unexpected error:
    <*fmt.wrapError | 0xc0005d98c0>: 
    failed to create HCP cluster idms-e2e-hcp-cluster: failed starting cluster creation "idms-e2e-hcp-cluster" in resourcegroup="idms-2tqxkbtvk5f7": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/idms-2tqxkbtvk5f7/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/idms-e2e-hcp-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/idms_lifecycle.go:113]: failed to create HCP cluster with ImageDigestMirrors
Unexpected error:
    <*fmt.wrapError | 0xc0005d98c0>: 
    failed to create HCP cluster idms-e2e-hcp-cluster: failed starting cluster creation "idms-e2e-hcp-cluster" in resourcegroup="idms-2tqxkbtvk5f7": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/idms-2tqxkbtvk5f7/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/idms-e2e-hcp-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.22 · suite stage/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:121]: HCP cluster rg-candidate-4-22-crjtmmr9jxwr/cluster-candidate-4-22-lrwxb2 should provision
Unexpected error:
    <*fmt.wrapError | 0xc000854320>: 
    failed to create HCP cluster cluster-candidate-4-22-lrwxb2: failed starting cluster creation "cluster-candidate-4-22-lrwxb2" in resourcegroup="rg-candidate-4-22-crjtmmr9jxwr": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-4-22-crjtmmr9jxwr/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-candidate-4-22-lrwxb2
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:121]: HCP cluster rg-candidate-4-22-crjtmmr9jxwr/cluster-candidate-4-22-lrwxb2 should provision
Unexpected error:
    <*fmt.wrapError | 0xc000854320>: 
    failed to create HCP cluster cluster-candidate-4-22-lrwxb2: failed starting cluster creation "cluster-candidate-4-22-lrwxb2" in resourcegroup="rg-candidate-4-22-crjtmmr9jxwr": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-4-22-crjtmmr9jxwr/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-candidate-4-22-lrwxb2
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Customer should be able to create an HCP cluster and manage pull secrets · suite stage/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_pullsecret.go:113]: failed to create HCP cluster for pull secret test
Unexpected error:
    <*fmt.wrapError | 0xc000fa1260>: 
    failed to create HCP cluster pullsecret-hcp-cluster: failed starting cluster creation "pullsecret-hcp-cluster" in resourcegroup="pullsecret-test-q9v98zb2fts7": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/pullsecret-test-q9v98zb2fts7/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/pullsecret-hcp-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_pullsecret.go:113]: failed to create HCP cluster for pull secret test
Unexpected error:
    <*fmt.wrapError | 0xc000fa1260>: 
    failed to create HCP cluster pullsecret-hcp-cluster: failed starting cluster creation "pullsecret-hcp-cluster" in resourcegroup="pullsecret-test-q9v98zb2fts7": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/pullsecret-test-q9v98zb2fts7/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/pullsecret-hcp-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Authorized CIDRs Connectivity should allow API access only from authorized VM IP · suite stage/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:111]: failed to create HCP cluster "cidr-connectivity-test" with authorized CIDRs
Unexpected error:
    <*fmt.wrapError | 0xc00085bf00>: 
    failed to create HCP cluster cidr-connectivity-test: failed starting cluster creation "cidr-connectivity-test" in resourcegroup="e2e-cidr-connectivity-gl4zhq2lcglb": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-gl4zhq2lcglb/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cidr-connectivity-test
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:111]: failed to create HCP cluster "cidr-connectivity-test" with authorized CIDRs
Unexpected error:
    <*fmt.wrapError | 0xc00085bf00>: 
    failed to create HCP cluster cidr-connectivity-test: failed starting cluster creation "cidr-connectivity-test" in resourcegroup="e2e-cidr-connectivity-gl4zhq2lcglb": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-gl4zhq2lcglb/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cidr-connectivity-test
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Customer should respect cluster-wide node limits with nodepool autoscaling · suite stage/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:291]: failed to create HCP cluster node-limit-cluster with MaxNodesTotal limit
Unexpected error:
    <*fmt.wrapError | 0xc000de5de0>: 
    failed to create HCP cluster node-limit-cluster: failed starting cluster creation "node-limit-cluster" in resourcegroup="node-limit-test-tpxjdfr7m75k": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/node-limit-test-tpxjdfr7m75k/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/node-limit-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:291]: failed to create HCP cluster node-limit-cluster with MaxNodesTotal limit
Unexpected error:
    <*fmt.wrapError | 0xc000de5de0>: 
    failed to create HCP cluster node-limit-cluster: failed starting cluster creation "node-limit-cluster" in resourcegroup="node-limit-test-tpxjdfr7m75k": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/node-limit-test-tpxjdfr7m75k/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/node-limit-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Customer should be able to create a cluster with an external auth config and get the external auth config · suite stage/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:94]: failed to create HCP cluster for external auth test
Unexpected error:
    <*fmt.wrapError | 0xc000b19d40>: 
    failed to create HCP cluster ea-cluster: failed starting cluster creation "ea-cluster" in resourcegroup="external-auth-cluster-x6r4vqqc8t2r": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/external-auth-cluster-x6r4vqqc8t2r/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/ea-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:94]: failed to create HCP cluster for external auth test
Unexpected error:
    <*fmt.wrapError | 0xc000b19d40>: 
    failed to create HCP cluster ea-cluster: failed starting cluster creation "ea-cluster" in resourcegroup="external-auth-cluster-x6r4vqqc8t2r": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/external-auth-cluster-x6r4vqqc8t2r/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/ea-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Nodepool Ephemeral OS Disk should create a nodepool with ephemeral OS disk when autoRepair is enabled · suite stage/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_ephemeral_osdisk.go:112]: failed to create HCP cluster ephemeral-disk
Unexpected error:
    <*fmt.wrapError | 0xc000f68820>: 
    failed to create HCP cluster ephemeral-disk: failed starting cluster creation "ephemeral-disk" in resourcegroup="ephemeral-osdisk-sb9gzrb6g2jh": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/ephemeral-osdisk-sb9gzrb6g2jh/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/ephemeral-disk
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_ephemeral_osdisk.go:112]: failed to create HCP cluster ephemeral-disk
Unexpected error:
    <*fmt.wrapError | 0xc000f68820>: 
    failed to create HCP cluster ephemeral-disk: failed starting cluster creation "ephemeral-disk" in resourcegroup="ephemeral-osdisk-sb9gzrb6g2jh": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/ephemeral-osdisk-sb9gzrb6g2jh/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/ephemeral-disk
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Customer should be able to use workload identity via the cluster OIDC issuer URL · suite stage/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/oidc_issuer_workload_identity.go:217]: failed to create HCP cluster oidc-wi-cluster
Unexpected error:
    <*fmt.wrapError | 0xc000709c40>: 
    failed to create HCP cluster oidc-wi-cluster: failed starting cluster creation "oidc-wi-cluster" in resourcegroup="oidc-wi-wlpbx58rn797": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/oidc-wi-wlpbx58rn797/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/oidc-wi-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/oidc_issuer_workload_identity.go:217]: failed to create HCP cluster oidc-wi-cluster
Unexpected error:
    <*fmt.wrapError | 0xc000709c40>: 
    failed to create HCP cluster oidc-wi-cluster: failed starting cluster creation "oidc-wi-cluster" in resourcegroup="oidc-wi-wlpbx58rn797": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/oidc-wi-wlpbx58rn797/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/oidc-wi-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Customer should upgrade and update a nodepool from 4.21.z to 4.21.zLatest · suite stage/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:133]: failed to create HCP cluster np-version-upgrade-cluster-k4shgp with version 4.21.23
Unexpected error:
    <*fmt.wrapError | 0xc000fa96c0>: 
    failed to create HCP cluster np-version-upgrade-cluster-k4shgp: failed starting cluster creation "np-version-upgrade-cluster-k4shgp" in resourcegroup="rg-np-version-upgrade-k4shgp-hx6vrnwrvs6f": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-version-upgrade-k4shgp-hx6vrnwrvs6f/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-version-upgrade-cluster-k4shgp
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:133]: failed to create HCP cluster np-version-upgrade-cluster-k4shgp with version 4.21.23
Unexpected error:
    <*fmt.wrapError | 0xc000fa96c0>: 
    failed to create HCP cluster np-version-upgrade-cluster-k4shgp: failed starting cluster creation "np-version-upgrade-cluster-k4shgp" in resourcegroup="rg-np-version-upgrade-k4shgp-hx6vrnwrvs6f": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-version-upgrade-k4shgp-hx6vrnwrvs6f/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-version-upgrade-cluster-k4shgp
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Customer should downgrade a nodepool version z-stream downgrade from 4.21.zLatest to 4.21.zPrevious · suite stage/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:593]: failed to create HCP cluster
Unexpected error:
    <*fmt.wrapError | 0xc000f90ea0>: 
    failed to create HCP cluster np-downgrade-xbf47m: failed starting cluster creation "np-downgrade-xbf47m" in resourcegroup="rg-np-downgrade-xbf47m-z528lwt2d5g6": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-downgrade-xbf47m-z528lwt2d5g6/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-downgrade-xbf47m
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:593]: failed to create HCP cluster
Unexpected error:
    <*fmt.wrapError | 0xc000f90ea0>: 
    failed to create HCP cluster np-downgrade-xbf47m: failed starting cluster creation "np-downgrade-xbf47m" in resourcegroup="rg-np-downgrade-xbf47m-z528lwt2d5g6": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-downgrade-xbf47m-z528lwt2d5g6/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-downgrade-xbf47m
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Customer should be able to create a cluster with default autoscaling and a nodepool with autoscaling enabled up to replica limits · suite stage/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:102]: failed to create HCP cluster np-autoscale-cluster
Unexpected error:
    <*fmt.wrapError | 0xc000900840>: 
    failed to create HCP cluster np-autoscale-cluster: failed starting cluster creation "np-autoscale-cluster" in resourcegroup="np-autoscaling-qjllkwz7mxww": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/np-autoscaling-qjllkwz7mxww/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-autoscale-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:102]: failed to create HCP cluster np-autoscale-cluster
Unexpected error:
    <*fmt.wrapError | 0xc000900840>: 
    failed to create HCP cluster np-autoscale-cluster: failed starting cluster creation "np-autoscale-cluster" in resourcegroup="np-autoscaling-qjllkwz7mxww": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/np-autoscaling-qjllkwz7mxww/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-autoscale-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 4.20 · suite stage/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:110]: failed to create HCP cluster "cluster-zstream-4-20-65llv5" with version 4.20.27 on candidate channel
Unexpected error:
    <*fmt.wrapError | 0xc000f8a7c0>: 
    failed to create HCP cluster cluster-zstream-4-20-65llv5: failed starting cluster creation "cluster-zstream-4-20-65llv5" in resourcegroup="rg-zstream-upgrade-4-20-5mt449btl2bs": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-20-5mt449btl2bs/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-zstream-4-20-65llv5
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:110]: failed to create HCP cluster "cluster-zstream-4-20-65llv5" with version 4.20.27 on candidate channel
Unexpected error:
    <*fmt.wrapError | 0xc000f8a7c0>: 
    failed to create HCP cluster cluster-zstream-4-20-65llv5: failed starting cluster creation "cluster-zstream-4-20-65llv5" in resourcegroup="rg-zstream-upgrade-4-20-5mt449btl2bs": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-20-5mt449btl2bs/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-zstream-4-20-65llv5
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test ARO HCP Service should successfully provision cluster when MI role assignments are added after cluster creation · suite stage/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:133]: failed to start cluster creation
Unexpected error:
    <*exported.ResponseError | 0xc00137c300>: 
    PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/delayed-rbac-xgkk8l4xk9hz/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/delayed-rbac-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:133]: failed to start cluster creation
Unexpected error:
    <*exported.ResponseError | 0xc00137c300>: 
    PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/delayed-rbac-xgkk8l4xk9hz/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/delayed-rbac-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Customer should upgrade a nodepool to a version without Cincinnati upgrade edge z-stream upgrade without Cincinnati edge in 4.20 · suite stage/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:314]: failed to create HCP cluster np-noedge-ks85cx
Unexpected error:
    <*fmt.wrapError | 0xc000f34180>: 
    failed to create HCP cluster np-noedge-ks85cx: failed starting cluster creation "np-noedge-ks85cx" in resourcegroup="rg-np-noedge-ks85cx-n9n6xvgv62lz": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-noedge-ks85cx-n9n6xvgv62lz/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-noedge-ks85cx
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:314]: failed to create HCP cluster np-noedge-ks85cx
Unexpected error:
    <*fmt.wrapError | 0xc000f34180>: 
    failed to create HCP cluster np-noedge-ks85cx: failed starting cluster creation "np-noedge-ks85cx" in resourcegroup="rg-np-noedge-ks85cx-n9n6xvgv62lz": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-noedge-ks85cx-n9n6xvgv62lz/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-noedge-ks85cx
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Customer should be able to create node pools with ARM64-based VMs · suite stage/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/arm64_nodepool.go:88]: failed to create HCP cluster "arm64-vm-hcp-cluster"
Unexpected error:
    <*fmt.wrapError | 0xc00021efc0>: 
    failed to create HCP cluster arm64-vm-hcp-cluster: failed starting cluster creation "arm64-vm-hcp-cluster" in resourcegroup="arm64-vm-cluster-jcr44k6brrm2": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/arm64-vm-cluster-jcr44k6brrm2/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/arm64-vm-hcp-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/arm64_nodepool.go:88]: failed to create HCP cluster "arm64-vm-hcp-cluster"
Unexpected error:
    <*fmt.wrapError | 0xc00021efc0>: 
    failed to create HCP cluster arm64-vm-hcp-cluster: failed starting cluster creation "arm64-vm-hcp-cluster" in resourcegroup="arm64-vm-cluster-jcr44k6brrm2": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/arm64-vm-cluster-jcr44k6brrm2/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/arm64-vm-hcp-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Customer should be able to create an HCP cluster with custom autoscaling and update the autoscaling configuration · suite stage/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:108]: failed to create HCP cluster "autoscaling-hcp-cluster" with custom autoscaling
Unexpected error:
    <*fmt.wrapError | 0xc00092e5c0>: 
    failed to create HCP cluster autoscaling-hcp-cluster: failed starting cluster creation "autoscaling-hcp-cluster" in resourcegroup="autoscaling-cluster-jvm2427qfmg2": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/autoscaling-cluster-jvm2427qfmg2/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/autoscaling-hcp-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:108]: failed to create HCP cluster "autoscaling-hcp-cluster" with custom autoscaling
Unexpected error:
    <*fmt.wrapError | 0xc00092e5c0>: 
    failed to create HCP cluster autoscaling-hcp-cluster: failed starting cluster creation "autoscaling-hcp-cluster" in resourcegroup="autoscaling-cluster-jvm2427qfmg2": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/autoscaling-cluster-jvm2427qfmg2/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/autoscaling-hcp-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.20 · suite stage/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:121]: HCP cluster rg-candidate-4-20-zf9fvwb6v85m/cluster-candidate-4-20-w2rlrv should provision
Unexpected error:
    <*fmt.wrapError | 0xc000e18400>: 
    failed to create HCP cluster cluster-candidate-4-20-w2rlrv: failed starting cluster creation "cluster-candidate-4-20-w2rlrv" in resourcegroup="rg-candidate-4-20-zf9fvwb6v85m": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-4-20-zf9fvwb6v85m/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-candidate-4-20-w2rlrv
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:121]: HCP cluster rg-candidate-4-20-zf9fvwb6v85m/cluster-candidate-4-20-w2rlrv should provision
Unexpected error:
    <*fmt.wrapError | 0xc000e18400>: 
    failed to create HCP cluster cluster-candidate-4-20-w2rlrv: failed starting cluster creation "cluster-candidate-4-20-w2rlrv" in resourcegroup="rg-candidate-4-20-zf9fvwb6v85m": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-4-20-zf9fvwb6v85m/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-candidate-4-20-w2rlrv
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Customer should create a cluster with private KAS and verify API server is only reachable from VNet while ingress remains public · suite stage/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:90]: failed to create HCP cluster "private-kas" with private KAS
Unexpected error:
    <*fmt.wrapError | 0xc000a1a400>: 
    failed to create HCP cluster private-kas: failed starting cluster creation "private-kas" in resourcegroup="private-kas-xp2hh24rsddd": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/private-kas-xp2hh24rsddd/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/private-kas
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:90]: failed to create HCP cluster "private-kas" with private KAS
Unexpected error:
    <*fmt.wrapError | 0xc000a1a400>: 
    failed to create HCP cluster private-kas: failed starting cluster creation "private-kas" in resourcegroup="private-kas-xp2hh24rsddd": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/private-kas-xp2hh24rsddd/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/private-kas
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 4.21 · suite stage/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:110]: failed to create HCP cluster "cluster-zstream-4-21-9g98w5" with version 4.21.22 on candidate channel
Unexpected error:
    <*fmt.wrapError | 0xc000ea5c60>: 
    failed to create HCP cluster cluster-zstream-4-21-9g98w5: failed starting cluster creation "cluster-zstream-4-21-9g98w5" in resourcegroup="rg-zstream-upgrade-4-21-jhq5xrsc2c5j": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-21-jhq5xrsc2c5j/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-zstream-4-21-9g98w5
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:110]: failed to create HCP cluster "cluster-zstream-4-21-9g98w5" with version 4.21.22 on candidate channel
Unexpected error:
    <*fmt.wrapError | 0xc000ea5c60>: 
    failed to create HCP cluster cluster-zstream-4-21-9g98w5: failed starting cluster creation "cluster-zstream-4-21-9g98w5" in resourcegroup="rg-zstream-upgrade-4-21-jhq5xrsc2c5j": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-21-jhq5xrsc2c5j/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-zstream-4-21-9g98w5
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Customer should be able to lifecycle and confirm external auth on a cluster · suite stage/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_list_and_verify.go:87]: failed to create HCP cluster for external auth list test
Unexpected error:
    <*fmt.wrapError | 0xc000c31f00>: 
    failed to create HCP cluster ea-list6xxgw5: failed starting cluster creation "ea-list6xxgw5" in resourcegroup="ea-list-28lkc8rtgw97": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/ea-list-28lkc8rtgw97/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/ea-list6xxgw5
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_list_and_verify.go:87]: failed to create HCP cluster for external auth list test
Unexpected error:
    <*fmt.wrapError | 0xc000c31f00>: 
    failed to create HCP cluster ea-list6xxgw5: failed starting cluster creation "ea-list6xxgw5" in resourcegroup="ea-list-28lkc8rtgw97": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/ea-list-28lkc8rtgw97/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/ea-list6xxgw5
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Customer should be able to create several HCP clusters in their customer resource group, but not in the same managed resource group · suite stage/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:111]: failed to begin creation of first cluster "basic-hcp-cluster"
Unexpected error:
    <*fmt.wrapError | 0xc000d90120>: 
    failed starting cluster creation "basic-hcp-cluster" in resourcegroup="customer-rg-bfgffn4qdtql": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/customer-rg-bfgffn4qdtql/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/basic-hcp-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:111]: failed to begin creation of first cluster "basic-hcp-cluster"
Unexpected error:
    <*fmt.wrapError | 0xc000d90120>: 
    failed starting cluster creation "basic-hcp-cluster" in resourcegroup="customer-rg-bfgffn4qdtql": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/customer-rg-bfgffn4qdtql/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/basic-hcp-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Customer should upgrade and update a nodepool from 4.20.z to 4.21.zLatest · suite stage/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:133]: failed to create HCP cluster np-version-upgrade-cluster-ljbjtt with version 4.21.23
Unexpected error:
    <*fmt.wrapError | 0xc000f5eb00>: 
    failed to create HCP cluster np-version-upgrade-cluster-ljbjtt: failed starting cluster creation "np-version-upgrade-cluster-ljbjtt" in resourcegroup="rg-np-version-upgrade-ljbjtt-nvkwjn8w7dzp": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-version-upgrade-ljbjtt-nvkwjn8w7dzp/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-version-upgrade-cluster-ljbjtt
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:133]: failed to create HCP cluster np-version-upgrade-cluster-ljbjtt with version 4.21.23
Unexpected error:
    <*fmt.wrapError | 0xc000f5eb00>: 
    failed to create HCP cluster np-version-upgrade-cluster-ljbjtt: failed starting cluster creation "np-version-upgrade-cluster-ljbjtt" in resourcegroup="rg-np-version-upgrade-ljbjtt-nvkwjn8w7dzp": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-version-upgrade-ljbjtt-nvkwjn8w7dzp/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-version-upgrade-cluster-ljbjtt
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Customer should be able to update node pool labels and taints · suite stage/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_labels_taints.go:92]: failed to create HCP cluster cluster-np-labels-taints
Unexpected error:
    <*fmt.wrapError | 0xc000e001a0>: 
    failed to create HCP cluster cluster-np-labels-taints: failed starting cluster creation "cluster-np-labels-taints" in resourcegroup="rg-np-labels-taints-wvdcmscq9q62": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-labels-taints-wvdcmscq9q62/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-np-labels-taints
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_labels_taints.go:92]: failed to create HCP cluster cluster-np-labels-taints
Unexpected error:
    <*fmt.wrapError | 0xc000e001a0>: 
    failed to create HCP cluster cluster-np-labels-taints: failed starting cluster creation "cluster-np-labels-taints" in resourcegroup="rg-np-labels-taints-wvdcmscq9q62": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-labels-taints-wvdcmscq9q62/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-np-labels-taints
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Create HCPOpenShiftCluster with Private KeyVault should create a cluster with private keyvault using v20251223preview API · suite stage/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:98]: failed to create HCP cluster "private-kv-cluster" with private keyvault
Unexpected error:
    <*fmt.wrapError | 0xc0009012c0>: 
    failed starting cluster creation "private-kv-cluster" in resourcegroup="private-keyvault-hpsbls8jzx2c": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/private-keyvault-hpsbls8jzx2c/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/private-kv-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:98]: failed to create HCP cluster "private-kv-cluster" with private keyvault
Unexpected error:
    <*fmt.wrapError | 0xc0009012c0>: 
    failed starting cluster creation "private-kv-cluster" in resourcegroup="private-keyvault-hpsbls8jzx2c": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/private-keyvault-hpsbls8jzx2c/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/private-kv-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 4.22 · suite stage/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:110]: failed to create HCP cluster "cluster-zstream-4-22-t86c52" with version 4.22.6 on candidate channel
Unexpected error:
    <*fmt.wrapError | 0xc000e87d60>: 
    failed to create HCP cluster cluster-zstream-4-22-t86c52: failed starting cluster creation "cluster-zstream-4-22-t86c52" in resourcegroup="rg-zstream-upgrade-4-22-j5drc7zvzv4c": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-22-j5drc7zvzv4c/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-zstream-4-22-t86c52
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:110]: failed to create HCP cluster "cluster-zstream-4-22-t86c52" with version 4.22.6 on candidate channel
Unexpected error:
    <*fmt.wrapError | 0xc000e87d60>: 
    failed to create HCP cluster cluster-zstream-4-22-t86c52: failed starting cluster creation "cluster-zstream-4-22-t86c52" in resourcegroup="rg-zstream-upgrade-4-22-j5drc7zvzv4c": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-22-j5drc7zvzv4c/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-zstream-4-22-t86c52
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Customer should downgrade a nodepool to a lower minor version from 4.21.zLatest to 4.19.zLatest · suite stage/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:741]: failed to create HCP cluster
Unexpected error:
    <*fmt.wrapError | 0xc000543980>: 
    failed to create HCP cluster np-dg-minor-vtnjvx: failed starting cluster creation "np-dg-minor-vtnjvx" in resourcegroup="rg-np-dg-minor-vtnjvx-dmk84bgmvg7f": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-dg-minor-vtnjvx-dmk84bgmvg7f/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-dg-minor-vtnjvx
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:741]: failed to create HCP cluster
Unexpected error:
    <*fmt.wrapError | 0xc000543980>: 
    failed to create HCP cluster np-dg-minor-vtnjvx: failed starting cluster creation "np-dg-minor-vtnjvx" in resourcegroup="rg-np-dg-minor-vtnjvx-dmk84bgmvg7f": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-dg-minor-vtnjvx-dmk84bgmvg7f/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-dg-minor-vtnjvx
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Customer should upgrade and update a nodepool from 4.20.z to 4.20.zLatest · suite stage/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:133]: failed to create HCP cluster np-version-upgrade-cluster-6qmf9t with version 4.20.32
Unexpected error:
    <*fmt.wrapError | 0xc0008812c0>: 
    failed to create HCP cluster np-version-upgrade-cluster-6qmf9t: failed starting cluster creation "np-version-upgrade-cluster-6qmf9t" in resourcegroup="rg-np-version-upgrade-6qmf9t-8jnxq85dds2t": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-version-upgrade-6qmf9t-8jnxq85dds2t/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-version-upgrade-cluster-6qmf9t
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:133]: failed to create HCP cluster np-version-upgrade-cluster-6qmf9t with version 4.20.32
Unexpected error:
    <*fmt.wrapError | 0xc0008812c0>: 
    failed to create HCP cluster np-version-upgrade-cluster-6qmf9t: failed starting cluster creation "np-version-upgrade-cluster-6qmf9t" in resourcegroup="rg-np-version-upgrade-6qmf9t-8jnxq85dds2t": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-version-upgrade-6qmf9t-8jnxq85dds2t/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-version-upgrade-cluster-6qmf9t
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Customer should be able to create an HCP cluster with Image Registry not present · suite stage/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/image_registry_cluster_create.go:89]: failed to create HCP cluster with image registry disabled
Unexpected error:
    <*fmt.wrapError | 0xc000f04540>: 
    failed to create HCP cluster disabled-image-registry-hcp-cluster: failed starting cluster creation "disabled-image-registry-hcp-cluster" in resourcegroup="disabled-image-registry-tp7hjqg2smz7": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/disabled-image-registry-tp7hjqg2smz7/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/disabled-image-registry-hcp-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/image_registry_cluster_create.go:89]: failed to create HCP cluster with image registry disabled
Unexpected error:
    <*fmt.wrapError | 0xc000f04540>: 
    failed to create HCP cluster disabled-image-registry-hcp-cluster: failed starting cluster creation "disabled-image-registry-hcp-cluster" in resourcegroup="disabled-image-registry-tp7hjqg2smz7": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/disabled-image-registry-tp7hjqg2smz7/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/disabled-image-registry-hcp-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Customer should upgrade a nodepool skipping one minor version (+2) from 4.20.z to 4.22.zLatest · suite stage/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:462]: failed to create HCP cluster np-skip-minor-cjnsh5 at version 4.22.8
Unexpected error:
    <*fmt.wrapError | 0xc000f0d340>: 
    failed to create HCP cluster np-skip-minor-cjnsh5: failed starting cluster creation "np-skip-minor-cjnsh5" in resourcegroup="rg-np-skip-minor-cjnsh5-4s22r9rmtsgn": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-skip-minor-cjnsh5-4s22r9rmtsgn/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-skip-minor-cjnsh5
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:462]: failed to create HCP cluster np-skip-minor-cjnsh5 at version 4.22.8
Unexpected error:
    <*fmt.wrapError | 0xc000f0d340>: 
    failed to create HCP cluster np-skip-minor-cjnsh5: failed starting cluster creation "np-skip-minor-cjnsh5" in resourcegroup="rg-np-skip-minor-cjnsh5-4s22r9rmtsgn": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-skip-minor-cjnsh5-4s22r9rmtsgn/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-skip-minor-cjnsh5
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Customer should not be able to reuse subnets and NSGs between clusters · suite stage/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_nsg_subnet_reuse.go:107]: failed to create seed HCP cluster basic-cluster
Unexpected error:
    <*fmt.wrapError | 0xc00041d7c0>: 
    failed to create HCP cluster basic-cluster: failed starting cluster creation "basic-cluster" in resourcegroup="rg-cluster-nsg-subnet-reuse-zdljvll8r8zl": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cluster-nsg-subnet-reuse-zdljvll8r8zl/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/basic-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_nsg_subnet_reuse.go:107]: failed to create seed HCP cluster basic-cluster
Unexpected error:
    <*fmt.wrapError | 0xc00041d7c0>: 
    failed to create HCP cluster basic-cluster: failed starting cluster creation "basic-cluster" in resourcegroup="rg-cluster-nsg-subnet-reuse-zdljvll8r8zl": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cluster-nsg-subnet-reuse-zdljvll8r8zl/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/basic-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Customer should be able to create a no-CNI private cluster with a private key vault, a nodepool and install cilium CNI successfully · suite stage/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:105]: failed to create HCP cluster "cilium-cluster" with no CNI and private etcd
Unexpected error:
    <*fmt.wrapError | 0xc000eb60c0>: 
    failed starting cluster creation "cilium-cluster" in resourcegroup="complex-cilium-kv-2v9tjtwzkc26": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/complex-cilium-kv-2v9tjtwzkc26/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cilium-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:105]: failed to create HCP cluster "cilium-cluster" with no CNI and private etcd
Unexpected error:
    <*fmt.wrapError | 0xc000eb60c0>: 
    failed starting cluster creation "cilium-cluster" in resourcegroup="complex-cilium-kv-2v9tjtwzkc26": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/complex-cilium-kv-2v9tjtwzkc26/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cilium-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Update HCPOpenShiftCluster Positive creates a cluster and updates tags with a PATCH request · suite stage/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:161]: failed to create HCP cluster for patch-tags test
Unexpected error:
    <*fmt.wrapError | 0xc000bca640>: 
    failed to create HCP cluster patch-tags-cluster: failed starting cluster creation "patch-tags-cluster" in resourcegroup="patch-tags-gfhw9l74bs76": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-tags-gfhw9l74bs76/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-tags-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:161]: failed to create HCP cluster for patch-tags test
Unexpected error:
    <*fmt.wrapError | 0xc000bca640>: 
    failed to create HCP cluster patch-tags-cluster: failed starting cluster creation "patch-tags-cluster" in resourcegroup="patch-tags-gfhw9l74bs76": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-tags-gfhw9l74bs76/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-tags-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Customer should not be able to perform various invalid operations on cluster resources · suite stage/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/simple_negative_cases.go:110]: failed to create HCP cluster negative-tests-cluster
Unexpected error:
    <*fmt.wrapError | 0xc0004d7e00>: 
    failed to create HCP cluster negative-tests-cluster: failed starting cluster creation "negative-tests-cluster" in resourcegroup="rg-negative-tests-bw9nrqpq4kgm": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-negative-tests-bw9nrqpq4kgm/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/negative-tests-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/simple_negative_cases.go:110]: failed to create HCP cluster negative-tests-cluster
Unexpected error:
    <*fmt.wrapError | 0xc0004d7e00>: 
    failed to create HCP cluster negative-tests-cluster: failed starting cluster creation "negative-tests-cluster" in resourcegroup="rg-negative-tests-bw9nrqpq4kgm": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-negative-tests-bw9nrqpq4kgm/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/negative-tests-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Update HCPOpenShiftCluster Negative creates a cluster and fails to update its name with a PATCH request · suite stage/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:84]: failed to create HCP cluster for patch-name test
Unexpected error:
    <*fmt.wrapError | 0xc000510000>: 
    failed to create HCP cluster patch-name-cluster: failed starting cluster creation "patch-name-cluster" in resourcegroup="patch-name-2l28rkhqw7tt": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-name-2l28rkhqw7tt/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-name-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:84]: failed to create HCP cluster for patch-name test
Unexpected error:
    <*fmt.wrapError | 0xc000510000>: 
    failed to create HCP cluster patch-name-cluster: failed starting cluster creation "patch-name-cluster" in resourcegroup="patch-name-2l28rkhqw7tt": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-name-2l28rkhqw7tt/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-name-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Customer should be able to successfully upgrade control plane minor version from 4.20 minor to 4.21 minor · suite stage/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:182]: failed to create HCP cluster "cp-ystream-upgrade-4-21-xjcbl2" with install version 4.20.28
Unexpected error:
    <*fmt.wrapError | 0xc000f8a3c0>: 
    failed to create HCP cluster cp-ystream-upgrade-4-21-xjcbl2: failed starting cluster creation "cp-ystream-upgrade-4-21-xjcbl2" in resourcegroup="rg-cp-ystream-upgrade-4-21-dj6kggx5gggc": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cp-ystream-upgrade-4-21-dj6kggx5gggc/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cp-ystream-upgrade-4-21-xjcbl2
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:182]: failed to create HCP cluster "cp-ystream-upgrade-4-21-xjcbl2" with install version 4.20.28
Unexpected error:
    <*fmt.wrapError | 0xc000f8a3c0>: 
    failed to create HCP cluster cp-ystream-upgrade-4-21-xjcbl2: failed starting cluster creation "cp-ystream-upgrade-4-21-xjcbl2" in resourcegroup="rg-cp-ystream-upgrade-4-21-dj6kggx5gggc": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cp-ystream-upgrade-4-21-dj6kggx5gggc/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cp-ystream-upgrade-4-21-xjcbl2
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Customer should be able to create an HCP cluster and custom node pool osDisk size · suite stage/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_nodepool_osdisk.go:80]: failed to create HCP cluster "hcp-cluster-np-128"
Unexpected error:
    <*fmt.wrapError | 0xc000ac1500>: 
    failed to create HCP cluster hcp-cluster-np-128: failed starting cluster creation "hcp-cluster-np-128" in resourcegroup="clusternp128-t8r5r94ls7k9": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/clusternp128-t8r5r94ls7k9/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/hcp-cluster-np-128
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_nodepool_osdisk.go:80]: failed to create HCP cluster "hcp-cluster-np-128"
Unexpected error:
    <*fmt.wrapError | 0xc000ac1500>: 
    failed to create HCP cluster hcp-cluster-np-128: failed starting cluster creation "hcp-cluster-np-128" in resourcegroup="clusternp128-t8r5r94ls7k9": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/clusternp128-t8r5r94ls7k9/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/hcp-cluster-np-128
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Customer should be able to create a cluster and node pools with aggregated advanced features · suite stage/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:184]: failed to create HCP cluster "agg-cluster" with aggregated settings
Unexpected error:
    <*fmt.wrapError | 0xc000a9ee80>: 
    failed starting cluster creation "agg-cluster" in resourcegroup="feature-aggregation-psqzvzzpdccv": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/feature-aggregation-psqzvzzpdccv/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/agg-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:184]: failed to create HCP cluster "agg-cluster" with aggregated settings
Unexpected error:
    <*fmt.wrapError | 0xc000a9ee80>: 
    failed starting cluster creation "agg-cluster" in resourcegroup="feature-aggregation-psqzvzzpdccv": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/feature-aggregation-psqzvzzpdccv/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/agg-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Customer should be able to forward control plane logs to a storage account and Event Hub via shoebox diagnostic settings · suite stage/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/shoebox_logs.go:270]: failed to create HCP cluster shoebox-hcp-cluster
Unexpected error:
    <*fmt.wrapError | 0xc000119ce0>: 
    failed to create HCP cluster shoebox-hcp-cluster: failed starting cluster creation "shoebox-hcp-cluster" in resourcegroup="shoebox-cluster-v8l2kdglf7kd": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/shoebox-cluster-v8l2kdglf7kd/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/shoebox-hcp-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/shoebox_logs.go:270]: failed to create HCP cluster shoebox-hcp-cluster
Unexpected error:
    <*fmt.wrapError | 0xc000119ce0>: 
    failed to create HCP cluster shoebox-hcp-cluster: failed starting cluster creation "shoebox-hcp-cluster" in resourcegroup="shoebox-cluster-v8l2kdglf7kd": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/shoebox-cluster-v8l2kdglf7kd/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/shoebox-hcp-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred