<
Single Day: Aug 4
Pick a day
>
View JSON API
↗ Open Failure patterns for this day

Environment: PROD

Time (UTC)JobFailed atResultPRFailed testsDetails
periodic-ci-Azure-ARO-HCP-main-periodic-prod-e2e-parallel-ocp-nightly
post-goodmerged PR
No additional run metadata captured
e2e failed n/a 39
e2e (39)
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
23:00:02 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP nightly channel for 4.22 · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:121]: HCP cluster rg-nightly-4-22-jt5xp2gjdzhd/cluster-nightly-4-22-rq88wm should provision
Unexpected error:
    <*fmt.wrapError | 0xc000880ae0>: 
    failed to create HCP cluster cluster-nightly-4-22-rq88wm: failed starting cluster creation "cluster-nightly-4-22-rq88wm" in resourcegroup="rg-nightly-4-22-jt5xp2gjdzhd": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-nightly-4-22-jt5xp2gjdzhd/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-nightly-4-22-rq88wm
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:121]: HCP cluster rg-nightly-4-22-jt5xp2gjdzhd/cluster-nightly-4-22-rq88wm should provision
Unexpected error:
    <*fmt.wrapError | 0xc000880ae0>: 
    failed to create HCP cluster cluster-nightly-4-22-rq88wm: failed starting cluster creation "cluster-nightly-4-22-rq88wm" in resourcegroup="rg-nightly-4-22-jt5xp2gjdzhd": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-nightly-4-22-jt5xp2gjdzhd/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-nightly-4-22-rq88wm
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
23:00:02 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 4.21 · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:110]: failed to create HCP cluster "cluster-zstream-4-21-pk8qg2" with version 4.21.23 on candidate channel
Unexpected error:
    <*fmt.wrapError | 0xc000850080>: 
    failed to create HCP cluster cluster-zstream-4-21-pk8qg2: failed starting cluster creation "cluster-zstream-4-21-pk8qg2" in resourcegroup="rg-zstream-upgrade-4-21-6c6665szhh6k": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-21-6c6665szhh6k/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-zstream-4-21-pk8qg2
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:110]: failed to create HCP cluster "cluster-zstream-4-21-pk8qg2" with version 4.21.23 on candidate channel
Unexpected error:
    <*fmt.wrapError | 0xc000850080>: 
    failed to create HCP cluster cluster-zstream-4-21-pk8qg2: failed starting cluster creation "cluster-zstream-4-21-pk8qg2" in resourcegroup="rg-zstream-upgrade-4-21-6c6665szhh6k": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-21-6c6665szhh6k/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-zstream-4-21-pk8qg2
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
23:00:02 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP nightly channel for 4.23 · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:121]: HCP cluster rg-nightly-4-23-v2x52h4f6m4g/cluster-nightly-4-23-wb9vfd should provision
Unexpected error:
    <*fmt.wrapError | 0xc000f69ea0>: 
    failed to create HCP cluster cluster-nightly-4-23-wb9vfd: failed starting cluster creation "cluster-nightly-4-23-wb9vfd" in resourcegroup="rg-nightly-4-23-v2x52h4f6m4g": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-nightly-4-23-v2x52h4f6m4g/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-nightly-4-23-wb9vfd
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:121]: HCP cluster rg-nightly-4-23-v2x52h4f6m4g/cluster-nightly-4-23-wb9vfd should provision
Unexpected error:
    <*fmt.wrapError | 0xc000f69ea0>: 
    failed to create HCP cluster cluster-nightly-4-23-wb9vfd: failed starting cluster creation "cluster-nightly-4-23-wb9vfd" in resourcegroup="rg-nightly-4-23-v2x52h4f6m4g": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-nightly-4-23-v2x52h4f6m4g/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-nightly-4-23-wb9vfd
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
23:00:02 UTC · failed at e2e · test Customer should upgrade a nodepool skipping one minor version (+2) from 4.20.z to 4.22.zLatest · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:462]: failed to create HCP cluster np-skip-minor-gdhn2r at version 4.22.0-0.nightly-multi-2026-08-03-132533
Unexpected error:
    <*fmt.wrapError | 0xc000a2c400>: 
    failed to create HCP cluster np-skip-minor-gdhn2r: failed starting cluster creation "np-skip-minor-gdhn2r" in resourcegroup="rg-np-skip-minor-gdhn2r-jbjxz2kwch2m": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-skip-minor-gdhn2r-jbjxz2kwch2m/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-skip-minor-gdhn2r
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:462]: failed to create HCP cluster np-skip-minor-gdhn2r at version 4.22.0-0.nightly-multi-2026-08-03-132533
Unexpected error:
    <*fmt.wrapError | 0xc000a2c400>: 
    failed to create HCP cluster np-skip-minor-gdhn2r: failed starting cluster creation "np-skip-minor-gdhn2r" in resourcegroup="rg-np-skip-minor-gdhn2r-jbjxz2kwch2m": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-skip-minor-gdhn2r-jbjxz2kwch2m/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-skip-minor-gdhn2r
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
23:00:02 UTC · failed at e2e · test Update HCPOpenShiftCluster Positive creates a cluster and updates tags with a PATCH request · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:161]: failed to create HCP cluster for patch-tags test
Unexpected error:
    <*fmt.wrapError | 0xc001417820>: 
    failed to create HCP cluster patch-tags-cluster: failed starting cluster creation "patch-tags-cluster" in resourcegroup="patch-tags-qtzqbxdx6p9w": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-tags-qtzqbxdx6p9w/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-tags-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:161]: failed to create HCP cluster for patch-tags test
Unexpected error:
    <*fmt.wrapError | 0xc001417820>: 
    failed to create HCP cluster patch-tags-cluster: failed starting cluster creation "patch-tags-cluster" in resourcegroup="patch-tags-qtzqbxdx6p9w": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-tags-qtzqbxdx6p9w/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-tags-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
23:00:02 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP nightly channel for 4.20 · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:121]: HCP cluster rg-nightly-4-20-s7lpkkft2rs6/cluster-nightly-4-20-p7xqb2 should provision
Unexpected error:
    <*fmt.wrapError | 0xc0001b54e0>: 
    failed to create HCP cluster cluster-nightly-4-20-p7xqb2: failed starting cluster creation "cluster-nightly-4-20-p7xqb2" in resourcegroup="rg-nightly-4-20-s7lpkkft2rs6": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-nightly-4-20-s7lpkkft2rs6/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-nightly-4-20-p7xqb2
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:121]: HCP cluster rg-nightly-4-20-s7lpkkft2rs6/cluster-nightly-4-20-p7xqb2 should provision
Unexpected error:
    <*fmt.wrapError | 0xc0001b54e0>: 
    failed to create HCP cluster cluster-nightly-4-20-p7xqb2: failed starting cluster creation "cluster-nightly-4-20-p7xqb2" in resourcegroup="rg-nightly-4-20-s7lpkkft2rs6": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-nightly-4-20-s7lpkkft2rs6/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-nightly-4-20-p7xqb2
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
23:00:02 UTC · failed at e2e · test Customer should be able to create a cluster and node pools with aggregated advanced features · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:184]: failed to create HCP cluster "agg-cluster" with aggregated settings
Unexpected error:
    <*fmt.wrapError | 0xc0008810e0>: 
    failed starting cluster creation "agg-cluster" in resourcegroup="feature-aggregation-6jlhmp4d9jzg": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/feature-aggregation-6jlhmp4d9jzg/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/agg-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:184]: failed to create HCP cluster "agg-cluster" with aggregated settings
Unexpected error:
    <*fmt.wrapError | 0xc0008810e0>: 
    failed starting cluster creation "agg-cluster" in resourcegroup="feature-aggregation-6jlhmp4d9jzg": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/feature-aggregation-6jlhmp4d9jzg/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/agg-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
23:00:02 UTC · failed at e2e · test Customer should be able to create an HCP cluster and manage ImageDigestMirrors · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/idms_lifecycle.go:113]: failed to create HCP cluster with ImageDigestMirrors
Unexpected error:
    <*fmt.wrapError | 0xc000e766e0>: 
    failed to create HCP cluster idms-e2e-hcp-cluster: failed starting cluster creation "idms-e2e-hcp-cluster" in resourcegroup="idms-mwvjgtb4v44t": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/idms-mwvjgtb4v44t/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/idms-e2e-hcp-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/idms_lifecycle.go:113]: failed to create HCP cluster with ImageDigestMirrors
Unexpected error:
    <*fmt.wrapError | 0xc000e766e0>: 
    failed to create HCP cluster idms-e2e-hcp-cluster: failed starting cluster creation "idms-e2e-hcp-cluster" in resourcegroup="idms-mwvjgtb4v44t": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/idms-mwvjgtb4v44t/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/idms-e2e-hcp-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
23:00:02 UTC · failed at e2e · test Customer should be able to create a cluster with an external auth config and get the external auth config · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:94]: failed to create HCP cluster for external auth test
Unexpected error:
    <*fmt.wrapError | 0xc00090c2e0>: 
    failed to create HCP cluster ea-cluster: failed starting cluster creation "ea-cluster" in resourcegroup="external-auth-cluster-6kglvgwpkg49": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/external-auth-cluster-6kglvgwpkg49/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/ea-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:94]: failed to create HCP cluster for external auth test
Unexpected error:
    <*fmt.wrapError | 0xc00090c2e0>: 
    failed to create HCP cluster ea-cluster: failed starting cluster creation "ea-cluster" in resourcegroup="external-auth-cluster-6kglvgwpkg49": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/external-auth-cluster-6kglvgwpkg49/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/ea-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
23:00:02 UTC · failed at e2e · test Customer should not be able to reuse subnets and NSGs between clusters · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_nsg_subnet_reuse.go:107]: failed to create seed HCP cluster basic-cluster
Unexpected error:
    <*fmt.wrapError | 0xc000239740>: 
    failed to create HCP cluster basic-cluster: failed starting cluster creation "basic-cluster" in resourcegroup="rg-cluster-nsg-subnet-reuse-mbrznx5m6r57": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cluster-nsg-subnet-reuse-mbrznx5m6r57/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/basic-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_nsg_subnet_reuse.go:107]: failed to create seed HCP cluster basic-cluster
Unexpected error:
    <*fmt.wrapError | 0xc000239740>: 
    failed to create HCP cluster basic-cluster: failed starting cluster creation "basic-cluster" in resourcegroup="rg-cluster-nsg-subnet-reuse-mbrznx5m6r57": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cluster-nsg-subnet-reuse-mbrznx5m6r57/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/basic-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
23:00:02 UTC · failed at e2e · test Customer should create a cluster with private KAS and verify API server is only reachable from VNet while ingress remains public · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:90]: failed to create HCP cluster "private-kas" with private KAS
Unexpected error:
    <*fmt.wrapError | 0xc000e04940>: 
    failed to create HCP cluster private-kas: failed starting cluster creation "private-kas" in resourcegroup="private-kas-v9gxcnvsrk8j": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/private-kas-v9gxcnvsrk8j/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/private-kas
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:90]: failed to create HCP cluster "private-kas" with private KAS
Unexpected error:
    <*fmt.wrapError | 0xc000e04940>: 
    failed to create HCP cluster private-kas: failed starting cluster creation "private-kas" in resourcegroup="private-kas-v9gxcnvsrk8j": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/private-kas-v9gxcnvsrk8j/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/private-kas
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
23:00:02 UTC · failed at e2e · test Customer should be able to create an HCP cluster then delete it by deleting the customer resource group · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delete_cx_rg.go:90]: failed to create HCP cluster "cx-rg-hcp-cluster"
Unexpected error:
    <*fmt.wrapError | 0xc000f8cf00>: 
    failed to create HCP cluster cx-rg-hcp-cluster: failed starting cluster creation "cx-rg-hcp-cluster" in resourcegroup="cx-rg-cluster-lbnrv57b5c5x": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/cx-rg-cluster-lbnrv57b5c5x/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cx-rg-hcp-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delete_cx_rg.go:90]: failed to create HCP cluster "cx-rg-hcp-cluster"
Unexpected error:
    <*fmt.wrapError | 0xc000f8cf00>: 
    failed to create HCP cluster cx-rg-hcp-cluster: failed starting cluster creation "cx-rg-hcp-cluster" in resourcegroup="cx-rg-cluster-lbnrv57b5c5x": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/cx-rg-cluster-lbnrv57b5c5x/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cx-rg-hcp-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
23:00:02 UTC · failed at e2e · test Customer should be able to test admin credentials before cluster ready, then full admin credential lifecycle · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:105]: failed to begin creating HCP cluster "admin-cred-lifecycle-jh28wk"
Unexpected error:
    <*fmt.wrapError | 0xc0001f85a0>: 
    failed starting cluster creation "admin-cred-lifecycle-jh28wk" in resourcegroup="admin-credential-lifecycle-test-s9pdm56vqt8r": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/admin-credential-lifecycle-test-s9pdm56vqt8r/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/admin-cred-lifecycle-jh28wk
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:105]: failed to begin creating HCP cluster "admin-cred-lifecycle-jh28wk"
Unexpected error:
    <*fmt.wrapError | 0xc0001f85a0>: 
    failed starting cluster creation "admin-cred-lifecycle-jh28wk" in resourcegroup="admin-credential-lifecycle-test-s9pdm56vqt8r": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/admin-credential-lifecycle-test-s9pdm56vqt8r/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/admin-cred-lifecycle-jh28wk
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
23:00:02 UTC · failed at e2e · test Customer should be able to create several HCP clusters in their customer resource group, but not in the same managed resource group · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:111]: failed to begin creation of first cluster "basic-hcp-cluster"
Unexpected error:
    <*fmt.wrapError | 0xc000865400>: 
    failed starting cluster creation "basic-hcp-cluster" in resourcegroup="customer-rg-rg9h9286s2wl": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/customer-rg-rg9h9286s2wl/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/basic-hcp-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:111]: failed to begin creation of first cluster "basic-hcp-cluster"
Unexpected error:
    <*fmt.wrapError | 0xc000865400>: 
    failed starting cluster creation "basic-hcp-cluster" in resourcegroup="customer-rg-rg9h9286s2wl": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/customer-rg-rg9h9286s2wl/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/basic-hcp-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
23:00:02 UTC · failed at e2e · test Customer should be able to create an HCP cluster and custom node pool osDisk size · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_nodepool_osdisk.go:80]: failed to create HCP cluster "hcp-cluster-np-128"
Unexpected error:
    <*fmt.wrapError | 0xc000df3680>: 
    failed to create HCP cluster hcp-cluster-np-128: failed starting cluster creation "hcp-cluster-np-128" in resourcegroup="clusternp128-pqrf25th2zbs": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/clusternp128-pqrf25th2zbs/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/hcp-cluster-np-128
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_nodepool_osdisk.go:80]: failed to create HCP cluster "hcp-cluster-np-128"
Unexpected error:
    <*fmt.wrapError | 0xc000df3680>: 
    failed to create HCP cluster hcp-cluster-np-128: failed starting cluster creation "hcp-cluster-np-128" in resourcegroup="clusternp128-pqrf25th2zbs": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/clusternp128-pqrf25th2zbs/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/hcp-cluster-np-128
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
23:00:02 UTC · failed at e2e · test Customer should respect cluster-wide node limits with nodepool autoscaling · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:291]: failed to create HCP cluster node-limit-cluster with MaxNodesTotal limit
Unexpected error:
    <*fmt.wrapError | 0xc000dc4c60>: 
    failed to create HCP cluster node-limit-cluster: failed starting cluster creation "node-limit-cluster" in resourcegroup="node-limit-test-j7xkpg8qdtdt": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/node-limit-test-j7xkpg8qdtdt/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/node-limit-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:291]: failed to create HCP cluster node-limit-cluster with MaxNodesTotal limit
Unexpected error:
    <*fmt.wrapError | 0xc000dc4c60>: 
    failed to create HCP cluster node-limit-cluster: failed starting cluster creation "node-limit-cluster" in resourcegroup="node-limit-test-j7xkpg8qdtdt": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/node-limit-test-j7xkpg8qdtdt/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/node-limit-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
23:00:02 UTC · failed at e2e · test Customer should be able to update node pool labels and taints · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_labels_taints.go:92]: failed to create HCP cluster cluster-np-labels-taints
Unexpected error:
    <*fmt.wrapError | 0xc000a2dce0>: 
    failed to create HCP cluster cluster-np-labels-taints: failed starting cluster creation "cluster-np-labels-taints" in resourcegroup="rg-np-labels-taints-gdvt27498ftd": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-labels-taints-gdvt27498ftd/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-np-labels-taints
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_labels_taints.go:92]: failed to create HCP cluster cluster-np-labels-taints
Unexpected error:
    <*fmt.wrapError | 0xc000a2dce0>: 
    failed to create HCP cluster cluster-np-labels-taints: failed starting cluster creation "cluster-np-labels-taints" in resourcegroup="rg-np-labels-taints-gdvt27498ftd": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-labels-taints-gdvt27498ftd/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-np-labels-taints
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
23:00:02 UTC · failed at e2e · test Customer should be able to create an HCP cluster with custom autoscaling and update the autoscaling configuration · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:108]: failed to create HCP cluster "autoscaling-hcp-cluster" with custom autoscaling
Unexpected error:
    <*fmt.wrapError | 0xc00158efe0>: 
    failed to create HCP cluster autoscaling-hcp-cluster: failed starting cluster creation "autoscaling-hcp-cluster" in resourcegroup="autoscaling-cluster-5s455h7jbszg": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/autoscaling-cluster-5s455h7jbszg/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/autoscaling-hcp-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:108]: failed to create HCP cluster "autoscaling-hcp-cluster" with custom autoscaling
Unexpected error:
    <*fmt.wrapError | 0xc00158efe0>: 
    failed to create HCP cluster autoscaling-hcp-cluster: failed starting cluster creation "autoscaling-hcp-cluster" in resourcegroup="autoscaling-cluster-5s455h7jbszg": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/autoscaling-cluster-5s455h7jbszg/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/autoscaling-hcp-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
23:00:02 UTC · failed at e2e · test Customer should be able to create node pools with ARM64-based VMs · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/arm64_nodepool.go:88]: failed to create HCP cluster "arm64-vm-hcp-cluster"
Unexpected error:
    <*fmt.wrapError | 0xc000edbb20>: 
    failed to create HCP cluster arm64-vm-hcp-cluster: failed starting cluster creation "arm64-vm-hcp-cluster" in resourcegroup="arm64-vm-cluster-9x2g9tdkl2w7": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/arm64-vm-cluster-9x2g9tdkl2w7/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/arm64-vm-hcp-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/arm64_nodepool.go:88]: failed to create HCP cluster "arm64-vm-hcp-cluster"
Unexpected error:
    <*fmt.wrapError | 0xc000edbb20>: 
    failed to create HCP cluster arm64-vm-hcp-cluster: failed starting cluster creation "arm64-vm-hcp-cluster" in resourcegroup="arm64-vm-cluster-9x2g9tdkl2w7": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/arm64-vm-cluster-9x2g9tdkl2w7/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/arm64-vm-hcp-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
23:00:02 UTC · failed at e2e · test Customer should create an HCP cluster and validate TLS certificates · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:95]: failed to create HCP cluster for TLS endpoint test
Unexpected error:
    <*fmt.wrapError | 0xc0008fc2c0>: 
    failed to create HCP cluster tls-endpoint-hcp-cluster: failed starting cluster creation "tls-endpoint-hcp-cluster" in resourcegroup="tls-endpoint-cluster-mn8lx8lwwpwl": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/tls-endpoint-cluster-mn8lx8lwwpwl/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/tls-endpoint-hcp-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:95]: failed to create HCP cluster for TLS endpoint test
Unexpected error:
    <*fmt.wrapError | 0xc0008fc2c0>: 
    failed to create HCP cluster tls-endpoint-hcp-cluster: failed starting cluster creation "tls-endpoint-hcp-cluster" in resourcegroup="tls-endpoint-cluster-mn8lx8lwwpwl": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/tls-endpoint-cluster-mn8lx8lwwpwl/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/tls-endpoint-hcp-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
23:00:02 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 4.22 · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:110]: failed to create HCP cluster "cluster-zstream-4-22-ltx6q5" with version 4.22.6 on candidate channel
Unexpected error:
    <*fmt.wrapError | 0xc000fa8e20>: 
    failed to create HCP cluster cluster-zstream-4-22-ltx6q5: failed starting cluster creation "cluster-zstream-4-22-ltx6q5" in resourcegroup="rg-zstream-upgrade-4-22-8lwtphq6x5n2": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-22-8lwtphq6x5n2/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-zstream-4-22-ltx6q5
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:110]: failed to create HCP cluster "cluster-zstream-4-22-ltx6q5" with version 4.22.6 on candidate channel
Unexpected error:
    <*fmt.wrapError | 0xc000fa8e20>: 
    failed to create HCP cluster cluster-zstream-4-22-ltx6q5: failed starting cluster creation "cluster-zstream-4-22-ltx6q5" in resourcegroup="rg-zstream-upgrade-4-22-8lwtphq6x5n2": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-22-8lwtphq6x5n2/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-zstream-4-22-ltx6q5
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
23:00:02 UTC · failed at e2e · test Customer should be able to create an HCP cluster and manage pull secrets · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_pullsecret.go:113]: failed to create HCP cluster for pull secret test
Unexpected error:
    <*fmt.wrapError | 0xc000f323a0>: 
    failed to create HCP cluster pullsecret-hcp-cluster: failed starting cluster creation "pullsecret-hcp-cluster" in resourcegroup="pullsecret-test-lg9h6crbbhps": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/pullsecret-test-lg9h6crbbhps/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/pullsecret-hcp-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_pullsecret.go:113]: failed to create HCP cluster for pull secret test
Unexpected error:
    <*fmt.wrapError | 0xc000f323a0>: 
    failed to create HCP cluster pullsecret-hcp-cluster: failed starting cluster creation "pullsecret-hcp-cluster" in resourcegroup="pullsecret-test-lg9h6crbbhps": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/pullsecret-test-lg9h6crbbhps/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/pullsecret-hcp-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
23:00:02 UTC · failed at e2e · test Nodepool Ephemeral OS Disk should create a nodepool with ephemeral OS disk when autoRepair is enabled · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_ephemeral_osdisk.go:112]: failed to create HCP cluster ephemeral-disk
Unexpected error:
    <*fmt.wrapError | 0xc001593c00>: 
    failed to create HCP cluster ephemeral-disk: failed starting cluster creation "ephemeral-disk" in resourcegroup="ephemeral-osdisk-66jhndjn8cs5": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/ephemeral-osdisk-66jhndjn8cs5/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/ephemeral-disk
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_ephemeral_osdisk.go:112]: failed to create HCP cluster ephemeral-disk
Unexpected error:
    <*fmt.wrapError | 0xc001593c00>: 
    failed to create HCP cluster ephemeral-disk: failed starting cluster creation "ephemeral-disk" in resourcegroup="ephemeral-osdisk-66jhndjn8cs5": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/ephemeral-osdisk-66jhndjn8cs5/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/ephemeral-disk
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
23:00:02 UTC · failed at e2e · test Customer should be able to lifecycle and confirm external auth on a cluster · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_list_and_verify.go:87]: failed to create HCP cluster for external auth list test
Unexpected error:
    <*fmt.wrapError | 0xc000a13da0>: 
    failed to create HCP cluster ea-listw2svx4: failed starting cluster creation "ea-listw2svx4" in resourcegroup="ea-list-v54xnxcjdrxr": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/ea-list-v54xnxcjdrxr/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/ea-listw2svx4
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_list_and_verify.go:87]: failed to create HCP cluster for external auth list test
Unexpected error:
    <*fmt.wrapError | 0xc000a13da0>: 
    failed to create HCP cluster ea-listw2svx4: failed starting cluster creation "ea-listw2svx4" in resourcegroup="ea-list-v54xnxcjdrxr": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/ea-list-v54xnxcjdrxr/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/ea-listw2svx4
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
23:00:02 UTC · failed at e2e · test Create HCPOpenShiftCluster with Private KeyVault should create a cluster with private keyvault using v20251223preview API · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:98]: failed to create HCP cluster "private-kv-cluster" with private keyvault
Unexpected error:
    <*fmt.wrapError | 0xc0009cd500>: 
    failed starting cluster creation "private-kv-cluster" in resourcegroup="private-keyvault-rxcj2cz4wx2k": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/private-keyvault-rxcj2cz4wx2k/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/private-kv-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:98]: failed to create HCP cluster "private-kv-cluster" with private keyvault
Unexpected error:
    <*fmt.wrapError | 0xc0009cd500>: 
    failed starting cluster creation "private-kv-cluster" in resourcegroup="private-keyvault-rxcj2cz4wx2k": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/private-keyvault-rxcj2cz4wx2k/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/private-kv-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
23:00:02 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 4.19 · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:110]: failed to create HCP cluster "cluster-zstream-4-19-h48rpc" with version 4.19.39 on candidate channel
Unexpected error:
    <*fmt.wrapError | 0xc0009213a0>: 
    failed to create HCP cluster cluster-zstream-4-19-h48rpc: failed starting cluster creation "cluster-zstream-4-19-h48rpc" in resourcegroup="rg-zstream-upgrade-4-19-knj6gst2bxn5": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-19-knj6gst2bxn5/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-zstream-4-19-h48rpc
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:110]: failed to create HCP cluster "cluster-zstream-4-19-h48rpc" with version 4.19.39 on candidate channel
Unexpected error:
    <*fmt.wrapError | 0xc0009213a0>: 
    failed to create HCP cluster cluster-zstream-4-19-h48rpc: failed starting cluster creation "cluster-zstream-4-19-h48rpc" in resourcegroup="rg-zstream-upgrade-4-19-knj6gst2bxn5": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-19-knj6gst2bxn5/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-zstream-4-19-h48rpc
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
23:00:02 UTC · failed at e2e · test ARO HCP Service should successfully provision cluster when MI role assignments are added after cluster creation · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:133]: failed to start cluster creation
Unexpected error:
    <*exported.ResponseError | 0xc0013393e0>: 
    PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/delayed-rbac-4xq9wmrr6kfv/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/delayed-rbac-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:133]: failed to start cluster creation
Unexpected error:
    <*exported.ResponseError | 0xc0013393e0>: 
    PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/delayed-rbac-4xq9wmrr6kfv/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/delayed-rbac-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
23:00:02 UTC · failed at e2e · test Customer should be able to use workload identity via the cluster OIDC issuer URL · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/oidc_issuer_workload_identity.go:217]: failed to create HCP cluster oidc-wi-cluster
Unexpected error:
    <*fmt.wrapError | 0xc000924460>: 
    failed to create HCP cluster oidc-wi-cluster: failed starting cluster creation "oidc-wi-cluster" in resourcegroup="oidc-wi-hv82rv8d9h8w": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/oidc-wi-hv82rv8d9h8w/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/oidc-wi-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/oidc_issuer_workload_identity.go:217]: failed to create HCP cluster oidc-wi-cluster
Unexpected error:
    <*fmt.wrapError | 0xc000924460>: 
    failed to create HCP cluster oidc-wi-cluster: failed starting cluster creation "oidc-wi-cluster" in resourcegroup="oidc-wi-hv82rv8d9h8w": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/oidc-wi-hv82rv8d9h8w/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/oidc-wi-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
23:00:02 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 4.20 · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:110]: failed to create HCP cluster "cluster-zstream-4-20-n7vsnm" with version 4.20.28 on candidate channel
Unexpected error:
    <*fmt.wrapError | 0xc0006961e0>: 
    failed to create HCP cluster cluster-zstream-4-20-n7vsnm: failed starting cluster creation "cluster-zstream-4-20-n7vsnm" in resourcegroup="rg-zstream-upgrade-4-20-vjfxd2hbndz8": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-20-vjfxd2hbndz8/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-zstream-4-20-n7vsnm
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:110]: failed to create HCP cluster "cluster-zstream-4-20-n7vsnm" with version 4.20.28 on candidate channel
Unexpected error:
    <*fmt.wrapError | 0xc0006961e0>: 
    failed to create HCP cluster cluster-zstream-4-20-n7vsnm: failed starting cluster creation "cluster-zstream-4-20-n7vsnm" in resourcegroup="rg-zstream-upgrade-4-20-vjfxd2hbndz8": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-20-vjfxd2hbndz8/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-zstream-4-20-n7vsnm
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
23:00:02 UTC · failed at e2e · test Customer should be able to create a HCP cluster and use cilium CNI plugin · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_cni_cilium.go:88]: failed to create HCP cluster "cilium-cl" without CNI
Unexpected error:
    <*fmt.wrapError | 0xc001454e20>: 
    failed to create HCP cluster cilium-cl: failed starting cluster creation "cilium-cl" in resourcegroup="cni-cilium-q4dj9ptfr9qh": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/cni-cilium-q4dj9ptfr9qh/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cilium-cl
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_cni_cilium.go:88]: failed to create HCP cluster "cilium-cl" without CNI
Unexpected error:
    <*fmt.wrapError | 0xc001454e20>: 
    failed to create HCP cluster cilium-cl: failed starting cluster creation "cilium-cl" in resourcegroup="cni-cilium-q4dj9ptfr9qh": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/cni-cilium-q4dj9ptfr9qh/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cilium-cl
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
23:00:02 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP nightly channel for 4.21 · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:121]: HCP cluster rg-nightly-4-21-697l6h5nhscl/cluster-nightly-4-21-p4kpmf should provision
Unexpected error:
    <*fmt.wrapError | 0xc000f36280>: 
    failed to create HCP cluster cluster-nightly-4-21-p4kpmf: failed starting cluster creation "cluster-nightly-4-21-p4kpmf" in resourcegroup="rg-nightly-4-21-697l6h5nhscl": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-nightly-4-21-697l6h5nhscl/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-nightly-4-21-p4kpmf
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:121]: HCP cluster rg-nightly-4-21-697l6h5nhscl/cluster-nightly-4-21-p4kpmf should provision
Unexpected error:
    <*fmt.wrapError | 0xc000f36280>: 
    failed to create HCP cluster cluster-nightly-4-21-p4kpmf: failed starting cluster creation "cluster-nightly-4-21-p4kpmf" in resourcegroup="rg-nightly-4-21-697l6h5nhscl": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-nightly-4-21-697l6h5nhscl/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-nightly-4-21-p4kpmf
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
23:00:02 UTC · failed at e2e · test Customer should not be able to perform various invalid operations on cluster resources · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/simple_negative_cases.go:110]: failed to create HCP cluster negative-tests-cluster
Unexpected error:
    <*fmt.wrapError | 0xc000f8a3e0>: 
    failed to create HCP cluster negative-tests-cluster: failed starting cluster creation "negative-tests-cluster" in resourcegroup="rg-negative-tests-z42jvfltwwc4": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-negative-tests-z42jvfltwwc4/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/negative-tests-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/simple_negative_cases.go:110]: failed to create HCP cluster negative-tests-cluster
Unexpected error:
    <*fmt.wrapError | 0xc000f8a3e0>: 
    failed to create HCP cluster negative-tests-cluster: failed starting cluster creation "negative-tests-cluster" in resourcegroup="rg-negative-tests-z42jvfltwwc4": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-negative-tests-z42jvfltwwc4/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/negative-tests-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
23:00:02 UTC · failed at e2e · test Customer should be able to forward control plane logs to a storage account and Event Hub via shoebox diagnostic settings · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/shoebox_logs.go:270]: failed to create HCP cluster shoebox-hcp-cluster
Unexpected error:
    <*fmt.wrapError | 0xc00107b4a0>: 
    failed to create HCP cluster shoebox-hcp-cluster: failed starting cluster creation "shoebox-hcp-cluster" in resourcegroup="shoebox-cluster-gq4nzbrwl6hd": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/shoebox-cluster-gq4nzbrwl6hd/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/shoebox-hcp-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/shoebox_logs.go:270]: failed to create HCP cluster shoebox-hcp-cluster
Unexpected error:
    <*fmt.wrapError | 0xc00107b4a0>: 
    failed to create HCP cluster shoebox-hcp-cluster: failed starting cluster creation "shoebox-hcp-cluster" in resourcegroup="shoebox-cluster-gq4nzbrwl6hd": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/shoebox-cluster-gq4nzbrwl6hd/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/shoebox-hcp-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
23:00:02 UTC · failed at e2e · test Customer should be able to create a no-CNI private cluster with a private key vault, a nodepool and install cilium CNI successfully · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:105]: failed to create HCP cluster "cilium-cluster" with no CNI and private etcd
Unexpected error:
    <*fmt.wrapError | 0xc000c760c0>: 
    failed starting cluster creation "cilium-cluster" in resourcegroup="complex-cilium-kv-w674kzl2r5nc": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/complex-cilium-kv-w674kzl2r5nc/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cilium-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:105]: failed to create HCP cluster "cilium-cluster" with no CNI and private etcd
Unexpected error:
    <*fmt.wrapError | 0xc000c760c0>: 
    failed starting cluster creation "cilium-cluster" in resourcegroup="complex-cilium-kv-w674kzl2r5nc": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/complex-cilium-kv-w674kzl2r5nc/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cilium-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
23:00:02 UTC · failed at e2e · test Update HCPOpenShiftCluster Negative creates a cluster and fails to update its name with a PATCH request · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:84]: failed to create HCP cluster for patch-name test
Unexpected error:
    <*fmt.wrapError | 0xc001060680>: 
    failed to create HCP cluster patch-name-cluster: failed starting cluster creation "patch-name-cluster" in resourcegroup="patch-name-z8fj6sbqtn9x": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-name-z8fj6sbqtn9x/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-name-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:84]: failed to create HCP cluster for patch-name test
Unexpected error:
    <*fmt.wrapError | 0xc001060680>: 
    failed to create HCP cluster patch-name-cluster: failed starting cluster creation "patch-name-cluster" in resourcegroup="patch-name-z8fj6sbqtn9x": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-name-z8fj6sbqtn9x/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-name-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
23:00:02 UTC · failed at e2e · test Customer should be able to create a cluster with default autoscaling and a nodepool with autoscaling enabled up to replica limits · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:102]: failed to create HCP cluster np-autoscale-cluster
Unexpected error:
    <*fmt.wrapError | 0xc000fb8900>: 
    failed to create HCP cluster np-autoscale-cluster: failed starting cluster creation "np-autoscale-cluster" in resourcegroup="np-autoscaling-h8qqf75dsn4m": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/np-autoscaling-h8qqf75dsn4m/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-autoscale-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:102]: failed to create HCP cluster np-autoscale-cluster
Unexpected error:
    <*fmt.wrapError | 0xc000fb8900>: 
    failed to create HCP cluster np-autoscale-cluster: failed starting cluster creation "np-autoscale-cluster" in resourcegroup="np-autoscaling-h8qqf75dsn4m": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/np-autoscaling-h8qqf75dsn4m/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-autoscale-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
23:00:02 UTC · failed at e2e · test Customer should downgrade a nodepool to a lower minor version from 4.21.zLatest to 4.19.zLatest · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:741]: failed to create HCP cluster
Unexpected error:
    <*fmt.wrapError | 0xc000f58ba0>: 
    failed to create HCP cluster np-dg-minor-btwp6s: failed starting cluster creation "np-dg-minor-btwp6s" in resourcegroup="rg-np-dg-minor-btwp6s-5dk7j589j98p": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-dg-minor-btwp6s-5dk7j589j98p/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-dg-minor-btwp6s
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:741]: failed to create HCP cluster
Unexpected error:
    <*fmt.wrapError | 0xc000f58ba0>: 
    failed to create HCP cluster np-dg-minor-btwp6s: failed starting cluster creation "np-dg-minor-btwp6s" in resourcegroup="rg-np-dg-minor-btwp6s-5dk7j589j98p": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-dg-minor-btwp6s-5dk7j589j98p/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-dg-minor-btwp6s
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
23:00:02 UTC · failed at e2e · test Authorized CIDRs Connectivity should allow API access only from authorized VM IP · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:111]: failed to create HCP cluster "cidr-connectivity-test" with authorized CIDRs
Unexpected error:
    <*fmt.wrapError | 0xc0016a76e0>: 
    failed to create HCP cluster cidr-connectivity-test: failed starting cluster creation "cidr-connectivity-test" in resourcegroup="e2e-cidr-connectivity-rjwvc6q4rxkr": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-rjwvc6q4rxkr/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cidr-connectivity-test
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:111]: failed to create HCP cluster "cidr-connectivity-test" with authorized CIDRs
Unexpected error:
    <*fmt.wrapError | 0xc0016a76e0>: 
    failed to create HCP cluster cidr-connectivity-test: failed starting cluster creation "cidr-connectivity-test" in resourcegroup="e2e-cidr-connectivity-rjwvc6q4rxkr": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-rjwvc6q4rxkr/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cidr-connectivity-test
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
23:00:02 UTC · failed at e2e · test Customer should be able to create an HCP cluster with Image Registry not present · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/image_registry_cluster_create.go:89]: failed to create HCP cluster with image registry disabled
Unexpected error:
    <*fmt.wrapError | 0xc000e01760>: 
    failed to create HCP cluster disabled-image-registry-hcp-cluster: failed starting cluster creation "disabled-image-registry-hcp-cluster" in resourcegroup="disabled-image-registry-msp67whvq569": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/disabled-image-registry-msp67whvq569/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/disabled-image-registry-hcp-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/image_registry_cluster_create.go:89]: failed to create HCP cluster with image registry disabled
Unexpected error:
    <*fmt.wrapError | 0xc000e01760>: 
    failed to create HCP cluster disabled-image-registry-hcp-cluster: failed starting cluster creation "disabled-image-registry-hcp-cluster" in resourcegroup="disabled-image-registry-msp67whvq569": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/disabled-image-registry-msp67whvq569/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/disabled-image-registry-hcp-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel
post-goodmerged PR
No additional run metadata captured
e2e failed n/a 7
e2e (7)
ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift
21:57:53 UTC · failed at e2e · test Customer should be able to create a no-CNI private cluster with a private key vault, a nodepool and install cilium CNI successfully · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:105]: failed to create HCP cluster "cilium-cluster" with no CNI and private etcd
Unexpected error:
    <*fmt.wrapError | 0xc0009063e0>: 
    failed waiting for cluster="cilium-cluster" in resourcegroup="complex-cilium-kv-nbtsdsjt7d7f" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/87740678-bd5f-49c6-ae8d-55cc48a0e73d
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/87740678-bd5f-49c6-ae8d-55cc48a0e73d",
      "name": "87740678-bd5f-49c6-ae8d-55cc48a0e73d",
      "status": "Failed",
      "startTime": "2026-08-04T22:01:48.859612091Z",
      "endTime": "2026-08-04T22:20:50.330983848Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:105]: failed to create HCP cluster "cilium-cluster" with no CNI and private etcd
Unexpected error:
    <*fmt.wrapError | 0xc0009063e0>: 
    failed waiting for cluster="cilium-cluster" in resourcegroup="complex-cilium-kv-nbtsdsjt7d7f" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/87740678-bd5f-49c6-ae8d-55cc48a0e73d
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/87740678-bd5f-49c6-ae8d-55cc48a0e73d",
      "name": "87740678-bd5f-49c6-ae8d-55cc48a0e73d",
      "status": "Failed",
      "startTime": "2026-08-04T22:01:48.859612091Z",
      "endTime": "2026-08-04T22:20:50.330983848Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; [hypershiftHostedCluster] hosted cluster is not available: ComponentsNotAvailable; hosted cluster degraded: UnavailableReplicas: [cluster-network-operator, router]; provider Microsoft.RedHatOpenShift
21:57:53 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.22 · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:121]: HCP cluster rg-candidate-4-22-rqf9dr7wzqcn/cluster-candidate-4-22-ppxdfc should provision
Unexpected error:
    <*fmt.wrapError | 0xc000880260>: 
    failed to create HCP cluster cluster-candidate-4-22-ppxdfc: failed waiting for cluster="cluster-candidate-4-22-ppxdfc" in resourcegroup="rg-candidate-4-22-rqf9dr7wzqcn" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/0432bd62-a18f-4bd9-b507-1577be3d81f2
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/0432bd62-a18f-4bd9-b507-1577be3d81f2",
      "name": "0432bd62-a18f-4bd9-b507-1577be3d81f2",
      "status": "Failed",
      "startTime": "2026-08-04T22:35:56.549310447Z",
      "endTime": "2026-08-04T22:55:01.632384457Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e; [hypershiftHostedCluster] hosted cluster is not available: ComponentsNotAvailable: Waiting for components to be available: cluster-network-operator, ingress-operator; hosted cluster degraded: UnavailableReplicas: [cluster-network-operator deployment has 1 unavailable replicas, router deployment has 1 unavailable replicas]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:121]: HCP cluster rg-candidate-4-22-rqf9dr7wzqcn/cluster-candidate-4-22-ppxdfc should provision
Unexpected error:
    <*fmt.wrapError | 0xc000880260>: 
    failed to create HCP cluster cluster-candidate-4-22-ppxdfc: failed waiting for cluster="cluster-candidate-4-22-ppxdfc" in resourcegroup="rg-candidate-4-22-rqf9dr7wzqcn" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/0432bd62-a18f-4bd9-b507-1577be3d81f2
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/0432bd62-a18f-4bd9-b507-1577be3d81f2",
      "name": "0432bd62-a18f-4bd9-b507-1577be3d81f2",
      "status": "Failed",
      "startTime": "2026-08-04T22:35:56.549310447Z",
      "endTime": "2026-08-04T22:55:01.632384457Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e; [hypershiftHostedCluster] hosted cluster is not available: ComponentsNotAvailable: Waiting for components to be available: cluster-network-operator, ingress-operator; hosted cluster degraded: UnavailableReplicas: [cluster-network-operator deployment has 1 unavailable replicas, router deployment has 1 unavailable replicas]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift
21:57:53 UTC · failed at e2e · test Customer should create a cluster with private KAS and verify API server is only reachable from VNet while ingress remains public · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:90]: failed to create HCP cluster "private-kas" with private KAS
Unexpected error:
    <*fmt.wrapError | 0xc000f37bc0>: 
    failed to create HCP cluster private-kas: failed waiting for cluster="private-kas" in resourcegroup="private-kas-fwqhhr5z96lv" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/57fd07e7-13da-4b2d-b9cf-eb06951f5632
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/57fd07e7-13da-4b2d-b9cf-eb06951f5632",
      "name": "57fd07e7-13da-4b2d-b9cf-eb06951f5632",
      "status": "Failed",
      "startTime": "2026-08-04T22:01:09.661544489Z",
      "endTime": "2026-08-04T22:20:10.487443049Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:90]: failed to create HCP cluster "private-kas" with private KAS
Unexpected error:
    <*fmt.wrapError | 0xc000f37bc0>: 
    failed to create HCP cluster private-kas: failed waiting for cluster="private-kas" in resourcegroup="private-kas-fwqhhr5z96lv" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/57fd07e7-13da-4b2d-b9cf-eb06951f5632
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/57fd07e7-13da-4b2d-b9cf-eb06951f5632",
      "name": "57fd07e7-13da-4b2d-b9cf-eb06951f5632",
      "status": "Failed",
      "startTime": "2026-08-04T22:01:09.661544489Z",
      "endTime": "2026-08-04T22:20:10.487443049Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
cluster entered terminal Failed state after role assignment deployment
21:57:53 UTC · failed at e2e · test ARO HCP Service should successfully provision cluster when MI role assignments are added after cluster creation · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:220]: Timed out after 2520.001s.
cluster should eventually succeed after role assignments are created
The function passed to Eventually failed at /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:216 with:
cluster entered terminal Failed state after role assignment deployment
Expected
    <armredhatopenshifthcp.ProvisioningState>: ...
not to equal
    <armredhatopenshifthcp.ProvisioningState>: ...
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:220]: Timed out after 2520.001s.
cluster should eventually succeed after role assignments are created
The function passed to Eventually failed at /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:216 with:
cluster entered terminal Failed state after role assignment deployment
Expected
    <armredhatopenshifthcp.ProvisioningState>: ...
not to equal
    <armredhatopenshifthcp.ProvisioningState>: ...
ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift
21:57:53 UTC · failed at e2e · test Customer should be able to create a cluster and node pools with aggregated advanced features · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:184]: failed to create HCP cluster "agg-cluster" with aggregated settings
Unexpected error:
    <*fmt.wrapError | 0xc001203de0>: 
    failed waiting for cluster="agg-cluster" in resourcegroup="feature-aggregation-csp6srtnl2bj" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/0d601f00-03b5-4b6f-a323-423bcaf913c2
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/0d601f00-03b5-4b6f-a323-423bcaf913c2",
      "name": "0d601f00-03b5-4b6f-a323-423bcaf913c2",
      "status": "Failed",
      "startTime": "2026-08-04T22:02:28.829566077Z",
      "endTime": "2026-08-04T22:21:30.393449465Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:184]: failed to create HCP cluster "agg-cluster" with aggregated settings
Unexpected error:
    <*fmt.wrapError | 0xc001203de0>: 
    failed waiting for cluster="agg-cluster" in resourcegroup="feature-aggregation-csp6srtnl2bj" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/0d601f00-03b5-4b6f-a323-423bcaf913c2
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/0d601f00-03b5-4b6f-a323-423bcaf913c2",
      "name": "0d601f00-03b5-4b6f-a323-423bcaf913c2",
      "status": "Failed",
      "startTime": "2026-08-04T22:02:28.829566077Z",
      "endTime": "2026-08-04T22:21:30.393449465Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift
21:57:53 UTC · failed at e2e · test Create HCPOpenShiftCluster with Private KeyVault should create a cluster with private keyvault using v20251223preview API · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:98]: failed to create HCP cluster "private-kv-cluster" with private keyvault
Unexpected error:
    <*fmt.wrapError | 0xc000ba1960>: 
    failed waiting for cluster="private-kv-cluster" in resourcegroup="private-keyvault-m2m95r25649h" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/a8a46c46-de01-4ce8-8094-85e125ce1f3d
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/a8a46c46-de01-4ce8-8094-85e125ce1f3d",
      "name": "a8a46c46-de01-4ce8-8094-85e125ce1f3d",
      "status": "Failed",
      "startTime": "2026-08-04T22:01:47.713085545Z",
      "endTime": "2026-08-04T22:20:50.315977298Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:98]: failed to create HCP cluster "private-kv-cluster" with private keyvault
Unexpected error:
    <*fmt.wrapError | 0xc000ba1960>: 
    failed waiting for cluster="private-kv-cluster" in resourcegroup="private-keyvault-m2m95r25649h" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/a8a46c46-de01-4ce8-8094-85e125ce1f3d
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/a8a46c46-de01-4ce8-8094-85e125ce1f3d",
      "name": "a8a46c46-de01-4ce8-8094-85e125ce1f3d",
      "status": "Failed",
      "startTime": "2026-08-04T22:01:47.713085545Z",
      "endTime": "2026-08-04T22:20:50.315977298Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift
21:57:53 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.20 · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:121]: HCP cluster rg-candidate-4-20-h76g7r66fxz4/cluster-candidate-4-20-grm2h2 should provision
Unexpected error:
    <*fmt.wrapError | 0xc000ec8d80>: 
    failed to create HCP cluster cluster-candidate-4-20-grm2h2: failed waiting for cluster="cluster-candidate-4-20-grm2h2" in resourcegroup="rg-candidate-4-20-h76g7r66fxz4" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/01d5fb6a-ae17-4667-a58d-f0edf654778a
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/01d5fb6a-ae17-4667-a58d-f0edf654778a",
      "name": "01d5fb6a-ae17-4667-a58d-f0edf654778a",
      "status": "Failed",
      "startTime": "2026-08-04T22:29:58.073640263Z",
      "endTime": "2026-08-04T22:49:01.562239332Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:121]: HCP cluster rg-candidate-4-20-h76g7r66fxz4/cluster-candidate-4-20-grm2h2 should provision
Unexpected error:
    <*fmt.wrapError | 0xc000ec8d80>: 
    failed to create HCP cluster cluster-candidate-4-20-grm2h2: failed waiting for cluster="cluster-candidate-4-20-grm2h2" in resourcegroup="rg-candidate-4-20-h76g7r66fxz4" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/01d5fb6a-ae17-4667-a58d-f0edf654778a
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/01d5fb6a-ae17-4667-a58d-f0edf654778a",
      "name": "01d5fb6a-ae17-4667-a58d-f0edf654778a",
      "status": "Failed",
      "startTime": "2026-08-04T22:29:58.073640263Z",
      "endTime": "2026-08-04T22:49:01.562239332Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel
post-goodmerged PR
No additional run metadata captured
e2e failed n/a 7
e2e (7)
ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift
19:14:09 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.20 · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:121]: HCP cluster rg-candidate-4-20-jk4vnfwfpv5f/cluster-candidate-4-20-ntgzdz should provision
Unexpected error:
    <*fmt.wrapError | 0xc00145e660>: 
    failed to create HCP cluster cluster-candidate-4-20-ntgzdz: failed waiting for cluster="cluster-candidate-4-20-ntgzdz" in resourcegroup="rg-candidate-4-20-jk4vnfwfpv5f" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/e5197554-e486-4b3c-965a-d565b9fd4851
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/e5197554-e486-4b3c-965a-d565b9fd4851",
      "name": "e5197554-e486-4b3c-965a-d565b9fd4851",
      "status": "Failed",
      "startTime": "2026-08-04T19:48:41.598432414Z",
      "endTime": "2026-08-04T20:07:43.143475997Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:121]: HCP cluster rg-candidate-4-20-jk4vnfwfpv5f/cluster-candidate-4-20-ntgzdz should provision
Unexpected error:
    <*fmt.wrapError | 0xc00145e660>: 
    failed to create HCP cluster cluster-candidate-4-20-ntgzdz: failed waiting for cluster="cluster-candidate-4-20-ntgzdz" in resourcegroup="rg-candidate-4-20-jk4vnfwfpv5f" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/e5197554-e486-4b3c-965a-d565b9fd4851
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/e5197554-e486-4b3c-965a-d565b9fd4851",
      "name": "e5197554-e486-4b3c-965a-d565b9fd4851",
      "status": "Failed",
      "startTime": "2026-08-04T19:48:41.598432414Z",
      "endTime": "2026-08-04T20:07:43.143475997Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; [hypershiftHostedCluster] hosted cluster has no installed version; hosted cluster degraded: UnavailableReplicas: [ignition-server-proxy, router]; provider Microsoft.RedHatOpenShift
19:14:09 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.22 · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:121]: HCP cluster rg-candidate-4-22-g5s7df6jfnm8/cluster-candidate-4-22-stc2rb should provision
Unexpected error:
    <*fmt.wrapError | 0xc000e22ee0>: 
    failed to create HCP cluster cluster-candidate-4-22-stc2rb: failed waiting for cluster="cluster-candidate-4-22-stc2rb" in resourcegroup="rg-candidate-4-22-g5s7df6jfnm8" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/b19bfc54-b671-4c19-87b0-81e77f407bf9
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/b19bfc54-b671-4c19-87b0-81e77f407bf9",
      "name": "b19bfc54-b671-4c19-87b0-81e77f407bf9",
      "status": "Failed",
      "startTime": "2026-08-04T19:52:11.833354934Z",
      "endTime": "2026-08-04T20:11:23.470763058Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e; [hypershiftHostedCluster] hosted cluster has no installed version; hosted cluster degraded: UnavailableReplicas: [ignition-server-proxy deployment has 1 unavailable replicas, router deployment has 1 unavailable replicas]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:121]: HCP cluster rg-candidate-4-22-g5s7df6jfnm8/cluster-candidate-4-22-stc2rb should provision
Unexpected error:
    <*fmt.wrapError | 0xc000e22ee0>: 
    failed to create HCP cluster cluster-candidate-4-22-stc2rb: failed waiting for cluster="cluster-candidate-4-22-stc2rb" in resourcegroup="rg-candidate-4-22-g5s7df6jfnm8" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/b19bfc54-b671-4c19-87b0-81e77f407bf9
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/b19bfc54-b671-4c19-87b0-81e77f407bf9",
      "name": "b19bfc54-b671-4c19-87b0-81e77f407bf9",
      "status": "Failed",
      "startTime": "2026-08-04T19:52:11.833354934Z",
      "endTime": "2026-08-04T20:11:23.470763058Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e; [hypershiftHostedCluster] hosted cluster has no installed version; hosted cluster degraded: UnavailableReplicas: [ignition-server-proxy deployment has 1 unavailable replicas, router deployment has 1 unavailable replicas]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; [hypershiftHostedCluster] hosted cluster has no installed version; hosted cluster degraded: UnavailableReplicas: [router]; provider Microsoft.RedHatOpenShift
19:14:09 UTC · failed at e2e · test Customer should create a cluster with private KAS and verify API server is only reachable from VNet while ingress remains public · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:90]: failed to create HCP cluster "private-kas" with private KAS
Unexpected error:
    <*fmt.wrapError | 0xc00113f260>: 
    failed to create HCP cluster private-kas: failed waiting for cluster="private-kas" in resourcegroup="private-kas-w7wdnr2sm6ms" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/c97f1841-2eb1-45a0-96c3-f9289da1a8a1
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/c97f1841-2eb1-45a0-96c3-f9289da1a8a1",
      "name": "c97f1841-2eb1-45a0-96c3-f9289da1a8a1",
      "status": "Failed",
      "startTime": "2026-08-04T19:17:55.492730235Z",
      "endTime": "2026-08-04T19:37:01.946103792Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e; [hypershiftHostedCluster] hosted cluster has no installed version; hosted cluster degraded: UnavailableReplicas: router deployment has 1 unavailable replicas"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:90]: failed to create HCP cluster "private-kas" with private KAS
Unexpected error:
    <*fmt.wrapError | 0xc00113f260>: 
    failed to create HCP cluster private-kas: failed waiting for cluster="private-kas" in resourcegroup="private-kas-w7wdnr2sm6ms" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/c97f1841-2eb1-45a0-96c3-f9289da1a8a1
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/c97f1841-2eb1-45a0-96c3-f9289da1a8a1",
      "name": "c97f1841-2eb1-45a0-96c3-f9289da1a8a1",
      "status": "Failed",
      "startTime": "2026-08-04T19:17:55.492730235Z",
      "endTime": "2026-08-04T19:37:01.946103792Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e; [hypershiftHostedCluster] hosted cluster has no installed version; hosted cluster degraded: UnavailableReplicas: router deployment has 1 unavailable replicas"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift
19:14:09 UTC · failed at e2e · test Customer should be able to create an HCP cluster and manage ImageDigestMirrors · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/idms_lifecycle.go:113]: failed to create HCP cluster with ImageDigestMirrors
Unexpected error:
    <*fmt.wrapError | 0xc000172800>: 
    failed to create HCP cluster idms-e2e-hcp-cluster: failed waiting for cluster="idms-e2e-hcp-cluster" in resourcegroup="idms-pmfmpth4kwpl" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/b2df1752-3761-4a5d-8dcc-3d004ec5ecd5
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/b2df1752-3761-4a5d-8dcc-3d004ec5ecd5",
      "name": "b2df1752-3761-4a5d-8dcc-3d004ec5ecd5",
      "status": "Failed",
      "startTime": "2026-08-04T20:02:24.877745984Z",
      "endTime": "2026-08-04T20:21:24.623836551Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/idms_lifecycle.go:113]: failed to create HCP cluster with ImageDigestMirrors
Unexpected error:
    <*fmt.wrapError | 0xc000172800>: 
    failed to create HCP cluster idms-e2e-hcp-cluster: failed waiting for cluster="idms-e2e-hcp-cluster" in resourcegroup="idms-pmfmpth4kwpl" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/b2df1752-3761-4a5d-8dcc-3d004ec5ecd5
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/b2df1752-3761-4a5d-8dcc-3d004ec5ecd5",
      "name": "b2df1752-3761-4a5d-8dcc-3d004ec5ecd5",
      "status": "Failed",
      "startTime": "2026-08-04T20:02:24.877745984Z",
      "endTime": "2026-08-04T20:21:24.623836551Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift
19:14:09 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.21 · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:121]: HCP cluster rg-candidate-4-21-t2vxvw5248sg/cluster-candidate-4-21-qmnjd2 should provision
Unexpected error:
    <*fmt.wrapError | 0xc000b87000>: 
    failed to create HCP cluster cluster-candidate-4-21-qmnjd2: failed waiting for cluster="cluster-candidate-4-21-qmnjd2" in resourcegroup="rg-candidate-4-21-t2vxvw5248sg" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/8e68bf5a-d57d-4748-9f94-ce3f7ebf5d45
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/8e68bf5a-d57d-4748-9f94-ce3f7ebf5d45",
      "name": "8e68bf5a-d57d-4748-9f94-ce3f7ebf5d45",
      "status": "Failed",
      "startTime": "2026-08-04T19:50:18.955172613Z",
      "endTime": "2026-08-04T20:09:23.207297882Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:121]: HCP cluster rg-candidate-4-21-t2vxvw5248sg/cluster-candidate-4-21-qmnjd2 should provision
Unexpected error:
    <*fmt.wrapError | 0xc000b87000>: 
    failed to create HCP cluster cluster-candidate-4-21-qmnjd2: failed waiting for cluster="cluster-candidate-4-21-qmnjd2" in resourcegroup="rg-candidate-4-21-t2vxvw5248sg" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/8e68bf5a-d57d-4748-9f94-ce3f7ebf5d45
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/8e68bf5a-d57d-4748-9f94-ce3f7ebf5d45",
      "name": "8e68bf5a-d57d-4748-9f94-ce3f7ebf5d45",
      "status": "Failed",
      "startTime": "2026-08-04T19:50:18.955172613Z",
      "endTime": "2026-08-04T20:09:23.207297882Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift
19:14:09 UTC · failed at e2e · test Customer should be able to create a no-CNI private cluster with a private key vault, a nodepool and install cilium CNI successfully · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:105]: failed to create HCP cluster "cilium-cluster" with no CNI and private etcd
Unexpected error:
    <*fmt.wrapError | 0xc000ab7720>: 
    failed waiting for cluster="cilium-cluster" in resourcegroup="complex-cilium-kv-njcvww66sbm2" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/56d59b8d-1979-4aed-99ad-660cbee12b33
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/56d59b8d-1979-4aed-99ad-660cbee12b33",
      "name": "56d59b8d-1979-4aed-99ad-660cbee12b33",
      "status": "Failed",
      "startTime": "2026-08-04T19:18:31.001734969Z",
      "endTime": "2026-08-04T19:37:41.982340399Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:105]: failed to create HCP cluster "cilium-cluster" with no CNI and private etcd
Unexpected error:
    <*fmt.wrapError | 0xc000ab7720>: 
    failed waiting for cluster="cilium-cluster" in resourcegroup="complex-cilium-kv-njcvww66sbm2" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/56d59b8d-1979-4aed-99ad-660cbee12b33
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/56d59b8d-1979-4aed-99ad-660cbee12b33",
      "name": "56d59b8d-1979-4aed-99ad-660cbee12b33",
      "status": "Failed",
      "startTime": "2026-08-04T19:18:31.001734969Z",
      "endTime": "2026-08-04T19:37:41.982340399Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
cluster entered terminal Failed state after role assignment deployment
19:14:09 UTC · failed at e2e · test ARO HCP Service should successfully provision cluster when MI role assignments are added after cluster creation · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:220]: Timed out after 2520.002s.
cluster should eventually succeed after role assignments are created
The function passed to Eventually failed at /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:216 with:
cluster entered terminal Failed state after role assignment deployment
Expected
    <armredhatopenshifthcp.ProvisioningState>: ...
not to equal
    <armredhatopenshifthcp.ProvisioningState>: ...
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:220]: Timed out after 2520.002s.
cluster should eventually succeed after role assignments are created
The function passed to Eventually failed at /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:216 with:
cluster entered terminal Failed state after role assignment deployment
Expected
    <armredhatopenshifthcp.ProvisioningState>: ...
not to equal
    <armredhatopenshifthcp.ProvisioningState>: ...
branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel
post-goodmerged PR
No additional run metadata captured
e2e failed n/a 1
e2e (1)
cluster entered terminal Failed state after role assignment deployment
16:17:49 UTC · failed at e2e · test ARO HCP Service should successfully provision cluster when MI role assignments are added after cluster creation · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:220]: Timed out after 2520.003s.
cluster should eventually succeed after role assignments are created
The function passed to Eventually failed at /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:216 with:
cluster entered terminal Failed state after role assignment deployment
Expected
    <armredhatopenshifthcp.ProvisioningState>: ...
not to equal
    <armredhatopenshifthcp.ProvisioningState>: ...
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:220]: Timed out after 2520.003s.
cluster should eventually succeed after role assignments are created
The function passed to Eventually failed at /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:216 with:
cluster entered terminal Failed state after role assignment deployment
Expected
    <armredhatopenshifthcp.ProvisioningState>: ...
not to equal
    <armredhatopenshifthcp.ProvisioningState>: ...
branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel
post-goodmerged PR
No additional run metadata captured
n/a passed n/a 0 n/a
branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel
post-goodmerged PR
No additional run metadata captured
n/a passed n/a 0 n/a
branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel
post-goodmerged PR
No additional run metadata captured
e2e failed n/a 2
e2e (2)
ERROR CODE: InternalServerError; detail message [hypershiftHostedCluster] hosted cluster has no installed version; hosted cluster degraded: UnavailableReplicas: [router]; provider Microsoft.RedHatOpenShift
08:58:31 UTC · failed at e2e · test Customer should create a cluster with private KAS and verify API server is only reachable from VNet while ingress remains public · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:90]: failed to create HCP cluster "private-kas" with private KAS
Unexpected error:
    <*fmt.wrapError | 0xc000ed50a0>: 
    failed to create HCP cluster private-kas: failed waiting for cluster="private-kas" in resourcegroup="private-kas-g59rzx96ggqs" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/centralindia/hcpOperationStatuses/30166c17-7166-438c-abb0-3f6693531389
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/centralindia/hcpOperationStatuses/30166c17-7166-438c-abb0-3f6693531389",
      "name": "30166c17-7166-438c-abb0-3f6693531389",
      "status": "Failed",
      "startTime": "2026-08-04T09:03:21.796374091Z",
      "endTime": "2026-08-04T09:22:31.515288045Z",
      "error": {
        "code": "InternalServerError",
        "message": "[hypershiftHostedCluster] hosted cluster has no installed version; hosted cluster degraded: UnavailableReplicas: router deployment has 1 unavailable replicas"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:90]: failed to create HCP cluster "private-kas" with private KAS
Unexpected error:
    <*fmt.wrapError | 0xc000ed50a0>: 
    failed to create HCP cluster private-kas: failed waiting for cluster="private-kas" in resourcegroup="private-kas-g59rzx96ggqs" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/centralindia/hcpOperationStatuses/30166c17-7166-438c-abb0-3f6693531389
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/centralindia/hcpOperationStatuses/30166c17-7166-438c-abb0-3f6693531389",
      "name": "30166c17-7166-438c-abb0-3f6693531389",
      "status": "Failed",
      "startTime": "2026-08-04T09:03:21.796374091Z",
      "endTime": "2026-08-04T09:22:31.515288045Z",
      "error": {
        "code": "InternalServerError",
        "message": "[hypershiftHostedCluster] hosted cluster has no installed version; hosted cluster degraded: UnavailableReplicas: router deployment has 1 unavailable replicas"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InternalServerError; detail message [hypershiftHostedCluster] hosted cluster has no installed version; hosted cluster degraded: UnavailableReplicas: [router]; provider Microsoft.RedHatOpenShift
08:58:31 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.22 · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:121]: HCP cluster rg-candidate-4-22-smj79nlv5zgb/cluster-candidate-4-22-7lcplf should provision
Unexpected error:
    <*fmt.wrapError | 0xc000992580>: 
    failed to create HCP cluster cluster-candidate-4-22-7lcplf: failed waiting for cluster="cluster-candidate-4-22-7lcplf" in resourcegroup="rg-candidate-4-22-smj79nlv5zgb" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/centralindia/hcpOperationStatuses/fb43d507-6da9-418d-9328-9479083254a1
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/centralindia/hcpOperationStatuses/fb43d507-6da9-418d-9328-9479083254a1",
      "name": "fb43d507-6da9-418d-9328-9479083254a1",
      "status": "Failed",
      "startTime": "2026-08-04T09:31:16.502576148Z",
      "endTime": "2026-08-04T09:50:23.673134189Z",
      "error": {
        "code": "InternalServerError",
        "message": "[hypershiftHostedCluster] hosted cluster has no installed version; hosted cluster degraded: UnavailableReplicas: router deployment has 1 unavailable replicas"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:121]: HCP cluster rg-candidate-4-22-smj79nlv5zgb/cluster-candidate-4-22-7lcplf should provision
Unexpected error:
    <*fmt.wrapError | 0xc000992580>: 
    failed to create HCP cluster cluster-candidate-4-22-7lcplf: failed waiting for cluster="cluster-candidate-4-22-7lcplf" in resourcegroup="rg-candidate-4-22-smj79nlv5zgb" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/centralindia/hcpOperationStatuses/fb43d507-6da9-418d-9328-9479083254a1
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/centralindia/hcpOperationStatuses/fb43d507-6da9-418d-9328-9479083254a1",
      "name": "fb43d507-6da9-418d-9328-9479083254a1",
      "status": "Failed",
      "startTime": "2026-08-04T09:31:16.502576148Z",
      "endTime": "2026-08-04T09:50:23.673134189Z",
      "error": {
        "code": "InternalServerError",
        "message": "[hypershiftHostedCluster] hosted cluster has no installed version; hosted cluster degraded: UnavailableReplicas: router deployment has 1 unavailable replicas"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel
post-goodmerged PR
No additional run metadata captured
n/a passed n/a 0 n/a
branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel
post-goodmerged PR
No additional run metadata captured
e2e failed n/a 3
e2e (3)
add password after 1 attempts; last attempt error: error status code received from the API; polling error: error status code received from the API
06:20:55 UTC · failed at e2e · test Authorized CIDRs Connectivity should allow API access only from authorized VM IP · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:230]: failed to add password to app registration
Unexpected error:
    <*fmt.wrapErrors | 0xc0017d1410>: 
    add password after 1 attempts; last attempt error: error status code received from the API; polling error: error status code received from the API
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:230]: failed to add password to app registration
Unexpected error:
    <*fmt.wrapErrors | 0xc0017d1410>: 
    add password after 1 attempts; last attempt error: error status code received from the API; polling error: error status code received from the API
    ...
occurred
ERROR CODE: InternalServerError; detail message [clusterServiceNodePoolStatus] <no_message>; provider Microsoft.RedHatOpenShift
06:20:55 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.21 · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:174]: failed to create node pool "np-z96hn6" for cluster "cluster-candidate-4-21-z96hn6"
Unexpected error:
    <*fmt.wrapError | 0xc0012db020>: 
    failed to create NodePool np-z96hn6: failed waiting for nodepool="np-z96hn6" for cluster "cluster-candidate-4-21-z96hn6" in resourcegroup="rg-candidate-4-21-zqtddwbk9hgt" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/centralindia/hcpOperationStatuses/2592eceb-27e3-4e0d-a28c-e989490c2313
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/centralindia/hcpOperationStatuses/2592eceb-27e3-4e0d-a28c-e989490c2313",
      "name": "2592eceb-27e3-4e0d-a28c-e989490c2313",
      "status": "Failed",
      "startTime": "2026-08-04T07:09:05.145723534Z",
      "endTime": "2026-08-04T07:28:07.92030806Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceNodePoolStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:174]: failed to create node pool "np-z96hn6" for cluster "cluster-candidate-4-21-z96hn6"
Unexpected error:
    <*fmt.wrapError | 0xc0012db020>: 
    failed to create NodePool np-z96hn6: failed waiting for nodepool="np-z96hn6" for cluster "cluster-candidate-4-21-z96hn6" in resourcegroup="rg-candidate-4-21-zqtddwbk9hgt" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/centralindia/hcpOperationStatuses/2592eceb-27e3-4e0d-a28c-e989490c2313
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/centralindia/hcpOperationStatuses/2592eceb-27e3-4e0d-a28c-e989490c2313",
      "name": "2592eceb-27e3-4e0d-a28c-e989490c2313",
      "status": "Failed",
      "startTime": "2026-08-04T07:09:05.145723534Z",
      "endTime": "2026-08-04T07:28:07.92030806Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceNodePoolStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InternalServerError; detail message [hypershiftHostedCluster] hosted cluster has no installed version; hosted cluster degraded: UnavailableReplicas: [router]; provider Microsoft.RedHatOpenShift
06:20:55 UTC · failed at e2e · test Customer should create a cluster with private KAS and verify API server is only reachable from VNet while ingress remains public · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:90]: failed to create HCP cluster "private-kas" with private KAS
Unexpected error:
    <*fmt.wrapError | 0xc001081320>: 
    failed to create HCP cluster private-kas: failed waiting for cluster="private-kas" in resourcegroup="private-kas-277sb4kgx5tj" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/centralindia/hcpOperationStatuses/e348c4de-3dfb-43c6-a43c-84605ddf641c
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/centralindia/hcpOperationStatuses/e348c4de-3dfb-43c6-a43c-84605ddf641c",
      "name": "e348c4de-3dfb-43c6-a43c-84605ddf641c",
      "status": "Failed",
      "startTime": "2026-08-04T06:27:13.396537861Z",
      "endTime": "2026-08-04T06:46:16.138209373Z",
      "error": {
        "code": "InternalServerError",
        "message": "[hypershiftHostedCluster] hosted cluster has no installed version; hosted cluster degraded: UnavailableReplicas: router deployment has 1 unavailable replicas"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:90]: failed to create HCP cluster "private-kas" with private KAS
Unexpected error:
    <*fmt.wrapError | 0xc001081320>: 
    failed to create HCP cluster private-kas: failed waiting for cluster="private-kas" in resourcegroup="private-kas-277sb4kgx5tj" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/centralindia/hcpOperationStatuses/e348c4de-3dfb-43c6-a43c-84605ddf641c
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/centralindia/hcpOperationStatuses/e348c4de-3dfb-43c6-a43c-84605ddf641c",
      "name": "e348c4de-3dfb-43c6-a43c-84605ddf641c",
      "status": "Failed",
      "startTime": "2026-08-04T06:27:13.396537861Z",
      "endTime": "2026-08-04T06:46:16.138209373Z",
      "error": {
        "code": "InternalServerError",
        "message": "[hypershiftHostedCluster] hosted cluster has no installed version; hosted cluster degraded: UnavailableReplicas: router deployment has 1 unavailable replicas"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel
post-goodmerged PR
No additional run metadata captured
n/a passed n/a 0 n/a
branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel
post-goodmerged PR
No additional run metadata captured
e2e failed n/a 6
e2e (6)
ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; [hypershiftHostedCluster] hosted cluster is not available: KASLoadBalancerNotReachable: APIServer external route not admitted; provider Microsoft.RedHatOpenShift
03:20:44 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.22 · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:121]: HCP cluster rg-candidate-4-22-8ql88m2bbgwm/cluster-candidate-4-22-wnqnss should provision
Unexpected error:
    <*fmt.wrapError | 0xc000d4a5e0>: 
    failed to create HCP cluster cluster-candidate-4-22-wnqnss: failed waiting for cluster="cluster-candidate-4-22-wnqnss" in resourcegroup="rg-candidate-4-22-8ql88m2bbgwm" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/australiaeast/hcpOperationStatuses/c09527cd-dc9e-4af7-999f-dc3c6c3a5408
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/australiaeast/hcpOperationStatuses/c09527cd-dc9e-4af7-999f-dc3c6c3a5408",
      "name": "c09527cd-dc9e-4af7-999f-dc3c6c3a5408",
      "status": "Failed",
      "startTime": "2026-08-04T04:07:58.639909171Z",
      "endTime": "2026-08-04T04:27:02.537134643Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e; [hypershiftHostedCluster] hosted cluster is not available: KASLoadBalancerNotReachable: APIServer external route not admitted"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:121]: HCP cluster rg-candidate-4-22-8ql88m2bbgwm/cluster-candidate-4-22-wnqnss should provision
Unexpected error:
    <*fmt.wrapError | 0xc000d4a5e0>: 
    failed to create HCP cluster cluster-candidate-4-22-wnqnss: failed waiting for cluster="cluster-candidate-4-22-wnqnss" in resourcegroup="rg-candidate-4-22-8ql88m2bbgwm" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/australiaeast/hcpOperationStatuses/c09527cd-dc9e-4af7-999f-dc3c6c3a5408
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/australiaeast/hcpOperationStatuses/c09527cd-dc9e-4af7-999f-dc3c6c3a5408",
      "name": "c09527cd-dc9e-4af7-999f-dc3c6c3a5408",
      "status": "Failed",
      "startTime": "2026-08-04T04:07:58.639909171Z",
      "endTime": "2026-08-04T04:27:02.537134643Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e; [hypershiftHostedCluster] hosted cluster is not available: KASLoadBalancerNotReachable: APIServer external route not admitted"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift
03:20:44 UTC · failed at e2e · test Customer should be able to create a cluster and node pools with aggregated advanced features · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:184]: failed to create HCP cluster "agg-cluster" with aggregated settings
Unexpected error:
    <*fmt.wrapError | 0xc000f05160>: 
    failed waiting for cluster="agg-cluster" in resourcegroup="feature-aggregation-qgmlhmq9l8px" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/australiaeast/hcpOperationStatuses/33e7ffd7-61c9-4a0a-943f-3a842c29fafd
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/australiaeast/hcpOperationStatuses/33e7ffd7-61c9-4a0a-943f-3a842c29fafd",
      "name": "33e7ffd7-61c9-4a0a-943f-3a842c29fafd",
      "status": "Failed",
      "startTime": "2026-08-04T03:38:56.855932669Z",
      "endTime": "2026-08-04T03:58:00.660059427Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:184]: failed to create HCP cluster "agg-cluster" with aggregated settings
Unexpected error:
    <*fmt.wrapError | 0xc000f05160>: 
    failed waiting for cluster="agg-cluster" in resourcegroup="feature-aggregation-qgmlhmq9l8px" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/australiaeast/hcpOperationStatuses/33e7ffd7-61c9-4a0a-943f-3a842c29fafd
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/australiaeast/hcpOperationStatuses/33e7ffd7-61c9-4a0a-943f-3a842c29fafd",
      "name": "33e7ffd7-61c9-4a0a-943f-3a842c29fafd",
      "status": "Failed",
      "startTime": "2026-08-04T03:38:56.855932669Z",
      "endTime": "2026-08-04T03:58:00.660059427Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
cluster entered terminal Failed state after role assignment deployment
03:20:44 UTC · failed at e2e · test ARO HCP Service should successfully provision cluster when MI role assignments are added after cluster creation · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:220]: Timed out after 2520.044s.
cluster should eventually succeed after role assignments are created
The function passed to Eventually failed at /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:216 with:
cluster entered terminal Failed state after role assignment deployment
Expected
    <armredhatopenshifthcp.ProvisioningState>: ...
not to equal
    <armredhatopenshifthcp.ProvisioningState>: ...
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:220]: Timed out after 2520.044s.
cluster should eventually succeed after role assignments are created
The function passed to Eventually failed at /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:216 with:
cluster entered terminal Failed state after role assignment deployment
Expected
    <armredhatopenshifthcp.ProvisioningState>: ...
not to equal
    <armredhatopenshifthcp.ProvisioningState>: ...
ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift
03:20:44 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.21 · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:121]: HCP cluster rg-candidate-4-21-qnnms78prvlc/cluster-candidate-4-21-tc2trs should provision
Unexpected error:
    <*fmt.wrapError | 0xc000dd8860>: 
    failed to create HCP cluster cluster-candidate-4-21-tc2trs: failed waiting for cluster="cluster-candidate-4-21-tc2trs" in resourcegroup="rg-candidate-4-21-qnnms78prvlc" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/australiaeast/hcpOperationStatuses/7015cf44-0e53-4ac7-964f-dde3a9911749
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/australiaeast/hcpOperationStatuses/7015cf44-0e53-4ac7-964f-dde3a9911749",
      "name": "7015cf44-0e53-4ac7-964f-dde3a9911749",
      "status": "Failed",
      "startTime": "2026-08-04T04:05:17.082597292Z",
      "endTime": "2026-08-04T04:24:22.4244114Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:121]: HCP cluster rg-candidate-4-21-qnnms78prvlc/cluster-candidate-4-21-tc2trs should provision
Unexpected error:
    <*fmt.wrapError | 0xc000dd8860>: 
    failed to create HCP cluster cluster-candidate-4-21-tc2trs: failed waiting for cluster="cluster-candidate-4-21-tc2trs" in resourcegroup="rg-candidate-4-21-qnnms78prvlc" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/australiaeast/hcpOperationStatuses/7015cf44-0e53-4ac7-964f-dde3a9911749
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/australiaeast/hcpOperationStatuses/7015cf44-0e53-4ac7-964f-dde3a9911749",
      "name": "7015cf44-0e53-4ac7-964f-dde3a9911749",
      "status": "Failed",
      "startTime": "2026-08-04T04:05:17.082597292Z",
      "endTime": "2026-08-04T04:24:22.4244114Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
tls: failed to verify certificate: x509: certificate signed by unknown authority
03:20:44 UTC · failed at e2e · test Customer should be able to create an HCP cluster with custom autoscaling and update the autoscaling configuration · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:203]: failed to verify cluster workloads and pods after autoscaling update
Unexpected error:
    <*errors.joinError | 0xc002814a50>: 
    VerifySimpleWebApp failed: [strict TLS verification] route was never reachable: Get "https://agnhost-e2e-sample-app-fpcgc.apps.aro.l6b6k4n3r7r3o9g.s2yb.australiaeast.aroapp-hcp.io": tls: failed to verify certificate: x509: certificate signed by unknown authority
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:203]: failed to verify cluster workloads and pods after autoscaling update
Unexpected error:
    <*errors.joinError | 0xc002814a50>: 
    VerifySimpleWebApp failed: [strict TLS verification] route was never reachable: Get "https://agnhost-e2e-sample-app-fpcgc.apps.aro.l6b6k4n3r7r3o9g.s2yb.australiaeast.aroapp-hcp.io": tls: failed to verify certificate: x509: certificate signed by unknown authority
    ...
occurred
ERROR CODE: InternalServerError; detail message [clusterServiceClusterStatus] <no_message>; provider Microsoft.RedHatOpenShift
03:20:44 UTC · failed at e2e · test Customer should be able to create a no-CNI private cluster with a private key vault, a nodepool and install cilium CNI successfully · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:105]: failed to create HCP cluster "cilium-cluster" with no CNI and private etcd
Unexpected error:
    <*fmt.wrapError | 0xc001540920>: 
    failed waiting for cluster="cilium-cluster" in resourcegroup="complex-cilium-kv-2dhr7ccczc27" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/australiaeast/hcpOperationStatuses/90778dc2-b431-4db7-ab53-7a6d058e9a4e
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/australiaeast/hcpOperationStatuses/90778dc2-b431-4db7-ab53-7a6d058e9a4e",
      "name": "90778dc2-b431-4db7-ab53-7a6d058e9a4e",
      "status": "Failed",
      "startTime": "2026-08-04T03:38:01.456556074Z",
      "endTime": "2026-08-04T03:57:20.419366468Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:105]: failed to create HCP cluster "cilium-cluster" with no CNI and private etcd
Unexpected error:
    <*fmt.wrapError | 0xc001540920>: 
    failed waiting for cluster="cilium-cluster" in resourcegroup="complex-cilium-kv-2dhr7ccczc27" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/australiaeast/hcpOperationStatuses/90778dc2-b431-4db7-ab53-7a6d058e9a4e
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/australiaeast/hcpOperationStatuses/90778dc2-b431-4db7-ab53-7a6d058e9a4e",
      "name": "90778dc2-b431-4db7-ab53-7a6d058e9a4e",
      "status": "Failed",
      "startTime": "2026-08-04T03:38:01.456556074Z",
      "endTime": "2026-08-04T03:57:20.419366468Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceClusterStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel
post-goodmerged PR
No additional run metadata captured
e2e failed n/a 3
e2e (3)
ERROR CODE: InternalServerError; detail message [hypershiftHostedCluster] hosted cluster has no installed version; hosted cluster degraded: UnavailableReplicas: [router]; provider Microsoft.RedHatOpenShift
03:12:33 UTC · failed at e2e · test Customer should create a cluster with private KAS and verify API server is only reachable from VNet while ingress remains public · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:90]: failed to create HCP cluster "private-kas" with private KAS
Unexpected error:
    <*fmt.wrapError | 0xc000e26a40>: 
    failed to create HCP cluster private-kas: failed waiting for cluster="private-kas" in resourcegroup="private-kas-wptdp8hlw57n" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/centralindia/hcpOperationStatuses/dc23222c-5587-4fff-9c67-80d77c47a7c2
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/centralindia/hcpOperationStatuses/dc23222c-5587-4fff-9c67-80d77c47a7c2",
      "name": "dc23222c-5587-4fff-9c67-80d77c47a7c2",
      "status": "Failed",
      "startTime": "2026-08-04T03:31:08.362922752Z",
      "endTime": "2026-08-04T03:50:09.128809967Z",
      "error": {
        "code": "InternalServerError",
        "message": "[hypershiftHostedCluster] hosted cluster has no installed version; hosted cluster degraded: UnavailableReplicas: router deployment has 1 unavailable replicas"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:90]: failed to create HCP cluster "private-kas" with private KAS
Unexpected error:
    <*fmt.wrapError | 0xc000e26a40>: 
    failed to create HCP cluster private-kas: failed waiting for cluster="private-kas" in resourcegroup="private-kas-wptdp8hlw57n" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/centralindia/hcpOperationStatuses/dc23222c-5587-4fff-9c67-80d77c47a7c2
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/centralindia/hcpOperationStatuses/dc23222c-5587-4fff-9c67-80d77c47a7c2",
      "name": "dc23222c-5587-4fff-9c67-80d77c47a7c2",
      "status": "Failed",
      "startTime": "2026-08-04T03:31:08.362922752Z",
      "endTime": "2026-08-04T03:50:09.128809967Z",
      "error": {
        "code": "InternalServerError",
        "message": "[hypershiftHostedCluster] hosted cluster has no installed version; hosted cluster degraded: UnavailableReplicas: router deployment has 1 unavailable replicas"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InternalServerError; detail message [hypershiftHostedCluster] hosted cluster has no installed version; hosted cluster degraded: UnavailableReplicas: [router]; provider Microsoft.RedHatOpenShift
03:12:33 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.22 · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:121]: HCP cluster rg-candidate-4-22-w4pnnkd4rqn5/cluster-candidate-4-22-ntkxvm should provision
Unexpected error:
    <*fmt.wrapError | 0xc000f9c3c0>: 
    failed to create HCP cluster cluster-candidate-4-22-ntkxvm: failed waiting for cluster="cluster-candidate-4-22-ntkxvm" in resourcegroup="rg-candidate-4-22-w4pnnkd4rqn5" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/centralindia/hcpOperationStatuses/70d9bd23-9091-4a38-9408-3e141e457996
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/centralindia/hcpOperationStatuses/70d9bd23-9091-4a38-9408-3e141e457996",
      "name": "70d9bd23-9091-4a38-9408-3e141e457996",
      "status": "Failed",
      "startTime": "2026-08-04T03:59:40.57100407Z",
      "endTime": "2026-08-04T04:18:40.033619916Z",
      "error": {
        "code": "InternalServerError",
        "message": "[hypershiftHostedCluster] hosted cluster has no installed version; hosted cluster degraded: UnavailableReplicas: router deployment has 1 unavailable replicas"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:121]: HCP cluster rg-candidate-4-22-w4pnnkd4rqn5/cluster-candidate-4-22-ntkxvm should provision
Unexpected error:
    <*fmt.wrapError | 0xc000f9c3c0>: 
    failed to create HCP cluster cluster-candidate-4-22-ntkxvm: failed waiting for cluster="cluster-candidate-4-22-ntkxvm" in resourcegroup="rg-candidate-4-22-w4pnnkd4rqn5" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/centralindia/hcpOperationStatuses/70d9bd23-9091-4a38-9408-3e141e457996
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/centralindia/hcpOperationStatuses/70d9bd23-9091-4a38-9408-3e141e457996",
      "name": "70d9bd23-9091-4a38-9408-3e141e457996",
      "status": "Failed",
      "startTime": "2026-08-04T03:59:40.57100407Z",
      "endTime": "2026-08-04T04:18:40.033619916Z",
      "error": {
        "code": "InternalServerError",
        "message": "[hypershiftHostedCluster] hosted cluster has no installed version; hosted cluster degraded: UnavailableReplicas: router deployment has 1 unavailable replicas"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
cluster entered terminal Failed state after role assignment deployment
03:12:33 UTC · failed at e2e · test ARO HCP Service should successfully provision cluster when MI role assignments are added after cluster creation · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:220]: Timed out after 2520.003s.
cluster should eventually succeed after role assignments are created
The function passed to Eventually failed at /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:216 with:
cluster entered terminal Failed state after role assignment deployment
Expected
    <armredhatopenshifthcp.ProvisioningState>: ...
not to equal
    <armredhatopenshifthcp.ProvisioningState>: ...
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:220]: Timed out after 2520.003s.
cluster should eventually succeed after role assignments are created
The function passed to Eventually failed at /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:216 with:
cluster entered terminal Failed state after role assignment deployment
Expected
    <armredhatopenshifthcp.ProvisioningState>: ...
not to equal
    <armredhatopenshifthcp.ProvisioningState>: ...
branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel
post-goodmerged PR
No additional run metadata captured
n/a passed n/a 0 n/a
branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel
post-goodmerged PR
No additional run metadata captured
n/a passed n/a 0 n/a
periodic-ci-Azure-ARO-HCP-main-periodic-prod-e2e-parallel
post-goodmerged PR
No additional run metadata captured
e2e failed n/a 44
e2e (44)
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Customer should not be able to perform various invalid operations on cluster resources · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/simple_negative_cases.go:110]: failed to create HCP cluster negative-tests-cluster
Unexpected error:
    <*fmt.wrapError | 0xc001011e80>: 
    failed to create HCP cluster negative-tests-cluster: failed starting cluster creation "negative-tests-cluster" in resourcegroup="rg-negative-tests-5nvgptdxjswx": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-negative-tests-5nvgptdxjswx/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/negative-tests-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/simple_negative_cases.go:110]: failed to create HCP cluster negative-tests-cluster
Unexpected error:
    <*fmt.wrapError | 0xc001011e80>: 
    failed to create HCP cluster negative-tests-cluster: failed starting cluster creation "negative-tests-cluster" in resourcegroup="rg-negative-tests-5nvgptdxjswx": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-negative-tests-5nvgptdxjswx/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/negative-tests-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Customer should be able to test admin credentials before cluster ready, then full admin credential lifecycle · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:105]: failed to begin creating HCP cluster "admin-cred-lifecycle-lxppll"
Unexpected error:
    <*fmt.wrapError | 0xc000202820>: 
    failed starting cluster creation "admin-cred-lifecycle-lxppll" in resourcegroup="admin-credential-lifecycle-test-9msk4cmfl8w2": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/admin-credential-lifecycle-test-9msk4cmfl8w2/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/admin-cred-lifecycle-lxppll
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:105]: failed to begin creating HCP cluster "admin-cred-lifecycle-lxppll"
Unexpected error:
    <*fmt.wrapError | 0xc000202820>: 
    failed starting cluster creation "admin-cred-lifecycle-lxppll" in resourcegroup="admin-credential-lifecycle-test-9msk4cmfl8w2": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/admin-credential-lifecycle-test-9msk4cmfl8w2/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/admin-cred-lifecycle-lxppll
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Customer should be able to create a no-CNI private cluster with a private key vault, a nodepool and install cilium CNI successfully · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:105]: failed to create HCP cluster "cilium-cluster" with no CNI and private etcd
Unexpected error:
    <*fmt.wrapError | 0xc000f88560>: 
    failed starting cluster creation "cilium-cluster" in resourcegroup="complex-cilium-kv-k9kz8gd7gppv": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/complex-cilium-kv-k9kz8gd7gppv/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cilium-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:105]: failed to create HCP cluster "cilium-cluster" with no CNI and private etcd
Unexpected error:
    <*fmt.wrapError | 0xc000f88560>: 
    failed starting cluster creation "cilium-cluster" in resourcegroup="complex-cilium-kv-k9kz8gd7gppv": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/complex-cilium-kv-k9kz8gd7gppv/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cilium-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Customer should upgrade and update a nodepool from 4.20.z to 4.21.zLatest · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:133]: failed to create HCP cluster np-version-upgrade-cluster-jrs8np with version 4.21.23
Unexpected error:
    <*fmt.wrapError | 0xc000f08c00>: 
    failed to create HCP cluster np-version-upgrade-cluster-jrs8np: failed starting cluster creation "np-version-upgrade-cluster-jrs8np" in resourcegroup="rg-np-version-upgrade-jrs8np-98wgsmqjvj8l": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-version-upgrade-jrs8np-98wgsmqjvj8l/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-version-upgrade-cluster-jrs8np
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:133]: failed to create HCP cluster np-version-upgrade-cluster-jrs8np with version 4.21.23
Unexpected error:
    <*fmt.wrapError | 0xc000f08c00>: 
    failed to create HCP cluster np-version-upgrade-cluster-jrs8np: failed starting cluster creation "np-version-upgrade-cluster-jrs8np" in resourcegroup="rg-np-version-upgrade-jrs8np-98wgsmqjvj8l": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-version-upgrade-jrs8np-98wgsmqjvj8l/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-version-upgrade-cluster-jrs8np
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Customer should be able to create an HCP cluster and custom node pool osDisk size · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_nodepool_osdisk.go:80]: failed to create HCP cluster "hcp-cluster-np-128"
Unexpected error:
    <*fmt.wrapError | 0xc000e82a00>: 
    failed to create HCP cluster hcp-cluster-np-128: failed starting cluster creation "hcp-cluster-np-128" in resourcegroup="clusternp128-mbs2gd2vx29j": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/clusternp128-mbs2gd2vx29j/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/hcp-cluster-np-128
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_nodepool_osdisk.go:80]: failed to create HCP cluster "hcp-cluster-np-128"
Unexpected error:
    <*fmt.wrapError | 0xc000e82a00>: 
    failed to create HCP cluster hcp-cluster-np-128: failed starting cluster creation "hcp-cluster-np-128" in resourcegroup="clusternp128-mbs2gd2vx29j": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/clusternp128-mbs2gd2vx29j/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/hcp-cluster-np-128
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Customer should be able to create a cluster and node pools with aggregated advanced features · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:184]: failed to create HCP cluster "agg-cluster" with aggregated settings
Unexpected error:
    <*fmt.wrapError | 0xc001029020>: 
    failed starting cluster creation "agg-cluster" in resourcegroup="feature-aggregation-rnlm6flzrgmd": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/feature-aggregation-rnlm6flzrgmd/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/agg-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:184]: failed to create HCP cluster "agg-cluster" with aggregated settings
Unexpected error:
    <*fmt.wrapError | 0xc001029020>: 
    failed starting cluster creation "agg-cluster" in resourcegroup="feature-aggregation-rnlm6flzrgmd": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/feature-aggregation-rnlm6flzrgmd/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/agg-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Customer should upgrade a nodepool to a version without Cincinnati upgrade edge z-stream upgrade without Cincinnati edge in 4.20 · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:314]: failed to create HCP cluster np-noedge-fj6r9s
Unexpected error:
    <*fmt.wrapError | 0xc000a9bf20>: 
    failed to create HCP cluster np-noedge-fj6r9s: failed starting cluster creation "np-noedge-fj6r9s" in resourcegroup="rg-np-noedge-fj6r9s-lqfrccz2bsz6": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-noedge-fj6r9s-lqfrccz2bsz6/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-noedge-fj6r9s
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:314]: failed to create HCP cluster np-noedge-fj6r9s
Unexpected error:
    <*fmt.wrapError | 0xc000a9bf20>: 
    failed to create HCP cluster np-noedge-fj6r9s: failed starting cluster creation "np-noedge-fj6r9s" in resourcegroup="rg-np-noedge-fj6r9s-lqfrccz2bsz6": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-noedge-fj6r9s-lqfrccz2bsz6/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-noedge-fj6r9s
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Create HCPOpenShiftCluster with Private KeyVault should create a cluster with private keyvault using v20251223preview API · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:98]: failed to create HCP cluster "private-kv-cluster" with private keyvault
Unexpected error:
    <*fmt.wrapError | 0xc000ea1200>: 
    failed starting cluster creation "private-kv-cluster" in resourcegroup="private-keyvault-j2b66h2fdbdt": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/private-keyvault-j2b66h2fdbdt/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/private-kv-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:98]: failed to create HCP cluster "private-kv-cluster" with private keyvault
Unexpected error:
    <*fmt.wrapError | 0xc000ea1200>: 
    failed starting cluster creation "private-kv-cluster" in resourcegroup="private-keyvault-j2b66h2fdbdt": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/private-keyvault-j2b66h2fdbdt/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/private-kv-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.21 · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:121]: HCP cluster rg-candidate-4-21-gjxx6n85vw95/cluster-candidate-4-21-cnn7h9 should provision
Unexpected error:
    <*fmt.wrapError | 0xc0015905c0>: 
    failed to create HCP cluster cluster-candidate-4-21-cnn7h9: failed starting cluster creation "cluster-candidate-4-21-cnn7h9" in resourcegroup="rg-candidate-4-21-gjxx6n85vw95": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-4-21-gjxx6n85vw95/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-candidate-4-21-cnn7h9
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:121]: HCP cluster rg-candidate-4-21-gjxx6n85vw95/cluster-candidate-4-21-cnn7h9 should provision
Unexpected error:
    <*fmt.wrapError | 0xc0015905c0>: 
    failed to create HCP cluster cluster-candidate-4-21-cnn7h9: failed starting cluster creation "cluster-candidate-4-21-cnn7h9" in resourcegroup="rg-candidate-4-21-gjxx6n85vw95": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-4-21-gjxx6n85vw95/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-candidate-4-21-cnn7h9
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Customer should be able to lifecycle and confirm external auth on a cluster · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_list_and_verify.go:87]: failed to create HCP cluster for external auth list test
Unexpected error:
    <*fmt.wrapError | 0xc000f8d3c0>: 
    failed to create HCP cluster ea-list76nllp: failed starting cluster creation "ea-list76nllp" in resourcegroup="ea-list-99624jpmp4tc": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/ea-list-99624jpmp4tc/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/ea-list76nllp
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_list_and_verify.go:87]: failed to create HCP cluster for external auth list test
Unexpected error:
    <*fmt.wrapError | 0xc000f8d3c0>: 
    failed to create HCP cluster ea-list76nllp: failed starting cluster creation "ea-list76nllp" in resourcegroup="ea-list-99624jpmp4tc": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/ea-list-99624jpmp4tc/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/ea-list76nllp
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Customer should be able to create an HCP cluster with custom autoscaling and update the autoscaling configuration · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:108]: failed to create HCP cluster "autoscaling-hcp-cluster" with custom autoscaling
Unexpected error:
    <*fmt.wrapError | 0xc0008fba80>: 
    failed to create HCP cluster autoscaling-hcp-cluster: failed starting cluster creation "autoscaling-hcp-cluster" in resourcegroup="autoscaling-cluster-44968wjw2szt": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/autoscaling-cluster-44968wjw2szt/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/autoscaling-hcp-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:108]: failed to create HCP cluster "autoscaling-hcp-cluster" with custom autoscaling
Unexpected error:
    <*fmt.wrapError | 0xc0008fba80>: 
    failed to create HCP cluster autoscaling-hcp-cluster: failed starting cluster creation "autoscaling-hcp-cluster" in resourcegroup="autoscaling-cluster-44968wjw2szt": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/autoscaling-cluster-44968wjw2szt/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/autoscaling-hcp-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Customer should be able to successfully upgrade control plane minor version from 4.20 minor to 4.21 minor · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:182]: failed to create HCP cluster "cp-ystream-upgrade-4-21-6464lp" with install version 4.20.28
Unexpected error:
    <*fmt.wrapError | 0xc00103fe00>: 
    failed to create HCP cluster cp-ystream-upgrade-4-21-6464lp: failed starting cluster creation "cp-ystream-upgrade-4-21-6464lp" in resourcegroup="rg-cp-ystream-upgrade-4-21-5mtd7n4pmsml": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cp-ystream-upgrade-4-21-5mtd7n4pmsml/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cp-ystream-upgrade-4-21-6464lp
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:182]: failed to create HCP cluster "cp-ystream-upgrade-4-21-6464lp" with install version 4.20.28
Unexpected error:
    <*fmt.wrapError | 0xc00103fe00>: 
    failed to create HCP cluster cp-ystream-upgrade-4-21-6464lp: failed starting cluster creation "cp-ystream-upgrade-4-21-6464lp" in resourcegroup="rg-cp-ystream-upgrade-4-21-5mtd7n4pmsml": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cp-ystream-upgrade-4-21-5mtd7n4pmsml/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cp-ystream-upgrade-4-21-6464lp
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Customer should downgrade a nodepool version z-stream downgrade from 4.21.zLatest to 4.21.zPrevious · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:593]: failed to create HCP cluster
Unexpected error:
    <*fmt.wrapError | 0xc0015106c0>: 
    failed to create HCP cluster np-downgrade-jd6hzz: failed starting cluster creation "np-downgrade-jd6hzz" in resourcegroup="rg-np-downgrade-jd6hzz-glphtjlrc7rc": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-downgrade-jd6hzz-glphtjlrc7rc/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-downgrade-jd6hzz
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:593]: failed to create HCP cluster
Unexpected error:
    <*fmt.wrapError | 0xc0015106c0>: 
    failed to create HCP cluster np-downgrade-jd6hzz: failed starting cluster creation "np-downgrade-jd6hzz" in resourcegroup="rg-np-downgrade-jd6hzz-glphtjlrc7rc": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-downgrade-jd6hzz-glphtjlrc7rc/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-downgrade-jd6hzz
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.20 · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:121]: HCP cluster rg-candidate-4-20-mq5tl24kw9mh/cluster-candidate-4-20-7jgdzv should provision
Unexpected error:
    <*fmt.wrapError | 0xc0009fe860>: 
    failed to create HCP cluster cluster-candidate-4-20-7jgdzv: failed starting cluster creation "cluster-candidate-4-20-7jgdzv" in resourcegroup="rg-candidate-4-20-mq5tl24kw9mh": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-4-20-mq5tl24kw9mh/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-candidate-4-20-7jgdzv
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:121]: HCP cluster rg-candidate-4-20-mq5tl24kw9mh/cluster-candidate-4-20-7jgdzv should provision
Unexpected error:
    <*fmt.wrapError | 0xc0009fe860>: 
    failed to create HCP cluster cluster-candidate-4-20-7jgdzv: failed starting cluster creation "cluster-candidate-4-20-7jgdzv" in resourcegroup="rg-candidate-4-20-mq5tl24kw9mh": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-4-20-mq5tl24kw9mh/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-candidate-4-20-7jgdzv
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Customer should be able to create an HCP cluster and manage ImageDigestMirrors · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/idms_lifecycle.go:113]: failed to create HCP cluster with ImageDigestMirrors
Unexpected error:
    <*fmt.wrapError | 0xc000e6a200>: 
    failed to create HCP cluster idms-e2e-hcp-cluster: failed starting cluster creation "idms-e2e-hcp-cluster" in resourcegroup="idms-5cmqzqkhfw4l": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/idms-5cmqzqkhfw4l/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/idms-e2e-hcp-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/idms_lifecycle.go:113]: failed to create HCP cluster with ImageDigestMirrors
Unexpected error:
    <*fmt.wrapError | 0xc000e6a200>: 
    failed to create HCP cluster idms-e2e-hcp-cluster: failed starting cluster creation "idms-e2e-hcp-cluster" in resourcegroup="idms-5cmqzqkhfw4l": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/idms-5cmqzqkhfw4l/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/idms-e2e-hcp-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test ARO HCP Service should successfully provision cluster when MI role assignments are added after cluster creation · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:133]: failed to start cluster creation
Unexpected error:
    <*exported.ResponseError | 0xc0012d2de0>: 
    PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/delayed-rbac-cdbn2wfnd9dq/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/delayed-rbac-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:133]: failed to start cluster creation
Unexpected error:
    <*exported.ResponseError | 0xc0012d2de0>: 
    PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/delayed-rbac-cdbn2wfnd9dq/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/delayed-rbac-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Customer should be able to create an HCP cluster with Image Registry not present · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/image_registry_cluster_create.go:89]: failed to create HCP cluster with image registry disabled
Unexpected error:
    <*fmt.wrapError | 0xc000e23240>: 
    failed to create HCP cluster disabled-image-registry-hcp-cluster: failed starting cluster creation "disabled-image-registry-hcp-cluster" in resourcegroup="disabled-image-registry-dzj6kgcsq5t9": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/disabled-image-registry-dzj6kgcsq5t9/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/disabled-image-registry-hcp-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/image_registry_cluster_create.go:89]: failed to create HCP cluster with image registry disabled
Unexpected error:
    <*fmt.wrapError | 0xc000e23240>: 
    failed to create HCP cluster disabled-image-registry-hcp-cluster: failed starting cluster creation "disabled-image-registry-hcp-cluster" in resourcegroup="disabled-image-registry-dzj6kgcsq5t9": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/disabled-image-registry-dzj6kgcsq5t9/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/disabled-image-registry-hcp-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Customer should downgrade a nodepool to a lower minor version from 4.21.zLatest to 4.19.zLatest · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:741]: failed to create HCP cluster
Unexpected error:
    <*fmt.wrapError | 0xc000f662a0>: 
    failed to create HCP cluster np-dg-minor-dsw5kp: failed starting cluster creation "np-dg-minor-dsw5kp" in resourcegroup="rg-np-dg-minor-dsw5kp-6889hg6sxmp8": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-dg-minor-dsw5kp-6889hg6sxmp8/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-dg-minor-dsw5kp
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:741]: failed to create HCP cluster
Unexpected error:
    <*fmt.wrapError | 0xc000f662a0>: 
    failed to create HCP cluster np-dg-minor-dsw5kp: failed starting cluster creation "np-dg-minor-dsw5kp" in resourcegroup="rg-np-dg-minor-dsw5kp-6889hg6sxmp8": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-dg-minor-dsw5kp-6889hg6sxmp8/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-dg-minor-dsw5kp
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Customer should respect cluster-wide node limits with nodepool autoscaling · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:291]: failed to create HCP cluster node-limit-cluster with MaxNodesTotal limit
Unexpected error:
    <*fmt.wrapError | 0xc000a8aa60>: 
    failed to create HCP cluster node-limit-cluster: failed starting cluster creation "node-limit-cluster" in resourcegroup="node-limit-test-t979dvq7zl86": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/node-limit-test-t979dvq7zl86/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/node-limit-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:291]: failed to create HCP cluster node-limit-cluster with MaxNodesTotal limit
Unexpected error:
    <*fmt.wrapError | 0xc000a8aa60>: 
    failed to create HCP cluster node-limit-cluster: failed starting cluster creation "node-limit-cluster" in resourcegroup="node-limit-test-t979dvq7zl86": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/node-limit-test-t979dvq7zl86/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/node-limit-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Customer should be able to create node pools with ARM64-based VMs · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/arm64_nodepool.go:88]: failed to create HCP cluster "arm64-vm-hcp-cluster"
Unexpected error:
    <*fmt.wrapError | 0xc00151f820>: 
    failed to create HCP cluster arm64-vm-hcp-cluster: failed starting cluster creation "arm64-vm-hcp-cluster" in resourcegroup="arm64-vm-cluster-8v6fnr7zpjcz": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/arm64-vm-cluster-8v6fnr7zpjcz/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/arm64-vm-hcp-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/arm64_nodepool.go:88]: failed to create HCP cluster "arm64-vm-hcp-cluster"
Unexpected error:
    <*fmt.wrapError | 0xc00151f820>: 
    failed to create HCP cluster arm64-vm-hcp-cluster: failed starting cluster creation "arm64-vm-hcp-cluster" in resourcegroup="arm64-vm-cluster-8v6fnr7zpjcz": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/arm64-vm-cluster-8v6fnr7zpjcz/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/arm64-vm-hcp-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 4.20 · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:110]: failed to create HCP cluster "cluster-zstream-4-20-hgprgx" with version 4.20.27 on candidate channel
Unexpected error:
    <*fmt.wrapError | 0xc000b882e0>: 
    failed to create HCP cluster cluster-zstream-4-20-hgprgx: failed starting cluster creation "cluster-zstream-4-20-hgprgx" in resourcegroup="rg-zstream-upgrade-4-20-lxn6c4htx824": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-20-lxn6c4htx824/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-zstream-4-20-hgprgx
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:110]: failed to create HCP cluster "cluster-zstream-4-20-hgprgx" with version 4.20.27 on candidate channel
Unexpected error:
    <*fmt.wrapError | 0xc000b882e0>: 
    failed to create HCP cluster cluster-zstream-4-20-hgprgx: failed starting cluster creation "cluster-zstream-4-20-hgprgx" in resourcegroup="rg-zstream-upgrade-4-20-lxn6c4htx824": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-20-lxn6c4htx824/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-zstream-4-20-hgprgx
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Customer should be able to create a HCP cluster and use cilium CNI plugin · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_cni_cilium.go:88]: failed to create HCP cluster "cilium-cl" without CNI
Unexpected error:
    <*fmt.wrapError | 0xc000880180>: 
    failed to create HCP cluster cilium-cl: failed starting cluster creation "cilium-cl" in resourcegroup="cni-cilium-7rc2g5595zqt": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/cni-cilium-7rc2g5595zqt/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cilium-cl
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_cni_cilium.go:88]: failed to create HCP cluster "cilium-cl" without CNI
Unexpected error:
    <*fmt.wrapError | 0xc000880180>: 
    failed to create HCP cluster cilium-cl: failed starting cluster creation "cilium-cl" in resourcegroup="cni-cilium-7rc2g5595zqt": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/cni-cilium-7rc2g5595zqt/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cilium-cl
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 4.21 · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:110]: failed to create HCP cluster "cluster-zstream-4-21-hpm766" with version 4.21.22 on candidate channel
Unexpected error:
    <*fmt.wrapError | 0xc000941ee0>: 
    failed to create HCP cluster cluster-zstream-4-21-hpm766: failed starting cluster creation "cluster-zstream-4-21-hpm766" in resourcegroup="rg-zstream-upgrade-4-21-7v5b8gnf84kb": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-21-7v5b8gnf84kb/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-zstream-4-21-hpm766
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:110]: failed to create HCP cluster "cluster-zstream-4-21-hpm766" with version 4.21.22 on candidate channel
Unexpected error:
    <*fmt.wrapError | 0xc000941ee0>: 
    failed to create HCP cluster cluster-zstream-4-21-hpm766: failed starting cluster creation "cluster-zstream-4-21-hpm766" in resourcegroup="rg-zstream-upgrade-4-21-7v5b8gnf84kb": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-21-7v5b8gnf84kb/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-zstream-4-21-hpm766
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Customer should create an HCP cluster and validate TLS certificates · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:95]: failed to create HCP cluster for TLS endpoint test
Unexpected error:
    <*fmt.wrapError | 0xc00080fd80>: 
    failed to create HCP cluster tls-endpoint-hcp-cluster: failed starting cluster creation "tls-endpoint-hcp-cluster" in resourcegroup="tls-endpoint-cluster-cw4bgwwmp7js": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/tls-endpoint-cluster-cw4bgwwmp7js/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/tls-endpoint-hcp-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:95]: failed to create HCP cluster for TLS endpoint test
Unexpected error:
    <*fmt.wrapError | 0xc00080fd80>: 
    failed to create HCP cluster tls-endpoint-hcp-cluster: failed starting cluster creation "tls-endpoint-hcp-cluster" in resourcegroup="tls-endpoint-cluster-cw4bgwwmp7js": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/tls-endpoint-cluster-cw4bgwwmp7js/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/tls-endpoint-hcp-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Customer should be able to create several HCP clusters in their customer resource group, but not in the same managed resource group · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:111]: failed to begin creation of first cluster "basic-hcp-cluster"
Unexpected error:
    <*fmt.wrapError | 0xc000ebcae0>: 
    failed starting cluster creation "basic-hcp-cluster" in resourcegroup="customer-rg-94ndpq9l2n2f": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/customer-rg-94ndpq9l2n2f/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/basic-hcp-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:111]: failed to begin creation of first cluster "basic-hcp-cluster"
Unexpected error:
    <*fmt.wrapError | 0xc000ebcae0>: 
    failed starting cluster creation "basic-hcp-cluster" in resourcegroup="customer-rg-94ndpq9l2n2f": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/customer-rg-94ndpq9l2n2f/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/basic-hcp-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 4.19 · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:110]: failed to create HCP cluster "cluster-zstream-4-19-5pjpdp" with version 4.19.39 on candidate channel
Unexpected error:
    <*fmt.wrapError | 0xc001012e20>: 
    failed to create HCP cluster cluster-zstream-4-19-5pjpdp: failed starting cluster creation "cluster-zstream-4-19-5pjpdp" in resourcegroup="rg-zstream-upgrade-4-19-k9wjch5mhx6h": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-19-k9wjch5mhx6h/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-zstream-4-19-5pjpdp
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:110]: failed to create HCP cluster "cluster-zstream-4-19-5pjpdp" with version 4.19.39 on candidate channel
Unexpected error:
    <*fmt.wrapError | 0xc001012e20>: 
    failed to create HCP cluster cluster-zstream-4-19-5pjpdp: failed starting cluster creation "cluster-zstream-4-19-5pjpdp" in resourcegroup="rg-zstream-upgrade-4-19-k9wjch5mhx6h": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-19-k9wjch5mhx6h/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-zstream-4-19-5pjpdp
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Customer should not be able to reuse subnets and NSGs between clusters · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_nsg_subnet_reuse.go:107]: failed to create seed HCP cluster basic-cluster
Unexpected error:
    <*fmt.wrapError | 0xc001507bc0>: 
    failed to create HCP cluster basic-cluster: failed starting cluster creation "basic-cluster" in resourcegroup="rg-cluster-nsg-subnet-reuse-nk96qwjnjfw8": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cluster-nsg-subnet-reuse-nk96qwjnjfw8/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/basic-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_nsg_subnet_reuse.go:107]: failed to create seed HCP cluster basic-cluster
Unexpected error:
    <*fmt.wrapError | 0xc001507bc0>: 
    failed to create HCP cluster basic-cluster: failed starting cluster creation "basic-cluster" in resourcegroup="rg-cluster-nsg-subnet-reuse-nk96qwjnjfw8": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cluster-nsg-subnet-reuse-nk96qwjnjfw8/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/basic-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Customer should upgrade and update a nodepool from 4.20.z to 4.20.zLatest · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:133]: failed to create HCP cluster np-version-upgrade-cluster-s9ff65 with version 4.20.32
Unexpected error:
    <*fmt.wrapError | 0xc0005a7b00>: 
    failed to create HCP cluster np-version-upgrade-cluster-s9ff65: failed starting cluster creation "np-version-upgrade-cluster-s9ff65" in resourcegroup="rg-np-version-upgrade-s9ff65-h2s2rm8554kt": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-version-upgrade-s9ff65-h2s2rm8554kt/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-version-upgrade-cluster-s9ff65
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:133]: failed to create HCP cluster np-version-upgrade-cluster-s9ff65 with version 4.20.32
Unexpected error:
    <*fmt.wrapError | 0xc0005a7b00>: 
    failed to create HCP cluster np-version-upgrade-cluster-s9ff65: failed starting cluster creation "np-version-upgrade-cluster-s9ff65" in resourcegroup="rg-np-version-upgrade-s9ff65-h2s2rm8554kt": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-version-upgrade-s9ff65-h2s2rm8554kt/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-version-upgrade-cluster-s9ff65
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Customer should be able to create an HCP cluster then delete it by deleting the customer resource group · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delete_cx_rg.go:90]: failed to create HCP cluster "cx-rg-hcp-cluster"
Unexpected error:
    <*fmt.wrapError | 0xc0010814a0>: 
    failed to create HCP cluster cx-rg-hcp-cluster: failed starting cluster creation "cx-rg-hcp-cluster" in resourcegroup="cx-rg-cluster-vz2n7w2tbb4j": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/cx-rg-cluster-vz2n7w2tbb4j/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cx-rg-hcp-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delete_cx_rg.go:90]: failed to create HCP cluster "cx-rg-hcp-cluster"
Unexpected error:
    <*fmt.wrapError | 0xc0010814a0>: 
    failed to create HCP cluster cx-rg-hcp-cluster: failed starting cluster creation "cx-rg-hcp-cluster" in resourcegroup="cx-rg-cluster-vz2n7w2tbb4j": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/cx-rg-cluster-vz2n7w2tbb4j/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cx-rg-hcp-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Customer should upgrade a nodepool skipping one minor version (+2) from 4.20.z to 4.22.zLatest · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:462]: failed to create HCP cluster np-skip-minor-9dn6kw at version 4.22.8
Unexpected error:
    <*fmt.wrapError | 0xc0002192e0>: 
    failed to create HCP cluster np-skip-minor-9dn6kw: failed starting cluster creation "np-skip-minor-9dn6kw" in resourcegroup="rg-np-skip-minor-9dn6kw-784v2q8cmmcb": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-skip-minor-9dn6kw-784v2q8cmmcb/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-skip-minor-9dn6kw
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:462]: failed to create HCP cluster np-skip-minor-9dn6kw at version 4.22.8
Unexpected error:
    <*fmt.wrapError | 0xc0002192e0>: 
    failed to create HCP cluster np-skip-minor-9dn6kw: failed starting cluster creation "np-skip-minor-9dn6kw" in resourcegroup="rg-np-skip-minor-9dn6kw-784v2q8cmmcb": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-skip-minor-9dn6kw-784v2q8cmmcb/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-skip-minor-9dn6kw
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Customer should be able to update node pool labels and taints · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_labels_taints.go:92]: failed to create HCP cluster cluster-np-labels-taints
Unexpected error:
    <*fmt.wrapError | 0xc000e2c960>: 
    failed to create HCP cluster cluster-np-labels-taints: failed starting cluster creation "cluster-np-labels-taints" in resourcegroup="rg-np-labels-taints-pjzclntzb6zp": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-labels-taints-pjzclntzb6zp/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-np-labels-taints
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_labels_taints.go:92]: failed to create HCP cluster cluster-np-labels-taints
Unexpected error:
    <*fmt.wrapError | 0xc000e2c960>: 
    failed to create HCP cluster cluster-np-labels-taints: failed starting cluster creation "cluster-np-labels-taints" in resourcegroup="rg-np-labels-taints-pjzclntzb6zp": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-labels-taints-pjzclntzb6zp/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-np-labels-taints
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Authorized CIDRs Connectivity should allow API access only from authorized VM IP · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:111]: failed to create HCP cluster "cidr-connectivity-test" with authorized CIDRs
Unexpected error:
    <*fmt.wrapError | 0xc000f741e0>: 
    failed to create HCP cluster cidr-connectivity-test: failed starting cluster creation "cidr-connectivity-test" in resourcegroup="e2e-cidr-connectivity-h926qtr7vrwm": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-h926qtr7vrwm/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cidr-connectivity-test
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:111]: failed to create HCP cluster "cidr-connectivity-test" with authorized CIDRs
Unexpected error:
    <*fmt.wrapError | 0xc000f741e0>: 
    failed to create HCP cluster cidr-connectivity-test: failed starting cluster creation "cidr-connectivity-test" in resourcegroup="e2e-cidr-connectivity-h926qtr7vrwm": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-h926qtr7vrwm/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cidr-connectivity-test
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Customer should create a cluster with private KAS and verify API server is only reachable from VNet while ingress remains public · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:90]: failed to create HCP cluster "private-kas" with private KAS
Unexpected error:
    <*fmt.wrapError | 0xc000a0bf20>: 
    failed to create HCP cluster private-kas: failed starting cluster creation "private-kas" in resourcegroup="private-kas-wnk697jgjh4j": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/private-kas-wnk697jgjh4j/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/private-kas
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:90]: failed to create HCP cluster "private-kas" with private KAS
Unexpected error:
    <*fmt.wrapError | 0xc000a0bf20>: 
    failed to create HCP cluster private-kas: failed starting cluster creation "private-kas" in resourcegroup="private-kas-wnk697jgjh4j": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/private-kas-wnk697jgjh4j/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/private-kas
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.22 · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:121]: HCP cluster rg-candidate-4-22-xnhfcd5225wt/cluster-candidate-4-22-qnpwxp should provision
Unexpected error:
    <*fmt.wrapError | 0xc000f0d980>: 
    failed to create HCP cluster cluster-candidate-4-22-qnpwxp: failed starting cluster creation "cluster-candidate-4-22-qnpwxp" in resourcegroup="rg-candidate-4-22-xnhfcd5225wt": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-4-22-xnhfcd5225wt/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-candidate-4-22-qnpwxp
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:121]: HCP cluster rg-candidate-4-22-xnhfcd5225wt/cluster-candidate-4-22-qnpwxp should provision
Unexpected error:
    <*fmt.wrapError | 0xc000f0d980>: 
    failed to create HCP cluster cluster-candidate-4-22-qnpwxp: failed starting cluster creation "cluster-candidate-4-22-qnpwxp" in resourcegroup="rg-candidate-4-22-xnhfcd5225wt": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-4-22-xnhfcd5225wt/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-candidate-4-22-qnpwxp
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Customer should be able to create an HCP cluster and manage pull secrets · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_pullsecret.go:113]: failed to create HCP cluster for pull secret test
Unexpected error:
    <*fmt.wrapError | 0xc000df8a00>: 
    failed to create HCP cluster pullsecret-hcp-cluster: failed starting cluster creation "pullsecret-hcp-cluster" in resourcegroup="pullsecret-test-sxrznrx9sc48": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/pullsecret-test-sxrznrx9sc48/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/pullsecret-hcp-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_pullsecret.go:113]: failed to create HCP cluster for pull secret test
Unexpected error:
    <*fmt.wrapError | 0xc000df8a00>: 
    failed to create HCP cluster pullsecret-hcp-cluster: failed starting cluster creation "pullsecret-hcp-cluster" in resourcegroup="pullsecret-test-sxrznrx9sc48": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/pullsecret-test-sxrznrx9sc48/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/pullsecret-hcp-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Customer should upgrade and update a nodepool from 4.21.z to 4.21.zLatest · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:133]: failed to create HCP cluster np-version-upgrade-cluster-p895c5 with version 4.21.23
Unexpected error:
    <*fmt.wrapError | 0xc000a79340>: 
    failed to create HCP cluster np-version-upgrade-cluster-p895c5: failed starting cluster creation "np-version-upgrade-cluster-p895c5" in resourcegroup="rg-np-version-upgrade-p895c5-htmdvlcpdjx7": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-version-upgrade-p895c5-htmdvlcpdjx7/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-version-upgrade-cluster-p895c5
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:133]: failed to create HCP cluster np-version-upgrade-cluster-p895c5 with version 4.21.23
Unexpected error:
    <*fmt.wrapError | 0xc000a79340>: 
    failed to create HCP cluster np-version-upgrade-cluster-p895c5: failed starting cluster creation "np-version-upgrade-cluster-p895c5" in resourcegroup="rg-np-version-upgrade-p895c5-htmdvlcpdjx7": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-version-upgrade-p895c5-htmdvlcpdjx7/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-version-upgrade-cluster-p895c5
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 4.22 · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:110]: failed to create HCP cluster "cluster-zstream-4-22-rhs656" with version 4.22.6 on candidate channel
Unexpected error:
    <*fmt.wrapError | 0xc000a6ddc0>: 
    failed to create HCP cluster cluster-zstream-4-22-rhs656: failed starting cluster creation "cluster-zstream-4-22-rhs656" in resourcegroup="rg-zstream-upgrade-4-22-5sqpw4sk2xfn": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-22-5sqpw4sk2xfn/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-zstream-4-22-rhs656
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:110]: failed to create HCP cluster "cluster-zstream-4-22-rhs656" with version 4.22.6 on candidate channel
Unexpected error:
    <*fmt.wrapError | 0xc000a6ddc0>: 
    failed to create HCP cluster cluster-zstream-4-22-rhs656: failed starting cluster creation "cluster-zstream-4-22-rhs656" in resourcegroup="rg-zstream-upgrade-4-22-5sqpw4sk2xfn": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-22-5sqpw4sk2xfn/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-zstream-4-22-rhs656
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Customer should be able to create a cluster with an external auth config and get the external auth config · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:94]: failed to create HCP cluster for external auth test
Unexpected error:
    <*fmt.wrapError | 0xc001099120>: 
    failed to create HCP cluster ea-cluster: failed starting cluster creation "ea-cluster" in resourcegroup="external-auth-cluster-fprfrwvn7699": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/external-auth-cluster-fprfrwvn7699/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/ea-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:94]: failed to create HCP cluster for external auth test
Unexpected error:
    <*fmt.wrapError | 0xc001099120>: 
    failed to create HCP cluster ea-cluster: failed starting cluster creation "ea-cluster" in resourcegroup="external-auth-cluster-fprfrwvn7699": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/external-auth-cluster-fprfrwvn7699/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/ea-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Nodepool Ephemeral OS Disk should create a nodepool with ephemeral OS disk when autoRepair is enabled · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_ephemeral_osdisk.go:112]: failed to create HCP cluster ephemeral-disk
Unexpected error:
    <*fmt.wrapError | 0xc001490b60>: 
    failed to create HCP cluster ephemeral-disk: failed starting cluster creation "ephemeral-disk" in resourcegroup="ephemeral-osdisk-q27qdlpcdb79": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/ephemeral-osdisk-q27qdlpcdb79/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/ephemeral-disk
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_ephemeral_osdisk.go:112]: failed to create HCP cluster ephemeral-disk
Unexpected error:
    <*fmt.wrapError | 0xc001490b60>: 
    failed to create HCP cluster ephemeral-disk: failed starting cluster creation "ephemeral-disk" in resourcegroup="ephemeral-osdisk-q27qdlpcdb79": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/ephemeral-osdisk-q27qdlpcdb79/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/ephemeral-disk
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Customer should be able to create a cluster with default autoscaling and a nodepool with autoscaling enabled up to replica limits · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:102]: failed to create HCP cluster np-autoscale-cluster
Unexpected error:
    <*fmt.wrapError | 0xc00031c140>: 
    failed to create HCP cluster np-autoscale-cluster: failed starting cluster creation "np-autoscale-cluster" in resourcegroup="np-autoscaling-6ddc42zzc4bb": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/np-autoscaling-6ddc42zzc4bb/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-autoscale-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:102]: failed to create HCP cluster np-autoscale-cluster
Unexpected error:
    <*fmt.wrapError | 0xc00031c140>: 
    failed to create HCP cluster np-autoscale-cluster: failed starting cluster creation "np-autoscale-cluster" in resourcegroup="np-autoscaling-6ddc42zzc4bb": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/np-autoscaling-6ddc42zzc4bb/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-autoscale-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Update HCPOpenShiftCluster Negative creates a cluster and fails to update its name with a PATCH request · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:84]: failed to create HCP cluster for patch-name test
Unexpected error:
    <*fmt.wrapError | 0xc0008804e0>: 
    failed to create HCP cluster patch-name-cluster: failed starting cluster creation "patch-name-cluster" in resourcegroup="patch-name-4cvts8b2twdf": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-name-4cvts8b2twdf/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-name-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:84]: failed to create HCP cluster for patch-name test
Unexpected error:
    <*fmt.wrapError | 0xc0008804e0>: 
    failed to create HCP cluster patch-name-cluster: failed starting cluster creation "patch-name-cluster" in resourcegroup="patch-name-4cvts8b2twdf": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-name-4cvts8b2twdf/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-name-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Customer should be able to forward control plane logs to a storage account and Event Hub via shoebox diagnostic settings · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/shoebox_logs.go:270]: failed to create HCP cluster shoebox-hcp-cluster
Unexpected error:
    <*fmt.wrapError | 0xc000ffcf40>: 
    failed to create HCP cluster shoebox-hcp-cluster: failed starting cluster creation "shoebox-hcp-cluster" in resourcegroup="shoebox-cluster-5wrmvw9kgftw": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/shoebox-cluster-5wrmvw9kgftw/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/shoebox-hcp-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/shoebox_logs.go:270]: failed to create HCP cluster shoebox-hcp-cluster
Unexpected error:
    <*fmt.wrapError | 0xc000ffcf40>: 
    failed to create HCP cluster shoebox-hcp-cluster: failed starting cluster creation "shoebox-hcp-cluster" in resourcegroup="shoebox-cluster-5wrmvw9kgftw": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/shoebox-cluster-5wrmvw9kgftw/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/shoebox-hcp-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Update HCPOpenShiftCluster Positive creates a cluster and updates tags with a PATCH request · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:161]: failed to create HCP cluster for patch-tags test
Unexpected error:
    <*fmt.wrapError | 0xc0004b5de0>: 
    failed to create HCP cluster patch-tags-cluster: failed starting cluster creation "patch-tags-cluster" in resourcegroup="patch-tags-mhfmwbn8d5cf": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-tags-mhfmwbn8d5cf/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-tags-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:161]: failed to create HCP cluster for patch-tags test
Unexpected error:
    <*fmt.wrapError | 0xc0004b5de0>: 
    failed to create HCP cluster patch-tags-cluster: failed starting cluster creation "patch-tags-cluster" in resourcegroup="patch-tags-mhfmwbn8d5cf": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-tags-mhfmwbn8d5cf/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-tags-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-deletion-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift
02:00:01 UTC · failed at e2e · test Customer should be able to use workload identity via the cluster OIDC issuer URL · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/oidc_issuer_workload_identity.go:217]: failed to create HCP cluster oidc-wi-cluster
Unexpected error:
    <*fmt.wrapError | 0xc000b0d940>: 
    failed to create HCP cluster oidc-wi-cluster: failed starting cluster creation "oidc-wi-cluster" in resourcegroup="oidc-wi-9z785jlz9xx8": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/oidc-wi-9z785jlz9xx8/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/oidc-wi-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/oidc_issuer_workload_identity.go:217]: failed to create HCP cluster oidc-wi-cluster
Unexpected error:
    <*fmt.wrapError | 0xc000b0d940>: 
    failed to create HCP cluster oidc-wi-cluster: failed starting cluster creation "oidc-wi-cluster" in resourcegroup="oidc-wi-9z785jlz9xx8": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/oidc-wi-9z785jlz9xx8/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/oidc-wi-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"aro-hcp.experimental.cluster.max-deletion-duration\": unrecognized experimental tag",
        "target": "tags[aro-hcp.experimental.cluster.max-deletion-duration]"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred