CIHealth Failure Patterns

Environment: INT

Failure patterns: 21 · Failures matched: 56 · Total runs: 46 · Runs affected: 29

Failure pattern

Failed at

Trend

Also in

timeout during CreateHCPClusterAndWait; context deadline exceeded

full failure pattern:

timeout during CreateHCPClusterAndWait; context deadline exceeded

Signal:

e2e

none

Full failure examples (3)

fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:166]: Unexpected error:
    <*fmt.wrapErrors | 0xc0009aed80>: 
    failed to create HCP cluster node-limit-cluster, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster node-limit-cluster in resource group node-limit-test-ccshfw, error: failed waiting for cluster="node-limit-cluster" in resourcegroup="node-limit-test-ccshfw" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster node-limit-cluster in resource group node-limit-test-ccshfw, error: context deadline exceeded
    {
        msg: "failed to create HCP cluster node-limit-cluster, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster node-limit-cluster in resource group node-limit-test-ccshfw, error: failed waiting for cluster=\"node-limit-cluster\" in resourcegroup=\"node-limit-test-ccshfw\" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster node-limit-cluster in resource group node-limit-test-ccshfw, error: context deadline exceeded",
        errs: [
            <*errors.errorString | 0xc000f45af0>{
                s: "timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster node-limit-cluster in resource group node-limit-test-ccshfw",
            },
            <*fmt.wrapErrors | 0xc0009aed50>{
                msg: "failed waiting for cluster=\"node-limit-cluster\" in resourcegroup=\"node-limit-test-ccshfw\" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster node-limit-cluster in resource group node-limit-test-ccshfw, error: context deadline exceeded",
                errs: [
                    <*errors.errorString | 0xc000f45af0>{
                        s: "timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster node-limit-cluster in resource group node-limit-test-ccshfw",
                    },
                    <context.deadlineExceededError>{},
                ],
            },
        ],
    }
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:166]: Unexpected error:
    <*fmt.wrapErrors | 0xc0009aed80>: 
    failed to create HCP cluster node-limit-cluster, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster node-limit-cluster in resource group node-limit-test-ccshfw, error: failed waiting for cluster="node-limit-cluster" in resourcegroup="node-limit-test-ccshfw" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster node-limit-cluster in resource group node-limit-test-ccshfw, error: context deadline exceeded
    {
        msg: "failed to create HCP cluster node-limit-cluster, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster node-limit-cluster in resource group node-limit-test-ccshfw, error: failed waiting for cluster=\"node-limit-cluster\" in resourcegroup=\"node-limit-test-ccshfw\" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster node-limit-cluster in resource group node-limit-test-ccshfw, error: context deadline exceeded",
        errs: [
            <*errors.errorString | 0xc000f45af0>{
                s: "timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster node-limit-cluster in resource group node-limit-test-ccshfw",
            },
            <*fmt.wrapErrors | 0xc0009aed50>{
                msg: "failed waiting for cluster=\"node-limit-cluster\" in resourcegroup=\"node-limit-test-ccshfw\" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster node-limit-cluster in resource group node-limit-test-ccshfw, error: context deadline exceeded",
                errs: [
                    <*errors.errorString | 0xc000f45af0>{
                        s: "timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster node-limit-cluster in resource group node-limit-test-ccshfw",
                    },
                    <context.deadlineExceededError>{},
                ],
            },
        ],
    }
occurred

fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:166]: Unexpected error:
    <*fmt.wrapErrors | 0xc00032e8d0>: 
    failed to create HCP cluster node-limit-cluster, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster node-limit-cluster in resource group node-limit-test-zkjhtw, error: failed waiting for cluster="node-limit-cluster" in resourcegroup="node-limit-test-zkjhtw" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster node-limit-cluster in resource group node-limit-test-zkjhtw, error: context deadline exceeded
    {
        msg: "failed to create HCP cluster node-limit-cluster, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster node-limit-cluster in resource group node-limit-test-zkjhtw, error: failed waiting for cluster=\"node-limit-cluster\" in resourcegroup=\"node-limit-test-zkjhtw\" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster node-limit-cluster in resource group node-limit-test-zkjhtw, error: context deadline exceeded",
        errs: [
            <*errors.errorString | 0xc0014f4be0>{
                s: "timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster node-limit-cluster in resource group node-limit-test-zkjhtw",
            },
            <*fmt.wrapErrors | 0xc00032e8a0>{
                msg: "failed waiting for cluster=\"node-limit-cluster\" in resourcegroup=\"node-limit-test-zkjhtw\" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster node-limit-cluster in resource group node-limit-test-zkjhtw, error: context deadline exceeded",
                errs: [
                    <*errors.errorString | 0xc0014f4be0>{
                        s: "timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster node-limit-cluster in resource group node-limit-test-zkjhtw",
                    },
                    <context.deadlineExceededError>{},
                ],
            },
        ],
    }
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:166]: Unexpected error:
    <*fmt.wrapErrors | 0xc00032e8d0>: 
    failed to create HCP cluster node-limit-cluster, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster node-limit-cluster in resource group node-limit-test-zkjhtw, error: failed waiting for cluster="node-limit-cluster" in resourcegroup="node-limit-test-zkjhtw" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster node-limit-cluster in resource group node-limit-test-zkjhtw, error: context deadline exceeded
    {
        msg: "failed to create HCP cluster node-limit-cluster, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster node-limit-cluster in resource group node-limit-test-zkjhtw, error: failed waiting for cluster=\"node-limit-cluster\" in resourcegroup=\"node-limit-test-zkjhtw\" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster node-limit-cluster in resource group node-limit-test-zkjhtw, error: context deadline exceeded",
        errs: [
            <*errors.errorString | 0xc0014f4be0>{
                s: "timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster node-limit-cluster in resource group node-limit-test-zkjhtw",
            },
            <*fmt.wrapErrors | 0xc00032e8a0>{
                msg: "failed waiting for cluster=\"node-limit-cluster\" in resourcegroup=\"node-limit-test-zkjhtw\" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster node-limit-cluster in resource group node-limit-test-zkjhtw, error: context deadline exceeded",
                errs: [
                    <*errors.errorString | 0xc0014f4be0>{
                        s: "timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster node-limit-cluster in resource group node-limit-test-zkjhtw",
                    },
                    <context.deadlineExceededError>{},
                ],
            },
        ],
    }
occurred

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_list_no_infra.go:86]: Unexpected error:
    <*fmt.wrapErrors | 0xc0010ea060>: 
    failed to create HCP cluster list-test-cluster-ccxwqz, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster list-test-cluster-ccxwqz in resource group cluster-listing-d94s94, error: failed waiting for cluster="list-test-cluster-ccxwqz" in resourcegroup="cluster-listing-d94s94" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster list-test-cluster-ccxwqz in resource group cluster-listing-d94s94, error: context deadline exceeded
    {
        msg: "failed to create HCP cluster list-test-cluster-ccxwqz, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster list-test-cluster-ccxwqz in resource group cluster-listing-d94s94, error: failed waiting for cluster=\"list-test-cluster-ccxwqz\" in resourcegroup=\"cluster-listing-d94s94\" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster list-test-cluster-ccxwqz in resource group cluster-listing-d94s94, error: context deadline exceeded",
        errs: [
            <*errors.errorString | 0xc000a83a70>{
                s: "timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster list-test-cluster-ccxwqz in resource group cluster-listing-d94s94",
            },
            <*fmt.wrapErrors | 0xc0010ea030>{
                msg: "failed waiting for cluster=\"list-test-cluster-ccxwqz\" in resourcegroup=\"cluster-listing-d94s94\" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster list-test-cluster-ccxwqz in resource group cluster-listing-d94s94, error: context deadline exceeded",
                errs: [
                    <*errors.errorString | 0xc000a83a70>{
                        s: "timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster list-test-cluster-ccxwqz in resource group cluster-listing-d94s94",
                    },
                    <context.deadlineExceededError>{},
                ],
            },
        ],
    }
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_list_no_infra.go:86]: Unexpected error:
    <*fmt.wrapErrors | 0xc0010ea060>: 
    failed to create HCP cluster list-test-cluster-ccxwqz, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster list-test-cluster-ccxwqz in resource group cluster-listing-d94s94, error: failed waiting for cluster="list-test-cluster-ccxwqz" in resourcegroup="cluster-listing-d94s94" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster list-test-cluster-ccxwqz in resource group cluster-listing-d94s94, error: context deadline exceeded
    {
        msg: "failed to create HCP cluster list-test-cluster-ccxwqz, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster list-test-cluster-ccxwqz in resource group cluster-listing-d94s94, error: failed waiting for cluster=\"list-test-cluster-ccxwqz\" in resourcegroup=\"cluster-listing-d94s94\" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster list-test-cluster-ccxwqz in resource group cluster-listing-d94s94, error: context deadline exceeded",
        errs: [
            <*errors.errorString | 0xc000a83a70>{
                s: "timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster list-test-cluster-ccxwqz in resource group cluster-listing-d94s94",
            },
            <*fmt.wrapErrors | 0xc0010ea030>{
                msg: "failed waiting for cluster=\"list-test-cluster-ccxwqz\" in resourcegroup=\"cluster-listing-d94s94\" to finish creating, caused by: timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster list-test-cluster-ccxwqz in resource group cluster-listing-d94s94, error: context deadline exceeded",
                errs: [
                    <*errors.errorString | 0xc000a83a70>{
                        s: "timeout '45.000000' minutes exceeded during CreateHCPClusterFromParam for cluster list-test-cluster-ccxwqz in resource group cluster-listing-d94s94",
                    },
                    <context.deadlineExceededError>{},
                ],
            },
        ],
    }
occurred

Contributing tests (3)

Failed At	Job	Test	Support
e2e	periodic-ci-Azure-ARO-HCP-main-periodic-integration-e2e-parallel	Customer should respect cluster-wide node limits with nodepool autoscaling	2
e2e	periodic-ci-Azure-ARO-HCP-main-periodic-integration-e2e-parallel	Customer should be able to create an HCP cluster with custom autoscaling	1
e2e	periodic-ci-Azure-ARO-HCP-main-periodic-integration-e2e-parallel	Customer should be able to list HCP clusters without node pools at both subscription and resource group levels	1

Affected runs (4)

Date (UTC)	Associated PR	Prow job
2026-03-14 00:12	n/a	2032610884349595648
2026-03-12 07:38	n/a	2031998313082916864
2026-03-11 00:05	n/a	2031521952853659648
2026-03-09 08:23	n/a	2030922443123593216

failed to cleanup resource group: at least one HCP cluster <cluster> to delete: failed waiting for hcpcluster="<cluster>" in resourcegroup="<resource-group>" to finish deleting: c...

full failure pattern:

failed to cleanup resource group: at least one HCP cluster <cluster> to delete: failed waiting for hcpcluster="<cluster>" in resourcegroup="<resource-group>" to finish deleting: context canceled

Signal:

e2e

none

Full failure examples (3)

fail [github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:243] A node timeout occurred and then the following failure was recorded in the timedout node before it exited:
Unexpected error:
    <*errors.joinError | 0xc001620390>: 
    failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster="disabled-image-registry-hcp-cluster" in resourcegroup="disabled-image-registry-vl49pp" to finish deleting: context canceled
    {
        errs: [
            <*fmt.wrapError | 0xc00080bde0>{
                msg: "failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster=\"disabled-image-registry-hcp-cluster\" in resourcegroup=\"disabled-image-registry-vl49pp\" to finish deleting: context canceled",
                err: <*fmt.wrapError | 0xc00080bdc0>{
                    msg: "at least one hcp cluster failed to delete: failed waiting for hcpCluster=\"disabled-image-registry-hcp-cluster\" in resourcegroup=\"disabled-image-registry-vl49pp\" to finish deleting: context canceled",
                    err: <*fmt.wrapError | 0xc00080bda0>{
                        msg: "failed waiting for hcpCluster=\"disabled-image-registry-hcp-cluster\" in resourcegroup=\"disabled-image-registry-vl49pp\" to finish deleting: context canceled",
                        err: <*errors.errorString | 0xb08a0c0>{
                            s: "context canceled",
                        },
                    },
                },
            },
        ],
    }
occurred  
fail [:0]: A node timeout occurred
fail [github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:243] A node timeout occurred and then the following failure was recorded in the timedout node before it exited:
Unexpected error:
    <*errors.joinError | 0xc001620390>: 
    failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster="disabled-image-registry-hcp-cluster" in resourcegroup="disabled-image-registry-vl49pp" to finish deleting: context canceled
    {
        errs: [
            <*fmt.wrapError | 0xc00080bde0>{
                msg: "failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster=\"disabled-image-registry-hcp-cluster\" in resourcegroup=\"disabled-image-registry-vl49pp\" to finish deleting: context canceled",
                err: <*fmt.wrapError | 0xc00080bdc0>{
                    msg: "at least one hcp cluster failed to delete: failed waiting for hcpCluster=\"disabled-image-registry-hcp-cluster\" in resourcegroup=\"disabled-image-registry-vl49pp\" to finish deleting: context canceled",
                    err: <*fmt.wrapError | 0xc00080bda0>{
                        msg: "failed waiting for hcpCluster=\"disabled-image-registry-hcp-cluster\" in resourcegroup=\"disabled-image-registry-vl49pp\" to finish deleting: context canceled",
                        err: <*errors.errorString | 0xb08a0c0>{
                            s: "context canceled",
                        },
                    },
                },
            },
        ],
    }
occurred  
fail [:0]: A node timeout occurred

fail [github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:243] A node timeout occurred and then the following failure was recorded in the timedout node before it exited:
Unexpected error:
    <*errors.joinError | 0xc000d462a0>: 
    failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster="ea-cluster" in resourcegroup="external-auth-cluster-kvlw4b" to finish deleting: context canceled
    {
        errs: [
            <*fmt.wrapError | 0xc0008ff140>{
                msg: "failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster=\"ea-cluster\" in resourcegroup=\"external-auth-cluster-kvlw4b\" to finish deleting: context canceled",
                err: <*fmt.wrapError | 0xc0008ff120>{
                    msg: "at least one hcp cluster failed to delete: failed waiting for hcpCluster=\"ea-cluster\" in resourcegroup=\"external-auth-cluster-kvlw4b\" to finish deleting: context canceled",
                    err: <*fmt.wrapError | 0xc0008ff100>{
                        msg: "failed waiting for hcpCluster=\"ea-cluster\" in resourcegroup=\"external-auth-cluster-kvlw4b\" to finish deleting: context canceled",
                        err: <*errors.errorString | 0xb08a0c0>{
                            s: "context canceled",
                        },
                    },
                },
            },
        ],
    }
occurred  
fail [:0]: A node timeout occurred
fail [github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:243] A node timeout occurred and then the following failure was recorded in the timedout node before it exited:
Unexpected error:
    <*errors.joinError | 0xc000d462a0>: 
    failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster="ea-cluster" in resourcegroup="external-auth-cluster-kvlw4b" to finish deleting: context canceled
    {
        errs: [
            <*fmt.wrapError | 0xc0008ff140>{
                msg: "failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster=\"ea-cluster\" in resourcegroup=\"external-auth-cluster-kvlw4b\" to finish deleting: context canceled",
                err: <*fmt.wrapError | 0xc0008ff120>{
                    msg: "at least one hcp cluster failed to delete: failed waiting for hcpCluster=\"ea-cluster\" in resourcegroup=\"external-auth-cluster-kvlw4b\" to finish deleting: context canceled",
                    err: <*fmt.wrapError | 0xc0008ff100>{
                        msg: "failed waiting for hcpCluster=\"ea-cluster\" in resourcegroup=\"external-auth-cluster-kvlw4b\" to finish deleting: context canceled",
                        err: <*errors.errorString | 0xb08a0c0>{
                            s: "context canceled",
                        },
                    },
                },
            },
        ],
    }
occurred  
fail [:0]: A node timeout occurred

fail [github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:243] A node timeout occurred and then the following failure was recorded in the timedout node before it exited:
Unexpected error:
    <*errors.joinError | 0xc000a38c90>: 
    failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster="list-test-cluster-2fj9s5" in resourcegroup="cluster-listing-hdwkfd" to finish deleting: context canceled
    {
        errs: [
            <*fmt.wrapError | 0xc0007639e0>{
                msg: "failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster=\"list-test-cluster-2fj9s5\" in resourcegroup=\"cluster-listing-hdwkfd\" to finish deleting: context canceled",
                err: <*fmt.wrapError | 0xc0007636c0>{
                    msg: "at least one hcp cluster failed to delete: failed waiting for hcpCluster=\"list-test-cluster-2fj9s5\" in resourcegroup=\"cluster-listing-hdwkfd\" to finish deleting: context canceled",
                    err: <*fmt.wrapError | 0xc000763560>{
                        msg: "failed waiting for hcpCluster=\"list-test-cluster-2fj9s5\" in resourcegroup=\"cluster-listing-hdwkfd\" to finish deleting: context canceled",
                        err: <*errors.errorString | 0xb08a0c0>{
                            s: "context canceled",
                        },
                    },
                },
            },
        ],
    }
occurred  
fail [:0]: A node timeout occurred
fail [github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:243] A node timeout occurred and then the following failure was recorded in the timedout node before it exited:
Unexpected error:
    <*errors.joinError | 0xc000a38c90>: 
    failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster="list-test-cluster-2fj9s5" in resourcegroup="cluster-listing-hdwkfd" to finish deleting: context canceled
    {
        errs: [
            <*fmt.wrapError | 0xc0007639e0>{
                msg: "failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster=\"list-test-cluster-2fj9s5\" in resourcegroup=\"cluster-listing-hdwkfd\" to finish deleting: context canceled",
                err: <*fmt.wrapError | 0xc0007636c0>{
                    msg: "at least one hcp cluster failed to delete: failed waiting for hcpCluster=\"list-test-cluster-2fj9s5\" in resourcegroup=\"cluster-listing-hdwkfd\" to finish deleting: context canceled",
                    err: <*fmt.wrapError | 0xc000763560>{
                        msg: "failed waiting for hcpCluster=\"list-test-cluster-2fj9s5\" in resourcegroup=\"cluster-listing-hdwkfd\" to finish deleting: context canceled",
                        err: <*errors.errorString | 0xb08a0c0>{
                            s: "context canceled",
                        },
                    },
                },
            },
        ],
    }
occurred  
fail [:0]: A node timeout occurred

Contributing tests (6)

Failed At	Job	Test	Support
e2e	periodic-ci-Azure-ARO-HCP-main-periodic-integration-e2e-parallel	Customer should be able to create a cluster with an external auth config and get the external auth config	1
e2e	periodic-ci-Azure-ARO-HCP-main-periodic-integration-e2e-parallel	Customer should be able to create an HCP cluster with Image Registry not present	1
e2e	periodic-ci-Azure-ARO-HCP-main-periodic-integration-e2e-parallel	Customer should be able to create an HCP cluster with back-level version 4.19	1
e2e	periodic-ci-Azure-ARO-HCP-main-periodic-integration-e2e-parallel	Customer should be able to list HCP clusters without node pools at both subscription and resource group levels	1
e2e	periodic-ci-Azure-ARO-HCP-main-periodic-integration-e2e-parallel	Customer should not be able to reuse subnets and NSGs between clusters	1
e2e	periodic-ci-Azure-ARO-HCP-main-periodic-integration-e2e-parallel	HCP Nodepools GPU instances creates and deletes vm type NC4asT4v3 in a single cluster	1

Affected runs (5)

Date (UTC)	Associated PR	Prow job
2026-03-13 05:39	n/a	2032330637133746176
2026-03-13 03:21	n/a	2032295845155573760
2026-03-12 21:14	n/a	2032203554982203392
2026-03-12 16:06	n/a	2032126189862129664
2026-03-11 14:41	n/a	2031742424082026496

RevokeCredentialsAndWait timed out after <minutes> minutes; context deadline exceeded

full failure pattern:

RevokeCredentialsAndWait timed out after <minutes> minutes; context deadline exceeded

Signal:

e2e

none

Full failure examples (3)

fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:193]: Unexpected error:
    <*fmt.wrapErrors | 0xc0010d0990>: 
    failed waiting for hcpCluster="admin-cred-lifecycle-pq72j7" in resourcegroup="admin-credential-lifecycle-test-h2qqpf" to finish revoking creds, caused by: timeout '15.000000' minutes exceeded during RevokeCredentialsAndWait for cluster admin-cred-lifecycle-pq72j7 in resource group admin-credential-lifecycle-test-h2qqpf, error: context deadline exceeded
    {
        msg: "failed waiting for hcpCluster=\"admin-cred-lifecycle-pq72j7\" in resourcegroup=\"admin-credential-lifecycle-test-h2qqpf\" to finish revoking creds, caused by: timeout '15.000000' minutes exceeded during RevokeCredentialsAndWait for cluster admin-cred-lifecycle-pq72j7 in resource group admin-credential-lifecycle-test-h2qqpf, error: context deadline exceeded",
        errs: [
            <*errors.errorString | 0xc000ba21f0>{
                s: "timeout '15.000000' minutes exceeded during RevokeCredentialsAndWait for cluster admin-cred-lifecycle-pq72j7 in resource group admin-credential-lifecycle-test-h2qqpf",
            },
            <context.deadlineExceededError>{},
        ],
    }
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:193]: Unexpected error:
    <*fmt.wrapErrors | 0xc0010d0990>: 
    failed waiting for hcpCluster="admin-cred-lifecycle-pq72j7" in resourcegroup="admin-credential-lifecycle-test-h2qqpf" to finish revoking creds, caused by: timeout '15.000000' minutes exceeded during RevokeCredentialsAndWait for cluster admin-cred-lifecycle-pq72j7 in resource group admin-credential-lifecycle-test-h2qqpf, error: context deadline exceeded
    {
        msg: "failed waiting for hcpCluster=\"admin-cred-lifecycle-pq72j7\" in resourcegroup=\"admin-credential-lifecycle-test-h2qqpf\" to finish revoking creds, caused by: timeout '15.000000' minutes exceeded during RevokeCredentialsAndWait for cluster admin-cred-lifecycle-pq72j7 in resource group admin-credential-lifecycle-test-h2qqpf, error: context deadline exceeded",
        errs: [
            <*errors.errorString | 0xc000ba21f0>{
                s: "timeout '15.000000' minutes exceeded during RevokeCredentialsAndWait for cluster admin-cred-lifecycle-pq72j7 in resource group admin-credential-lifecycle-test-h2qqpf",
            },
            <context.deadlineExceededError>{},
        ],
    }
occurred

fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:193]: Unexpected error:
    <*fmt.wrapErrors | 0xc0015288d0>: 
    failed waiting for hcpCluster="admin-cred-lifecycle-xxmzq6" in resourcegroup="admin-credential-lifecycle-test-gb2rdr" to finish revoking creds, caused by: timeout '15.000000' minutes exceeded during RevokeCredentialsAndWait for cluster admin-cred-lifecycle-xxmzq6 in resource group admin-credential-lifecycle-test-gb2rdr, error: context deadline exceeded
    {
        msg: "failed waiting for hcpCluster=\"admin-cred-lifecycle-xxmzq6\" in resourcegroup=\"admin-credential-lifecycle-test-gb2rdr\" to finish revoking creds, caused by: timeout '15.000000' minutes exceeded during RevokeCredentialsAndWait for cluster admin-cred-lifecycle-xxmzq6 in resource group admin-credential-lifecycle-test-gb2rdr, error: context deadline exceeded",
        errs: [
            <*errors.errorString | 0xc000037e40>{
                s: "timeout '15.000000' minutes exceeded during RevokeCredentialsAndWait for cluster admin-cred-lifecycle-xxmzq6 in resource group admin-credential-lifecycle-test-gb2rdr",
            },
            <context.deadlineExceededError>{},
        ],
    }
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:193]: Unexpected error:
    <*fmt.wrapErrors | 0xc0015288d0>: 
    failed waiting for hcpCluster="admin-cred-lifecycle-xxmzq6" in resourcegroup="admin-credential-lifecycle-test-gb2rdr" to finish revoking creds, caused by: timeout '15.000000' minutes exceeded during RevokeCredentialsAndWait for cluster admin-cred-lifecycle-xxmzq6 in resource group admin-credential-lifecycle-test-gb2rdr, error: context deadline exceeded
    {
        msg: "failed waiting for hcpCluster=\"admin-cred-lifecycle-xxmzq6\" in resourcegroup=\"admin-credential-lifecycle-test-gb2rdr\" to finish revoking creds, caused by: timeout '15.000000' minutes exceeded during RevokeCredentialsAndWait for cluster admin-cred-lifecycle-xxmzq6 in resource group admin-credential-lifecycle-test-gb2rdr, error: context deadline exceeded",
        errs: [
            <*errors.errorString | 0xc000037e40>{
                s: "timeout '15.000000' minutes exceeded during RevokeCredentialsAndWait for cluster admin-cred-lifecycle-xxmzq6 in resource group admin-credential-lifecycle-test-gb2rdr",
            },
            <context.deadlineExceededError>{},
        ],
    }
occurred

fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:193]: Unexpected error:
    <*fmt.wrapErrors | 0xc001148150>: 
    failed waiting for hcpCluster="admin-cred-lifecycle-7z8gkd" in resourcegroup="admin-credential-lifecycle-test-mf5bt8" to finish revoking creds, caused by: timeout '15.000000' minutes exceeded during RevokeCredentialsAndWait for cluster admin-cred-lifecycle-7z8gkd in resource group admin-credential-lifecycle-test-mf5bt8, error: context deadline exceeded
    {
        msg: "failed waiting for hcpCluster=\"admin-cred-lifecycle-7z8gkd\" in resourcegroup=\"admin-credential-lifecycle-test-mf5bt8\" to finish revoking creds, caused by: timeout '15.000000' minutes exceeded during RevokeCredentialsAndWait for cluster admin-cred-lifecycle-7z8gkd in resource group admin-credential-lifecycle-test-mf5bt8, error: context deadline exceeded",
        errs: [
            <*errors.errorString | 0xc00050faa0>{
                s: "timeout '15.000000' minutes exceeded during RevokeCredentialsAndWait for cluster admin-cred-lifecycle-7z8gkd in resource group admin-credential-lifecycle-test-mf5bt8",
            },
            <context.deadlineExceededError>{},
        ],
    }
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:193]: Unexpected error:
    <*fmt.wrapErrors | 0xc001148150>: 
    failed waiting for hcpCluster="admin-cred-lifecycle-7z8gkd" in resourcegroup="admin-credential-lifecycle-test-mf5bt8" to finish revoking creds, caused by: timeout '15.000000' minutes exceeded during RevokeCredentialsAndWait for cluster admin-cred-lifecycle-7z8gkd in resource group admin-credential-lifecycle-test-mf5bt8, error: context deadline exceeded
    {
        msg: "failed waiting for hcpCluster=\"admin-cred-lifecycle-7z8gkd\" in resourcegroup=\"admin-credential-lifecycle-test-mf5bt8\" to finish revoking creds, caused by: timeout '15.000000' minutes exceeded during RevokeCredentialsAndWait for cluster admin-cred-lifecycle-7z8gkd in resource group admin-credential-lifecycle-test-mf5bt8, error: context deadline exceeded",
        errs: [
            <*errors.errorString | 0xc00050faa0>{
                s: "timeout '15.000000' minutes exceeded during RevokeCredentialsAndWait for cluster admin-cred-lifecycle-7z8gkd in resource group admin-credential-lifecycle-test-mf5bt8",
            },
            <context.deadlineExceededError>{},
        ],
    }
occurred

Contributing tests (1)

Failed At	Job	Test	Support
e2e	periodic-ci-Azure-ARO-HCP-main-periodic-integration-e2e-parallel	Customer should be able to test admin credentials before cluster ready, then full admin credential lifecycle	5

Affected runs (5)

Date (UTC)	Associated PR	Prow job
2026-03-15 21:48	n/a	2033299216234713088
2026-03-15 19:54	n/a	2033270683638697984
2026-03-15 03:22	n/a	2033020998986502144
2026-03-14 15:31	n/a	2032842010720735232
2026-03-14 13:01	n/a	2032804286143926272

ERROR CODE: InvalidIdentityValues; provider Microsoft.ManagedIdentity

full failure pattern:

ERROR CODE: InvalidIdentityValues; provider Microsoft.ManagedIdentity

Signal:

e2e

none

Full failure examples (3)

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:415]: Unexpected error:
    <*exported.ResponseError | 0xc000f97890>: 
    PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-zpzk8c/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cidr-connectivity-test
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidIdentityValues
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidIdentityValues",
        "message": "Invalid value for the identities '/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-8/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cloud-controller-manager,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-8/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cloud-network-config,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-8/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cluster-api-azure,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-8/providers/Microsoft.ManagedIdentity/userAssignedIdentities/control-plane,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-8/providers/Microsoft.ManagedIdentity/userAssignedIdentities/disk-csi-driver,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-8/providers/Microsoft.ManagedIdentity/userAssignedIdentities/file-csi-driver,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-8/providers/Microsoft.ManagedIdentity/userAssignedIdentities/image-registry,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-8/providers/Microsoft.ManagedIdentity/userAssignedIdentities/ingress,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-8/providers/Microsoft.ManagedIdentity/userAssignedIdentities/kms,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-8/providers/Microsoft.ManagedIdentity/userAssignedIdentities/service'. The 'UserAssignedIdentities' property keys should only be empty null during PATCH requests."
      }
    }
    --------------------------------------------------------------------------------
    
    {
        ErrorCode: "InvalidIdentityValues",
        StatusCode: 400,
        RawResponse: {
            Status: "400 Bad Request",
            StatusCode: 400,
            Proto: "HTTP/2.0",
            ProtoMajor: 2,
            ProtoMinor: 0,
            Header: {
                "X-Ms-Routing-Request-Id": [
                    "UKSOUTH:20260311T151231Z:9c11778c-ff60-487b-a423-651593da8efe",
                ],
                "Strict-Transport-Security": [
                    "max-age=31536000; includeSubDomains",
                ],
                "X-Msedge-Ref": [
                    "Ref A: 4505E2BF16A0421FA071A149B08EF585 Ref B: CHI261140504060 Ref C: 2026-03-11T15:12:31Z",
                ],
                "Cache-Control": ["no-cache"],
                "Content-Type": [
                    "application/json; charset=utf-8",
                ],
                "X-Ms-Failure-Cause": ["gateway"],
                "X-Content-Type-Options": ["nosniff"],
                "X-Cache": ["CONFIG_NOCACHE"],
                "Date": [
                    "Wed, 11 Mar 2026 15:12:31 GMT",
                ],
                "Pragma": ["no-cache"],
                "Content-Length": ["1927"],
                "Expires": ["-1"],
                "X-Ms-Request-Id": [
                    "9c11778c-ff60-487b-a423-651593da8efe",
                ],
                "X-Ms-Correlation-Request-Id": [
                    "9c11778c-ff60-487b-a423-651593da8efe",
                ],
            },
            Body: <*exported.nopClosingBytesReader | 0xc001100fc0>{
                s: "{\"error\":{\"code\":\"InvalidIdentityValues\",\"message\":\"Invalid value for the identities '/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-8/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cloud-controller-manager,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-8/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cloud-network-config,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-8/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cluster-api-azure,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-8/providers/Microsoft.ManagedIdentity/userAssignedIdentities/control-plane,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-8/providers/Microsoft.ManagedIdentity/userAssignedIdentities/disk-csi-driver,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-8/providers/Microsoft.ManagedIdentity/userAssignedIdentities/file-csi-driver,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-8/providers/Microsoft.ManagedIdentity/userAssignedIdentities/image-registry,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-8/providers/Microsoft.ManagedIdentity/userAssignedIdentities/ingress,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-8/providers/Microsoft.ManagedIdentity/userAssignedIdentities/kms,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-8/providers/Microsoft.ManagedIdentity/userAssignedIdentities/service'. The 'UserAssignedIdentities' property keys should only be empty null during PATCH requests.\"}}",
                i: 0,
            },
            ContentLength: 1927,
            TransferEncoding: nil,
            Close: false,
            Uncompressed: false,
            Trailer: nil,
            Request: {
                Method: "PUT",
                URL: {
                    Scheme: "https",
                    Opaque: "",
                    User: nil,
                    Host: "management.azure.com",
                    Path: "/subscriptions/64f0619f-ebc2-4156-...

Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.

Learn more here: https://onsi.github.io/gomega/#adjusting-output

occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:415]: Unexpected error:
    <*exported.ResponseError | 0xc000f97890>: 
    PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-zpzk8c/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cidr-connectivity-test
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidIdentityValues
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidIdentityValues",
        "message": "Invalid value for the identities '/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-8/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cloud-controller-manager,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-8/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cloud-network-config,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-8/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cluster-api-azure,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-8/providers/Microsoft.ManagedIdentity/userAssignedIdentities/control-plane,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-8/providers/Microsoft.ManagedIdentity/userAssignedIdentities/disk-csi-driver,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-8/providers/Microsoft.ManagedIdentity/userAssignedIdentities/file-csi-driver,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-8/providers/Microsoft.ManagedIdentity/userAssignedIdentities/image-registry,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-8/providers/Microsoft.ManagedIdentity/userAssignedIdentities/ingress,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-8/providers/Microsoft.ManagedIdentity/userAssignedIdentities/kms,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-8/providers/Microsoft.ManagedIdentity/userAssignedIdentities/service'. The 'UserAssignedIdentities' property keys should only be empty null during PATCH requests."
      }
    }
    --------------------------------------------------------------------------------
    
    {
        ErrorCode: "InvalidIdentityValues",
        StatusCode: 400,
        RawResponse: {
            Status: "400 Bad Request",
            StatusCode: 400,
            Proto: "HTTP/2.0",
            ProtoMajor: 2,
            ProtoMinor: 0,
            Header: {
                "X-Ms-Routing-Request-Id": [
                    "UKSOUTH:20260311T151231Z:9c11778c-ff60-487b-a423-651593da8efe",
                ],
                "Strict-Transport-Security": [
                    "max-age=31536000; includeSubDomains",
                ],
                "X-Msedge-Ref": [
                    "Ref A: 4505E2BF16A0421FA071A149B08EF585 Ref B: CHI261140504060 Ref C: 2026-03-11T15:12:31Z",
                ],
                "Cache-Control": ["no-cache"],
                "Content-Type": [
                    "application/json; charset=utf-8",
                ],
                "X-Ms-Failure-Cause": ["gateway"],
                "X-Content-Type-Options": ["nosniff"],
                "X-Cache": ["CONFIG_NOCACHE"],
                "Date": [
                    "Wed, 11 Mar 2026 15:12:31 GMT",
                ],
                "Pragma": ["no-cache"],
                "Content-Length": ["1927"],
                "Expires": ["-1"],
                "X-Ms-Request-Id": [
                    "9c11778c-ff60-487b-a423-651593da8efe",
                ],
                "X-Ms-Correlation-Request-Id": [
                    "9c11778c-ff60-487b-a423-651593da8efe",
                ],
            },
            Body: <*exported.nopClosingBytesReader | 0xc001100fc0>{
                s: "{\"error\":{\"code\":\"InvalidIdentityValues\",\"message\":\"Invalid value for the identities '/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-8/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cloud-controller-manager,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-8/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cloud-network-config,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-8/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cluster-api-azure,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-8/providers/Microsoft.ManagedIdentity/userAssignedIdentities/control-plane,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-8/providers/Microsoft.ManagedIdentity/userAssignedIdentities/disk-csi-driver,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-8/providers/Microsoft.ManagedIdentity/userAssignedIdentities/file-csi-driver,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-8/providers/Microsoft.ManagedIdentity/userAssignedIdentities/image-registry,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-8/providers/Microsoft.ManagedIdentity/userAssignedIdentities/ingress,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-8/providers/Microsoft.ManagedIdentity/userAssignedIdentities/kms,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-8/providers/Microsoft.ManagedIdentity/userAssignedIdentities/service'. The 'UserAssignedIdentities' property keys should only be empty null during PATCH requests.\"}}",
                i: 0,
            },
            ContentLength: 1927,
            TransferEncoding: nil,
            Close: false,
            Uncompressed: false,
            Trailer: nil,
            Request: {
                Method: "PUT",
                URL: {
                    Scheme: "https",
                    Opaque: "",
                    User: nil,
                    Host: "management.azure.com",
                    Path: "/subscriptions/64f0619f-ebc2-4156-...

Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.

Learn more here: https://onsi.github.io/gomega/#adjusting-output

occurred

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:415]: Unexpected error:
    <*exported.ResponseError | 0xc0009bd380>: 
    PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-nld688/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cidr-connectivity-test
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidIdentityValues
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidIdentityValues",
        "message": "Invalid value for the identities '/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-67/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cloud-controller-manager,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-67/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cloud-network-config,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-67/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cluster-api-azure,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-67/providers/Microsoft.ManagedIdentity/userAssignedIdentities/control-plane,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-67/providers/Microsoft.ManagedIdentity/userAssignedIdentities/disk-csi-driver,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-67/providers/Microsoft.ManagedIdentity/userAssignedIdentities/file-csi-driver,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-67/providers/Microsoft.ManagedIdentity/userAssignedIdentities/image-registry,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-67/providers/Microsoft.ManagedIdentity/userAssignedIdentities/ingress,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-67/providers/Microsoft.ManagedIdentity/userAssignedIdentities/kms,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-67/providers/Microsoft.ManagedIdentity/userAssignedIdentities/service'. The 'UserAssignedIdentities' property keys should only be empty null during PATCH requests."
      }
    }
    --------------------------------------------------------------------------------
    
    {
        ErrorCode: "InvalidIdentityValues",
        StatusCode: 400,
        RawResponse: {
            Status: "400 Bad Request",
            StatusCode: 400,
            Proto: "HTTP/2.0",
            ProtoMajor: 2,
            ProtoMinor: 0,
            Header: {
                "Cache-Control": ["no-cache"],
                "Pragma": ["no-cache"],
                "Content-Type": [
                    "application/json; charset=utf-8",
                ],
                "Expires": ["-1"],
                "X-Ms-Routing-Request-Id": [
                    "UKSOUTH:20260311T130855Z:1f5ad0e5-72af-4f34-87a3-cbd696db9711",
                ],
                "Strict-Transport-Security": [
                    "max-age=31536000; includeSubDomains",
                ],
                "X-Content-Type-Options": ["nosniff"],
                "X-Cache": ["CONFIG_NOCACHE"],
                "Content-Length": ["1937"],
                "X-Ms-Failure-Cause": ["gateway"],
                "X-Ms-Request-Id": [
                    "1f5ad0e5-72af-4f34-87a3-cbd696db9711",
                ],
                "X-Ms-Correlation-Request-Id": [
                    "1f5ad0e5-72af-4f34-87a3-cbd696db9711",
                ],
                "X-Msedge-Ref": [
                    "Ref A: 79888FDC7E984BA8A98CF39D24F6722A Ref B: CH1AA2020610011 Ref C: 2026-03-11T13:08:55Z",
                ],
                "Date": [
                    "Wed, 11 Mar 2026 13:08:54 GMT",
                ],
            },
            Body: <*exported.nopClosingBytesReader | 0xc000d91100>{
                s: "{\"error\":{\"code\":\"InvalidIdentityValues\",\"message\":\"Invalid value for the identities '/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-67/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cloud-controller-manager,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-67/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cloud-network-config,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-67/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cluster-api-azure,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-67/providers/Microsoft.ManagedIdentity/userAssignedIdentities/control-plane,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-67/providers/Microsoft.ManagedIdentity/userAssignedIdentities/disk-csi-driver,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-67/providers/Microsoft.ManagedIdentity/userAssignedIdentities/file-csi-driver,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-67/providers/Microsoft.ManagedIdentity/userAssignedIdentities/image-registry,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-67/providers/Microsoft.ManagedIdentity/userAssignedIdentities/ingress,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-67/providers/Microsoft.ManagedIdentity/userAssignedIdentities/kms,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-67/providers/Microsoft.ManagedIdentity/userAssignedIdentities/service'. The 'UserAssignedIdentities' property keys should only be empty null during PATCH requests.\"}}",
                i: 0,
            },
            ContentLength: 1937,
            TransferEncoding: nil,
            Close: false,
            Uncompressed: false,
            Trailer: nil,
            Request: {
                Method: "PUT",
                URL: {
                    Scheme: "https",
                    Opaque: "",
                    User: nil,
                    Host: "management.azure.com",
                    Path: "/subscriptions/64f0619f-...

Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.

Learn more here: https://onsi.github.io/gomega/#adjusting-output

occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:415]: Unexpected error:
    <*exported.ResponseError | 0xc0009bd380>: 
    PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-nld688/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cidr-connectivity-test
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidIdentityValues
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidIdentityValues",
        "message": "Invalid value for the identities '/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-67/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cloud-controller-manager,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-67/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cloud-network-config,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-67/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cluster-api-azure,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-67/providers/Microsoft.ManagedIdentity/userAssignedIdentities/control-plane,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-67/providers/Microsoft.ManagedIdentity/userAssignedIdentities/disk-csi-driver,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-67/providers/Microsoft.ManagedIdentity/userAssignedIdentities/file-csi-driver,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-67/providers/Microsoft.ManagedIdentity/userAssignedIdentities/image-registry,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-67/providers/Microsoft.ManagedIdentity/userAssignedIdentities/ingress,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-67/providers/Microsoft.ManagedIdentity/userAssignedIdentities/kms,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-67/providers/Microsoft.ManagedIdentity/userAssignedIdentities/service'. The 'UserAssignedIdentities' property keys should only be empty null during PATCH requests."
      }
    }
    --------------------------------------------------------------------------------
    
    {
        ErrorCode: "InvalidIdentityValues",
        StatusCode: 400,
        RawResponse: {
            Status: "400 Bad Request",
            StatusCode: 400,
            Proto: "HTTP/2.0",
            ProtoMajor: 2,
            ProtoMinor: 0,
            Header: {
                "Cache-Control": ["no-cache"],
                "Pragma": ["no-cache"],
                "Content-Type": [
                    "application/json; charset=utf-8",
                ],
                "Expires": ["-1"],
                "X-Ms-Routing-Request-Id": [
                    "UKSOUTH:20260311T130855Z:1f5ad0e5-72af-4f34-87a3-cbd696db9711",
                ],
                "Strict-Transport-Security": [
                    "max-age=31536000; includeSubDomains",
                ],
                "X-Content-Type-Options": ["nosniff"],
                "X-Cache": ["CONFIG_NOCACHE"],
                "Content-Length": ["1937"],
                "X-Ms-Failure-Cause": ["gateway"],
                "X-Ms-Request-Id": [
                    "1f5ad0e5-72af-4f34-87a3-cbd696db9711",
                ],
                "X-Ms-Correlation-Request-Id": [
                    "1f5ad0e5-72af-4f34-87a3-cbd696db9711",
                ],
                "X-Msedge-Ref": [
                    "Ref A: 79888FDC7E984BA8A98CF39D24F6722A Ref B: CH1AA2020610011 Ref C: 2026-03-11T13:08:55Z",
                ],
                "Date": [
                    "Wed, 11 Mar 2026 13:08:54 GMT",
                ],
            },
            Body: <*exported.nopClosingBytesReader | 0xc000d91100>{
                s: "{\"error\":{\"code\":\"InvalidIdentityValues\",\"message\":\"Invalid value for the identities '/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-67/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cloud-controller-manager,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-67/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cloud-network-config,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-67/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cluster-api-azure,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-67/providers/Microsoft.ManagedIdentity/userAssignedIdentities/control-plane,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-67/providers/Microsoft.ManagedIdentity/userAssignedIdentities/disk-csi-driver,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-67/providers/Microsoft.ManagedIdentity/userAssignedIdentities/file-csi-driver,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-67/providers/Microsoft.ManagedIdentity/userAssignedIdentities/image-registry,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-67/providers/Microsoft.ManagedIdentity/userAssignedIdentities/ingress,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-67/providers/Microsoft.ManagedIdentity/userAssignedIdentities/kms,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-67/providers/Microsoft.ManagedIdentity/userAssignedIdentities/service'. The 'UserAssignedIdentities' property keys should only be empty null during PATCH requests.\"}}",
                i: 0,
            },
            ContentLength: 1937,
            TransferEncoding: nil,
            Close: false,
            Uncompressed: false,
            Trailer: nil,
            Request: {
                Method: "PUT",
                URL: {
                    Scheme: "https",
                    Opaque: "",
                    User: nil,
                    Host: "management.azure.com",
                    Path: "/subscriptions/64f0619f-...

Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.

Learn more here: https://onsi.github.io/gomega/#adjusting-output

occurred

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:415]: Unexpected error:
    <*exported.ResponseError | 0xc00087cb10>: 
    PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-q89nkn/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cidr-connectivity-test
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidIdentityValues
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidIdentityValues",
        "message": "Invalid value for the identities '/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-143/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cloud-controller-manager,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-143/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cloud-network-config,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-143/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cluster-api-azure,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-143/providers/Microsoft.ManagedIdentity/userAssignedIdentities/control-plane,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-143/providers/Microsoft.ManagedIdentity/userAssignedIdentities/disk-csi-driver,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-143/providers/Microsoft.ManagedIdentity/userAssignedIdentities/file-csi-driver,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-143/providers/Microsoft.ManagedIdentity/userAssignedIdentities/image-registry,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-143/providers/Microsoft.ManagedIdentity/userAssignedIdentities/ingress,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-143/providers/Microsoft.ManagedIdentity/userAssignedIdentities/kms,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-143/providers/Microsoft.ManagedIdentity/userAssignedIdentities/service'. The 'UserAssignedIdentities' property keys should only be empty null during PATCH requests."
      }
    }
    --------------------------------------------------------------------------------
    
    {
        ErrorCode: "InvalidIdentityValues",
        StatusCode: 400,
        RawResponse: {
            Status: "400 Bad Request",
            StatusCode: 400,
            Proto: "HTTP/2.0",
            ProtoMajor: 2,
            ProtoMinor: 0,
            Header: {
                "Expires": ["-1"],
                "X-Ms-Request-Id": [
                    "5d0f1a83-e9e3-4322-9402-079527a36e43",
                ],
                "X-Ms-Correlation-Request-Id": [
                    "5d0f1a83-e9e3-4322-9402-079527a36e43",
                ],
                "Strict-Transport-Security": [
                    "max-age=31536000; includeSubDomains",
                ],
                "X-Msedge-Ref": [
                    "Ref A: 7799C63EF4DB4BDE88CFAD7F106271DB Ref B: CHI261140512040 Ref C: 2026-03-11T06:03:42Z",
                ],
                "Pragma": ["no-cache"],
                "Content-Length": ["1947"],
                "X-Ms-Failure-Cause": ["gateway"],
                "X-Ms-Routing-Request-Id": [
                    "UKSOUTH:20260311T060342Z:5d0f1a83-e9e3-4322-9402-079527a36e43",
                ],
                "X-Content-Type-Options": ["nosniff"],
                "X-Cache": ["CONFIG_NOCACHE"],
                "Date": [
                    "Wed, 11 Mar 2026 06:03:41 GMT",
                ],
                "Cache-Control": ["no-cache"],
                "Content-Type": [
                    "application/json; charset=utf-8",
                ],
            },
            Body: <*exported.nopClosingBytesReader | 0xc00011d160>{
                s: "{\"error\":{\"code\":\"InvalidIdentityValues\",\"message\":\"Invalid value for the identities '/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-143/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cloud-controller-manager,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-143/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cloud-network-config,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-143/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cluster-api-azure,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-143/providers/Microsoft.ManagedIdentity/userAssignedIdentities/control-plane,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-143/providers/Microsoft.ManagedIdentity/userAssignedIdentities/disk-csi-driver,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-143/providers/Microsoft.ManagedIdentity/userAssignedIdentities/file-csi-driver,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-143/providers/Microsoft.ManagedIdentity/userAssignedIdentities/image-registry,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-143/providers/Microsoft.ManagedIdentity/userAssignedIdentities/ingress,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-143/providers/Microsoft.ManagedIdentity/userAssignedIdentities/kms,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-143/providers/Microsoft.ManagedIdentity/userAssignedIdentities/service'. The 'UserAssignedIdentities' property keys should only be empty null during PATCH requests.\"}}",
                i: 0,
            },
            ContentLength: 1947,
            TransferEncoding: nil,
            Close: false,
            Uncompressed: false,
            Trailer: nil,
            Request: {
                Method: "PUT",
                URL: {
                    Scheme: "https",
                    Opaque: "",
                    User: nil,
                    Host: "management.azure.com",
                    Path: "/subscriptions...

Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.

Learn more here: https://onsi.github.io/gomega/#adjusting-output

occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:415]: Unexpected error:
    <*exported.ResponseError | 0xc00087cb10>: 
    PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-q89nkn/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cidr-connectivity-test
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidIdentityValues
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidIdentityValues",
        "message": "Invalid value for the identities '/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-143/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cloud-controller-manager,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-143/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cloud-network-config,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-143/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cluster-api-azure,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-143/providers/Microsoft.ManagedIdentity/userAssignedIdentities/control-plane,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-143/providers/Microsoft.ManagedIdentity/userAssignedIdentities/disk-csi-driver,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-143/providers/Microsoft.ManagedIdentity/userAssignedIdentities/file-csi-driver,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-143/providers/Microsoft.ManagedIdentity/userAssignedIdentities/image-registry,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-143/providers/Microsoft.ManagedIdentity/userAssignedIdentities/ingress,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-143/providers/Microsoft.ManagedIdentity/userAssignedIdentities/kms,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-143/providers/Microsoft.ManagedIdentity/userAssignedIdentities/service'. The 'UserAssignedIdentities' property keys should only be empty null during PATCH requests."
      }
    }
    --------------------------------------------------------------------------------
    
    {
        ErrorCode: "InvalidIdentityValues",
        StatusCode: 400,
        RawResponse: {
            Status: "400 Bad Request",
            StatusCode: 400,
            Proto: "HTTP/2.0",
            ProtoMajor: 2,
            ProtoMinor: 0,
            Header: {
                "Expires": ["-1"],
                "X-Ms-Request-Id": [
                    "5d0f1a83-e9e3-4322-9402-079527a36e43",
                ],
                "X-Ms-Correlation-Request-Id": [
                    "5d0f1a83-e9e3-4322-9402-079527a36e43",
                ],
                "Strict-Transport-Security": [
                    "max-age=31536000; includeSubDomains",
                ],
                "X-Msedge-Ref": [
                    "Ref A: 7799C63EF4DB4BDE88CFAD7F106271DB Ref B: CHI261140512040 Ref C: 2026-03-11T06:03:42Z",
                ],
                "Pragma": ["no-cache"],
                "Content-Length": ["1947"],
                "X-Ms-Failure-Cause": ["gateway"],
                "X-Ms-Routing-Request-Id": [
                    "UKSOUTH:20260311T060342Z:5d0f1a83-e9e3-4322-9402-079527a36e43",
                ],
                "X-Content-Type-Options": ["nosniff"],
                "X-Cache": ["CONFIG_NOCACHE"],
                "Date": [
                    "Wed, 11 Mar 2026 06:03:41 GMT",
                ],
                "Cache-Control": ["no-cache"],
                "Content-Type": [
                    "application/json; charset=utf-8",
                ],
            },
            Body: <*exported.nopClosingBytesReader | 0xc00011d160>{
                s: "{\"error\":{\"code\":\"InvalidIdentityValues\",\"message\":\"Invalid value for the identities '/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-143/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cloud-controller-manager,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-143/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cloud-network-config,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-143/providers/Microsoft.ManagedIdentity/userAssignedIdentities/cluster-api-azure,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-143/providers/Microsoft.ManagedIdentity/userAssignedIdentities/control-plane,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-143/providers/Microsoft.ManagedIdentity/userAssignedIdentities/disk-csi-driver,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-143/providers/Microsoft.ManagedIdentity/userAssignedIdentities/file-csi-driver,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-143/providers/Microsoft.ManagedIdentity/userAssignedIdentities/image-registry,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-143/providers/Microsoft.ManagedIdentity/userAssignedIdentities/ingress,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-143/providers/Microsoft.ManagedIdentity/userAssignedIdentities/kms,/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-test-msi-containers-int-143/providers/Microsoft.ManagedIdentity/userAssignedIdentities/service'. The 'UserAssignedIdentities' property keys should only be empty null during PATCH requests.\"}}",
                i: 0,
            },
            ContentLength: 1947,
            TransferEncoding: nil,
            Close: false,
            Uncompressed: false,
            Trailer: nil,
            Request: {
                Method: "PUT",
                URL: {
                    Scheme: "https",
                    Opaque: "",
                    User: nil,
                    Host: "management.azure.com",
                    Path: "/subscriptions...

Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.

Learn more here: https://onsi.github.io/gomega/#adjusting-output

occurred

Contributing tests (1)

Failed At	Job	Test	Support
e2e	periodic-ci-Azure-ARO-HCP-main-periodic-integration-e2e-parallel	Authorized CIDRs Connectivity should allow API access only from authorized VM IP	4

Affected runs (4)

Date (UTC)	Associated PR	Prow job
2026-03-11 14:41	n/a	2031742424082026496
2026-03-11 12:39	n/a	2031711680643207168
2026-03-11 05:33	n/a	2031604521683128320
2026-03-11 03:22	n/a	2031571552545280000

not to be nil

full failure pattern:

not to be nil

Signal:

e2e

none

Full failure examples (2)

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:91]: Expected
    <*string | 0x0>: nil
not to be nil
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:91]: Expected
    <*string | 0x0>: nil
not to be nil

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:145]: Expected
    <*string | 0x0>: nil
not to be nil
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:145]: Expected
    <*string | 0x0>: nil
not to be nil

Contributing tests (2)

Failed At	Job	Test	Support
e2e	periodic-ci-Azure-ARO-HCP-main-periodic-integration-e2e-parallel	Customer should create an HCP cluster and validate TLS certificates	3
e2e	periodic-ci-Azure-ARO-HCP-main-periodic-integration-e2e-parallel	Authorized CIDRs Connectivity should allow API access only from authorized VM IP	2

Affected runs (3)

Date (UTC)	Associated PR	Prow job
2026-03-11 16:52	n/a	2031775377503817728
2026-03-11 10:16	n/a	2031675611008733184
2026-03-11 03:22	n/a	2031571552545280000

ERROR CODE: LockContention; detail message Failed to acquire lock: timed out; provider Microsoft.RedHatOpenShift

full failure pattern:

ERROR CODE: LockContention; detail message Failed to acquire lock: timed out; provider Microsoft.RedHatOpenShift

Signal:

e2e

none

Full failure examples (3)

fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_list_and_verify.go:118]: Unexpected error:
    <*fmt.wrapError | 0xc00038c420>: 
    failed creating external auth "ea-list" in resourcegroup="ea-list-9lhb8x" for cluster="ea-listgv59sm": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/ea-list-9lhb8x/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/ea-listgv59sm/externalAuths/ea-list
    --------------------------------------------------------------------------------
    RESPONSE 503: 503 Service Unavailable
    ERROR CODE: LockContention
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "LockContention",
        "message": "Failed to acquire lock: timed out",
        "target": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
      }
    }
    --------------------------------------------------------------------------------
    
    {
        msg: "failed creating external auth \"ea-list\" in resourcegroup=\"ea-list-9lhb8x\" for cluster=\"ea-listgv59sm\": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/ea-list-9lhb8x/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/ea-listgv59sm/externalAuths/ea-list\n--------------------------------------------------------------------------------\nRESPONSE 503: 503 Service Unavailable\nERROR CODE: LockContention\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"LockContention\",\n    \"message\": \"Failed to acquire lock: timed out\",\n    \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX\"\n  }\n}\n--------------------------------------------------------------------------------\n",
        err: <*exported.ResponseError | 0xc00076e300>{
            ErrorCode: "LockContention",
            StatusCode: 503,
            RawResponse: {
                Status: "503 Service Unavailable",
                StatusCode: 503,
                Proto: "HTTP/2.0",
                ProtoMajor: 2,
                ProtoMinor: 0,
                Header: {
                    "Retry-After": ["10"],
                    "X-Ms-Failure-Cause": ["service"],
                    "X-Ms-Operation-Identifier": [
                        "appId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,tenantId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,objectId=c209f8df-52ae-48fb-98ea-380f58b04652/uksouth/4616b9c4-eae5-42c0-b42e-e60136e4b27a",
                    ],
                    "X-Cache": ["CONFIG_NOCACHE"],
                    "Date": [
                        "Thu, 12 Mar 2026 22:53:06 GMT",
                    ],
                    "Content-Length": ["186"],
                    "X-Ms-Request-Id": [
                        "0daaf6b9-dea5-419a-a781-f5d3f0c88c48",
                    ],
                    "X-Ms-Ratelimit-Remaining-Subscription-Writes": ["199"],
                    "Strict-Transport-Security": [
                        "max-age=31536000; includeSubDomains",
                    ],
                    "Expires": ["-1"],
                    "X-Ms-Error-Code": ["LockContention"],
                    "X-Ms-Ratelimit-Remaining-Subscription-Global-Writes": ["2999"],
                    "X-Content-Type-Options": ["nosniff"],
                    "X-Msedge-Ref": [
                        "Ref A: DD91E82BD13A4C8F974F7F6335BA4D4F Ref B: CHI261140506025 Ref C: 2026-03-12T22:52:57Z",
                    ],
                    "Cache-Control": ["no-cache"],
                    "Pragma": ["no-cache"],
                    "Content-Type": ["application/json"],
                    "X-Envoy-Upstream-Service-Time": ["10007"],
                    "X-Ms-Correlation-Request-Id": [
                        "132f0482-3ab7-4255-98f8-45a73cf0c24c",
                    ],
                    "X-Ms-Routing-Request-Id": [
                        "UKSOUTH:20260312T225307Z:132f0482-3ab7-4255-98f8-45a73cf0c24c",
                    ],
                },
                Body: <*exported.nopClosingBytesReader | 0xc00038c200>{
                    s: [123, 10, 32, 32, 32, 32, 34, 101, 114, 114, 111, 114, 34, 58, 32, 123, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 99, 111, 100, 101, 34, 58, 32, 34, 76, 111, 99, 107, 67, 111, 110, 116, 101, 110, 116, 105, 111, 110, 34, 44, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 109, 101, 115, 115, 97, 103, 101, 34, 58, 32, 34, 70, 97, 105, 108, 101, 100, 32, 116, 111, 32, 97, 99, 113, 117, 105, 114, 101, 32, 108, 111, 99, 107, 58, 32, 116, 105, 109, 101, 100, 32, 111, 117, 116, 34, 44, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 116, 97, 114, 103, 101, 116, 34, 58, 32, 34, 47, 115, 117, 98, 115, 99, 114, 105, 112, 116, 105, 111, 110, 115, 47, 54, 52, 102, 48, 54, 49, 57, 102, 45, 101, 98, 99, 50, 45, 52, 49, 53, 54, 45, 57, 100, 57, 49, 45, 99, 52, 99, 55, 56, 49, 100, 101, 55, 101, 53, 52, 34, 10, 32, 32, 32, 32, 125, 10, 125]...

Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.

Learn more here: https://onsi.github.io/gomega/#adjusting-output

occurred
fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_list_and_verify.go:118]: Unexpected error:
    <*fmt.wrapError | 0xc00038c420>: 
    failed creating external auth "ea-list" in resourcegroup="ea-list-9lhb8x" for cluster="ea-listgv59sm": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/ea-list-9lhb8x/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/ea-listgv59sm/externalAuths/ea-list
    --------------------------------------------------------------------------------
    RESPONSE 503: 503 Service Unavailable
    ERROR CODE: LockContention
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "LockContention",
        "message": "Failed to acquire lock: timed out",
        "target": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
      }
    }
    --------------------------------------------------------------------------------
    
    {
        msg: "failed creating external auth \"ea-list\" in resourcegroup=\"ea-list-9lhb8x\" for cluster=\"ea-listgv59sm\": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/ea-list-9lhb8x/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/ea-listgv59sm/externalAuths/ea-list\n--------------------------------------------------------------------------------\nRESPONSE 503: 503 Service Unavailable\nERROR CODE: LockContention\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"LockContention\",\n    \"message\": \"Failed to acquire lock: timed out\",\n    \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX\"\n  }\n}\n--------------------------------------------------------------------------------\n",
        err: <*exported.ResponseError | 0xc00076e300>{
            ErrorCode: "LockContention",
            StatusCode: 503,
            RawResponse: {
                Status: "503 Service Unavailable",
                StatusCode: 503,
                Proto: "HTTP/2.0",
                ProtoMajor: 2,
                ProtoMinor: 0,
                Header: {
                    "Retry-After": ["10"],
                    "X-Ms-Failure-Cause": ["service"],
                    "X-Ms-Operation-Identifier": [
                        "appId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,tenantId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,objectId=c209f8df-52ae-48fb-98ea-380f58b04652/uksouth/4616b9c4-eae5-42c0-b42e-e60136e4b27a",
                    ],
                    "X-Cache": ["CONFIG_NOCACHE"],
                    "Date": [
                        "Thu, 12 Mar 2026 22:53:06 GMT",
                    ],
                    "Content-Length": ["186"],
                    "X-Ms-Request-Id": [
                        "0daaf6b9-dea5-419a-a781-f5d3f0c88c48",
                    ],
                    "X-Ms-Ratelimit-Remaining-Subscription-Writes": ["199"],
                    "Strict-Transport-Security": [
                        "max-age=31536000; includeSubDomains",
                    ],
                    "Expires": ["-1"],
                    "X-Ms-Error-Code": ["LockContention"],
                    "X-Ms-Ratelimit-Remaining-Subscription-Global-Writes": ["2999"],
                    "X-Content-Type-Options": ["nosniff"],
                    "X-Msedge-Ref": [
                        "Ref A: DD91E82BD13A4C8F974F7F6335BA4D4F Ref B: CHI261140506025 Ref C: 2026-03-12T22:52:57Z",
                    ],
                    "Cache-Control": ["no-cache"],
                    "Pragma": ["no-cache"],
                    "Content-Type": ["application/json"],
                    "X-Envoy-Upstream-Service-Time": ["10007"],
                    "X-Ms-Correlation-Request-Id": [
                        "132f0482-3ab7-4255-98f8-45a73cf0c24c",
                    ],
                    "X-Ms-Routing-Request-Id": [
                        "UKSOUTH:20260312T225307Z:132f0482-3ab7-4255-98f8-45a73cf0c24c",
                    ],
                },
                Body: <*exported.nopClosingBytesReader | 0xc00038c200>{
                    s: [123, 10, 32, 32, 32, 32, 34, 101, 114, 114, 111, 114, 34, 58, 32, 123, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 99, 111, 100, 101, 34, 58, 32, 34, 76, 111, 99, 107, 67, 111, 110, 116, 101, 110, 116, 105, 111, 110, 34, 44, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 109, 101, 115, 115, 97, 103, 101, 34, 58, 32, 34, 70, 97, 105, 108, 101, 100, 32, 116, 111, 32, 97, 99, 113, 117, 105, 114, 101, 32, 108, 111, 99, 107, 58, 32, 116, 105, 109, 101, 100, 32, 111, 117, 116, 34, 44, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 116, 97, 114, 103, 101, 116, 34, 58, 32, 34, 47, 115, 117, 98, 115, 99, 114, 105, 112, 116, 105, 111, 110, 115, 47, 54, 52, 102, 48, 54, 49, 57, 102, 45, 101, 98, 99, 50, 45, 52, 49, 53, 54, 45, 57, 100, 57, 49, 45, 99, 52, 99, 55, 56, 49, 100, 101, 55, 101, 53, 52, 34, 10, 32, 32, 32, 32, 125, 10, 125]...

Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.

Learn more here: https://onsi.github.io/gomega/#adjusting-output

occurred

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_pullsecret.go:145]: Unexpected error:
    <*fmt.wrapError | 0xc00148e400>: 
    failed to create NodePool np-1: failed starting nodepool creation "np-1" for cluster "pullsecret-hcp-cluster" in resourcegroup="pullsecret-test-h2fml7": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/pullsecret-test-h2fml7/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/pullsecret-hcp-cluster/nodePools/np-1
    --------------------------------------------------------------------------------
    RESPONSE 503: 503 Service Unavailable
    ERROR CODE: LockContention
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "LockContention",
        "message": "Failed to acquire lock: timed out",
        "target": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
      }
    }
    --------------------------------------------------------------------------------
    
    {
        msg: "failed to create NodePool np-1: failed starting nodepool creation \"np-1\" for cluster \"pullsecret-hcp-cluster\" in resourcegroup=\"pullsecret-test-h2fml7\": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/pullsecret-test-h2fml7/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/pullsecret-hcp-cluster/nodePools/np-1\n--------------------------------------------------------------------------------\nRESPONSE 503: 503 Service Unavailable\nERROR CODE: LockContention\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"LockContention\",\n    \"message\": \"Failed to acquire lock: timed out\",\n    \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX\"\n  }\n}\n--------------------------------------------------------------------------------\n",
        err: <*fmt.wrapError | 0xc00148e3e0>{
            msg: "failed starting nodepool creation \"np-1\" for cluster \"pullsecret-hcp-cluster\" in resourcegroup=\"pullsecret-test-h2fml7\": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/pullsecret-test-h2fml7/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/pullsecret-hcp-cluster/nodePools/np-1\n--------------------------------------------------------------------------------\nRESPONSE 503: 503 Service Unavailable\nERROR CODE: LockContention\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"LockContention\",\n    \"message\": \"Failed to acquire lock: timed out\",\n    \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX\"\n  }\n}\n--------------------------------------------------------------------------------\n",
            err: <*exported.ResponseError | 0xc000b78690>{
                ErrorCode: "LockContention",
                StatusCode: 503,
                RawResponse: {
                    Status: "503 Service Unavailable",
                    StatusCode: 503,
                    Proto: "HTTP/2.0",
                    ProtoMajor: 2,
                    ProtoMinor: 0,
                    Header: {
                        "Expires": ["-1"],
                        "X-Ms-Ratelimit-Remaining-Subscription-Writes": ["199"],
                        "Strict-Transport-Security": [
                            "max-age=31536000; includeSubDomains",
                        ],
                        "Content-Length": ["186"],
                        "X-Ms-Error-Code": ["LockContention"],
                        "X-Envoy-Upstream-Service-Time": ["10025"],
                        "X-Ms-Operation-Identifier": [
                            "appId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,tenantId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,objectId=c209f8df-52ae-48fb-98ea-380f58b04652/uksouth/ab289601-a1be-4eea-baec-7844c0a021f7",
                        ],
                        "X-Ms-Ratelimit-Remaining-Subscription-Global-Writes": ["2999"],
                        "X-Cache": ["CONFIG_NOCACHE"],
                        "Date": [
                            "Thu, 12 Mar 2026 22:53:06 GMT",
                        ],
                        "Content-Type": ["application/json"],
                        "Retry-After": ["10"],
                        "X-Ms-Failure-Cause": ["service"],
                        "X-Msedge-Ref": [
                            "Ref A: 6E495E82722B47FAB217CA309B5EFC10 Ref B: CHI261140501060 Ref C: 2026-03-12T22:52:57Z",
                        ],
                        "Cache-Control": ["no-cache"],
                        "Pragma": ["no-cache"],
                        "X-Ms-Request-Id": [
                            "c86fd44b-9640-4399-b837-438b4fc03302",
                        ],
                        "X-Ms-Correlation-Request-Id": [
                            "8be5b8ee-e739-4b49-a85a-b05296aaa904",
                        ],
               ...

Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.

Learn more here: https://onsi.github.io/gomega/#adjusting-output

occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_pullsecret.go:145]: Unexpected error:
    <*fmt.wrapError | 0xc00148e400>: 
    failed to create NodePool np-1: failed starting nodepool creation "np-1" for cluster "pullsecret-hcp-cluster" in resourcegroup="pullsecret-test-h2fml7": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/pullsecret-test-h2fml7/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/pullsecret-hcp-cluster/nodePools/np-1
    --------------------------------------------------------------------------------
    RESPONSE 503: 503 Service Unavailable
    ERROR CODE: LockContention
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "LockContention",
        "message": "Failed to acquire lock: timed out",
        "target": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
      }
    }
    --------------------------------------------------------------------------------
    
    {
        msg: "failed to create NodePool np-1: failed starting nodepool creation \"np-1\" for cluster \"pullsecret-hcp-cluster\" in resourcegroup=\"pullsecret-test-h2fml7\": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/pullsecret-test-h2fml7/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/pullsecret-hcp-cluster/nodePools/np-1\n--------------------------------------------------------------------------------\nRESPONSE 503: 503 Service Unavailable\nERROR CODE: LockContention\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"LockContention\",\n    \"message\": \"Failed to acquire lock: timed out\",\n    \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX\"\n  }\n}\n--------------------------------------------------------------------------------\n",
        err: <*fmt.wrapError | 0xc00148e3e0>{
            msg: "failed starting nodepool creation \"np-1\" for cluster \"pullsecret-hcp-cluster\" in resourcegroup=\"pullsecret-test-h2fml7\": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/pullsecret-test-h2fml7/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/pullsecret-hcp-cluster/nodePools/np-1\n--------------------------------------------------------------------------------\nRESPONSE 503: 503 Service Unavailable\nERROR CODE: LockContention\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"LockContention\",\n    \"message\": \"Failed to acquire lock: timed out\",\n    \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX\"\n  }\n}\n--------------------------------------------------------------------------------\n",
            err: <*exported.ResponseError | 0xc000b78690>{
                ErrorCode: "LockContention",
                StatusCode: 503,
                RawResponse: {
                    Status: "503 Service Unavailable",
                    StatusCode: 503,
                    Proto: "HTTP/2.0",
                    ProtoMajor: 2,
                    ProtoMinor: 0,
                    Header: {
                        "Expires": ["-1"],
                        "X-Ms-Ratelimit-Remaining-Subscription-Writes": ["199"],
                        "Strict-Transport-Security": [
                            "max-age=31536000; includeSubDomains",
                        ],
                        "Content-Length": ["186"],
                        "X-Ms-Error-Code": ["LockContention"],
                        "X-Envoy-Upstream-Service-Time": ["10025"],
                        "X-Ms-Operation-Identifier": [
                            "appId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,tenantId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,objectId=c209f8df-52ae-48fb-98ea-380f58b04652/uksouth/ab289601-a1be-4eea-baec-7844c0a021f7",
                        ],
                        "X-Ms-Ratelimit-Remaining-Subscription-Global-Writes": ["2999"],
                        "X-Cache": ["CONFIG_NOCACHE"],
                        "Date": [
                            "Thu, 12 Mar 2026 22:53:06 GMT",
                        ],
                        "Content-Type": ["application/json"],
                        "Retry-After": ["10"],
                        "X-Ms-Failure-Cause": ["service"],
                        "X-Msedge-Ref": [
                            "Ref A: 6E495E82722B47FAB217CA309B5EFC10 Ref B: CHI261140501060 Ref C: 2026-03-12T22:52:57Z",
                        ],
                        "Cache-Control": ["no-cache"],
                        "Pragma": ["no-cache"],
                        "X-Ms-Request-Id": [
                            "c86fd44b-9640-4399-b837-438b4fc03302",
                        ],
                        "X-Ms-Correlation-Request-Id": [
                            "8be5b8ee-e739-4b49-a85a-b05296aaa904",
                        ],
               ...

Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.

Learn more here: https://onsi.github.io/gomega/#adjusting-output

occurred

fail [github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:243]: Unexpected error:
    <*errors.joinError | 0xc0015420c0>: 
    failed to cleanup resource group: at least one hcp cluster failed to delete: DELETE https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/clusternp128-pqj6m6/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/hcp-cluster-np-128
    --------------------------------------------------------------------------------
    RESPONSE 503: 503 Service Unavailable
    ERROR CODE: LockContention
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "LockContention",
        "message": "Failed to acquire lock: timed out",
        "target": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
      }
    }
    --------------------------------------------------------------------------------
    
    {
        errs: [
            <*fmt.wrapError | 0xc000840400>{
                msg: "failed to cleanup resource group: at least one hcp cluster failed to delete: DELETE https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/clusternp128-pqj6m6/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/hcp-cluster-np-128\n--------------------------------------------------------------------------------\nRESPONSE 503: 503 Service Unavailable\nERROR CODE: LockContention\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"LockContention\",\n    \"message\": \"Failed to acquire lock: timed out\",\n    \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX\"\n  }\n}\n--------------------------------------------------------------------------------\n",
                err: <*fmt.wrapError | 0xc0008403e0>{
                    msg: "at least one hcp cluster failed to delete: DELETE https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/clusternp128-pqj6m6/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/hcp-cluster-np-128\n--------------------------------------------------------------------------------\nRESPONSE 503: 503 Service Unavailable\nERROR CODE: LockContention\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"LockContention\",\n    \"message\": \"Failed to acquire lock: timed out\",\n    \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX\"\n  }\n}\n--------------------------------------------------------------------------------\n",
                    err: <*exported.ResponseError | 0xc0013a6450>{
                        ErrorCode: "LockContention",
                        StatusCode: 503,
                        RawResponse: {
                            Status: "503 Service Unavailable",
                            StatusCode: 503,
                            Proto: "HTTP/2.0",
                            ProtoMajor: 2,
                            ProtoMinor: 0,
                            Header: {
                                "Cache-Control": ["no-cache"],
                                "Content-Length": ["186"],
                                "X-Ms-Error-Code": ["LockContention"],
                                "X-Ms-Request-Id": [
                                    "57007d8e-6281-4f79-87e3-6b62771677b7",
                                ],
                                "X-Ms-Failure-Cause": ["service"],
                                "X-Ms-Ratelimit-Remaining-Subscription-Global-Deletes": ["2999"],
                                "X-Content-Type-Options": ["nosniff"],
                                "Date": [
                                    "Thu, 12 Mar 2026 20:29:07 GMT",
                                ],
                                "Pragma": ["no-cache"],
                                "Content-Type": ["application/json"],
                                "X-Envoy-Upstream-Service-Time": ["10009"],
                                "X-Ms-Operation-Identifier": [
                                    "appId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,tenantId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,objectId=c209f8df-52ae-48fb-98ea-380f58b04652/uksouth/9a19b057-0917-4393-a2cf-ff9874abdb4b",
                                ],
                                "X-Ms-Correlation-Request-Id": [
                                    "cd3510ea-1cec-45db-8f3f-4cac3c6e7c36",
                                ],
                                "Strict-Transport-Security": [
                                    "max-age=31536000; includeSubDomains",
                                ],
                                "X-Cache": ["CONFIG_NOCACHE"],
                                "X-Msedge-Ref": [
                                    "Ref A: DC15C07BB4844226852ED55513D0D6F1 Ref B: CHI261140507054 Re...

Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.

Learn more here: https://onsi.github.io/gomega/#adjusting-output

occurred
fail [github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:243]: Unexpected error:
    <*errors.joinError | 0xc0015420c0>: 
    failed to cleanup resource group: at least one hcp cluster failed to delete: DELETE https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/clusternp128-pqj6m6/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/hcp-cluster-np-128
    --------------------------------------------------------------------------------
    RESPONSE 503: 503 Service Unavailable
    ERROR CODE: LockContention
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "LockContention",
        "message": "Failed to acquire lock: timed out",
        "target": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
      }
    }
    --------------------------------------------------------------------------------
    
    {
        errs: [
            <*fmt.wrapError | 0xc000840400>{
                msg: "failed to cleanup resource group: at least one hcp cluster failed to delete: DELETE https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/clusternp128-pqj6m6/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/hcp-cluster-np-128\n--------------------------------------------------------------------------------\nRESPONSE 503: 503 Service Unavailable\nERROR CODE: LockContention\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"LockContention\",\n    \"message\": \"Failed to acquire lock: timed out\",\n    \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX\"\n  }\n}\n--------------------------------------------------------------------------------\n",
                err: <*fmt.wrapError | 0xc0008403e0>{
                    msg: "at least one hcp cluster failed to delete: DELETE https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/clusternp128-pqj6m6/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/hcp-cluster-np-128\n--------------------------------------------------------------------------------\nRESPONSE 503: 503 Service Unavailable\nERROR CODE: LockContention\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"LockContention\",\n    \"message\": \"Failed to acquire lock: timed out\",\n    \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX\"\n  }\n}\n--------------------------------------------------------------------------------\n",
                    err: <*exported.ResponseError | 0xc0013a6450>{
                        ErrorCode: "LockContention",
                        StatusCode: 503,
                        RawResponse: {
                            Status: "503 Service Unavailable",
                            StatusCode: 503,
                            Proto: "HTTP/2.0",
                            ProtoMajor: 2,
                            ProtoMinor: 0,
                            Header: {
                                "Cache-Control": ["no-cache"],
                                "Content-Length": ["186"],
                                "X-Ms-Error-Code": ["LockContention"],
                                "X-Ms-Request-Id": [
                                    "57007d8e-6281-4f79-87e3-6b62771677b7",
                                ],
                                "X-Ms-Failure-Cause": ["service"],
                                "X-Ms-Ratelimit-Remaining-Subscription-Global-Deletes": ["2999"],
                                "X-Content-Type-Options": ["nosniff"],
                                "Date": [
                                    "Thu, 12 Mar 2026 20:29:07 GMT",
                                ],
                                "Pragma": ["no-cache"],
                                "Content-Type": ["application/json"],
                                "X-Envoy-Upstream-Service-Time": ["10009"],
                                "X-Ms-Operation-Identifier": [
                                    "appId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,tenantId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,objectId=c209f8df-52ae-48fb-98ea-380f58b04652/uksouth/9a19b057-0917-4393-a2cf-ff9874abdb4b",
                                ],
                                "X-Ms-Correlation-Request-Id": [
                                    "cd3510ea-1cec-45db-8f3f-4cac3c6e7c36",
                                ],
                                "Strict-Transport-Security": [
                                    "max-age=31536000; includeSubDomains",
                                ],
                                "X-Cache": ["CONFIG_NOCACHE"],
                                "X-Msedge-Ref": [
                                    "Ref A: DC15C07BB4844226852ED55513D0D6F1 Ref B: CHI261140507054 Re...

Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.

Learn more here: https://onsi.github.io/gomega/#adjusting-output

occurred

Contributing tests (4)

Failed At	Job	Test	Support
e2e	periodic-ci-Azure-ARO-HCP-main-periodic-integration-e2e-parallel	Customer should be able to create an HCP cluster and custom node pool osDisk size	1
e2e	periodic-ci-Azure-ARO-HCP-main-periodic-integration-e2e-parallel	Customer should be able to create an HCP cluster and manage pull secrets	1
e2e	periodic-ci-Azure-ARO-HCP-main-periodic-integration-e2e-parallel	Customer should be able to lifecycle and confirm external auth on a cluster	1
e2e	periodic-ci-Azure-ARO-HCP-main-periodic-integration-e2e-parallel	Update HCPOpenShiftCluster Positive creates a cluster and updates tags with a PATCH request	1

Affected runs (3)

Date (UTC)	Associated PR	Prow job
2026-03-12 22:14	n/a	2032218705995239424
2026-03-12 18:57	n/a	2032169081666277376
2026-03-10 11:07	n/a	2031326084175237120

timeout during GetAdminRESTConfigForHCPCluster while waiting for hcpcluster creds

full failure pattern:

timeout during GetAdminRESTConfigForHCPCluster while waiting for hcpcluster creds

Signal:

e2e

none

Full failure examples (3)

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:170]: Unexpected error:
    <*fmt.wrapErrors | 0xc00154e1e0>: 
    failed waiting for hcpCluster="patch-tags-cluster" in resourcegroup="patch-tags-hb8wm7" to finish getting creds, caused by: timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster patch-tags-cluster in resource group patch-tags-hb8wm7, error: context deadline exceeded
    {
        msg: "failed waiting for hcpCluster=\"patch-tags-cluster\" in resourcegroup=\"patch-tags-hb8wm7\" to finish getting creds, caused by: timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster patch-tags-cluster in resource group patch-tags-hb8wm7, error: context deadline exceeded",
        errs: [
            <*errors.errorString | 0xc001533930>{
                s: "timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster patch-tags-cluster in resource group patch-tags-hb8wm7",
            },
            <context.deadlineExceededError>{},
        ],
    }
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:170]: Unexpected error:
    <*fmt.wrapErrors | 0xc00154e1e0>: 
    failed waiting for hcpCluster="patch-tags-cluster" in resourcegroup="patch-tags-hb8wm7" to finish getting creds, caused by: timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster patch-tags-cluster in resource group patch-tags-hb8wm7, error: context deadline exceeded
    {
        msg: "failed waiting for hcpCluster=\"patch-tags-cluster\" in resourcegroup=\"patch-tags-hb8wm7\" to finish getting creds, caused by: timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster patch-tags-cluster in resource group patch-tags-hb8wm7, error: context deadline exceeded",
        errs: [
            <*errors.errorString | 0xc001533930>{
                s: "timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster patch-tags-cluster in resource group patch-tags-hb8wm7",
            },
            <context.deadlineExceededError>{},
        ],
    }
occurred

fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:103]: Unexpected error:
    <*fmt.wrapErrors | 0xc001502510>: 
    failed waiting for hcpCluster="ea-cluster" in resourcegroup="external-auth-cluster-7rvs2t" to finish getting creds, caused by: timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster ea-cluster in resource group external-auth-cluster-7rvs2t, error: context deadline exceeded
    {
        msg: "failed waiting for hcpCluster=\"ea-cluster\" in resourcegroup=\"external-auth-cluster-7rvs2t\" to finish getting creds, caused by: timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster ea-cluster in resource group external-auth-cluster-7rvs2t, error: context deadline exceeded",
        errs: [
            <*errors.errorString | 0xc000879850>{
                s: "timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster ea-cluster in resource group external-auth-cluster-7rvs2t",
            },
            <context.deadlineExceededError>{},
        ],
    }
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:103]: Unexpected error:
    <*fmt.wrapErrors | 0xc001502510>: 
    failed waiting for hcpCluster="ea-cluster" in resourcegroup="external-auth-cluster-7rvs2t" to finish getting creds, caused by: timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster ea-cluster in resource group external-auth-cluster-7rvs2t, error: context deadline exceeded
    {
        msg: "failed waiting for hcpCluster=\"ea-cluster\" in resourcegroup=\"external-auth-cluster-7rvs2t\" to finish getting creds, caused by: timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster ea-cluster in resource group external-auth-cluster-7rvs2t, error: context deadline exceeded",
        errs: [
            <*errors.errorString | 0xc000879850>{
                s: "timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster ea-cluster in resource group external-auth-cluster-7rvs2t",
            },
            <context.deadlineExceededError>{},
        ],
    }
occurred

fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:173]: Unexpected error:
    <*fmt.wrapErrors | 0xc00133e300>: 
    failed waiting for hcpCluster="admin-cred-lifecycle-htfxlh" in resourcegroup="admin-credential-lifecycle-test-qq776z" to finish getting creds, caused by: timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster admin-cred-lifecycle-htfxlh in resource group admin-credential-lifecycle-test-qq776z, error: context deadline exceeded
    {
        msg: "failed waiting for hcpCluster=\"admin-cred-lifecycle-htfxlh\" in resourcegroup=\"admin-credential-lifecycle-test-qq776z\" to finish getting creds, caused by: timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster admin-cred-lifecycle-htfxlh in resource group admin-credential-lifecycle-test-qq776z, error: context deadline exceeded",
        errs: [
            <*errors.errorString | 0xc00135d130>{
                s: "timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster admin-cred-lifecycle-htfxlh in resource group admin-credential-lifecycle-test-qq776z",
            },
            <context.deadlineExceededError>{},
        ],
    }
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:173]: Unexpected error:
    <*fmt.wrapErrors | 0xc00133e300>: 
    failed waiting for hcpCluster="admin-cred-lifecycle-htfxlh" in resourcegroup="admin-credential-lifecycle-test-qq776z" to finish getting creds, caused by: timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster admin-cred-lifecycle-htfxlh in resource group admin-credential-lifecycle-test-qq776z, error: context deadline exceeded
    {
        msg: "failed waiting for hcpCluster=\"admin-cred-lifecycle-htfxlh\" in resourcegroup=\"admin-credential-lifecycle-test-qq776z\" to finish getting creds, caused by: timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster admin-cred-lifecycle-htfxlh in resource group admin-credential-lifecycle-test-qq776z, error: context deadline exceeded",
        errs: [
            <*errors.errorString | 0xc00135d130>{
                s: "timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster admin-cred-lifecycle-htfxlh in resource group admin-credential-lifecycle-test-qq776z",
            },
            <context.deadlineExceededError>{},
        ],
    }
occurred

Contributing tests (3)

Failed At	Job	Test	Support
e2e	periodic-ci-Azure-ARO-HCP-main-periodic-integration-e2e-parallel	Customer should be able to create a cluster with an external auth config and get the external auth config	1
e2e	periodic-ci-Azure-ARO-HCP-main-periodic-integration-e2e-parallel	Customer should be able to test admin credentials before cluster ready, then full admin credential lifecycle	1
e2e	periodic-ci-Azure-ARO-HCP-main-periodic-integration-e2e-parallel	Update HCPOpenShiftCluster Positive creates a cluster and updates tags with a PATCH request	1

Affected runs (2)

Date (UTC)	Associated PR	Prow job
2026-03-12 07:38	n/a	2031998313082916864
2026-03-09 23:12	n/a	2031146014437543936

ERROR CODE: InvalidTemplate; detail message Deployment template validation failed: 'The following parameters were supplied, but do not correspond to any parameters defined in the ...

full failure pattern:

ERROR CODE: InvalidTemplate; detail message Deployment template validation failed: 'The following parameters were supplied, but do not correspond to any parameters defined in the template: 'tags'. The parameters defined in the template are: 'clusterName, identities, openshiftControlPlaneVersionId, openshiftNodePoolVersionId, persistTagValue, usePooledIdentities'.

Signal:

e2e

none

Full failure examples (2)

fail [github.com/Azure/ARO-HCP/test/e2e/gpu_nodepools_create_delete.go:100]: Unexpected error:
    <*fmt.wrapError | 0xc000848600>: 
    failed creating deployment "aro-hcp-demo" in resourcegroup="gpu-nodepools-NC4asT4v3-zhtprd": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourcegroups/gpu-nodepools-NC4asT4v3-zhtprd/providers/Microsoft.Resources/deployments/aro-hcp-demo
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidTemplate
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidTemplate",
        "message": "Deployment template validation failed: 'The following parameters were supplied, but do not correspond to any parameters defined in the template: 'tags'. The parameters defined in the template are: 'clusterName, identities, openshiftControlPlaneVersionId, openshiftNodePoolVersionId, persistTagValue, usePooledIdentities'. Please see https://aka.ms/arm-pass-parameter-values for usage details.'.",
        "additionalInfo": [
          {
            "type": "TemplateViolation",
            "info": {
              "lineNumber": 0,
              "linePosition": 0,
              "path": ""
            }
          }
        ]
      }
    }
    --------------------------------------------------------------------------------
    
    {
        msg: "failed creating deployment \"aro-hcp-demo\" in resourcegroup=\"gpu-nodepools-NC4asT4v3-zhtprd\": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourcegroups/gpu-nodepools-NC4asT4v3-zhtprd/providers/Microsoft.Resources/deployments/aro-hcp-demo\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: InvalidTemplate\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"InvalidTemplate\",\n    \"message\": \"Deployment template validation failed: 'The following parameters were supplied, but do not correspond to any parameters defined in the template: 'tags'. The parameters defined in the template are: 'clusterName, identities, openshiftControlPlaneVersionId, openshiftNodePoolVersionId, persistTagValue, usePooledIdentities'. Please see https://aka.ms/arm-pass-parameter-values for usage details.'.\",\n    \"additionalInfo\": [\n      {\n        \"type\": \"TemplateViolation\",\n        \"info\": {\n          \"lineNumber\": 0,\n          \"linePosition\": 0,\n          \"path\": \"\"\n        }\n      }\n    ]\n  }\n}\n--------------------------------------------------------------------------------\n",
        err: <*exported.ResponseError | 0xc00135a990>{
            ErrorCode: "InvalidTemplate",
            StatusCode: 400,
            RawResponse: {
                Status: "400 Bad Request",
                StatusCode: 400,
                Proto: "HTTP/2.0",
                ProtoMajor: 2,
                ProtoMinor: 0,
                Header: {
                    "Pragma": ["no-cache"],
                    "X-Ms-Failure-Cause": ["gateway"],
                    "X-Ms-Request-Id": [
                        "f4d69cdb-d05b-44a9-9a92-6eddb4d52e6e",
                    ],
                    "X-Ms-Routing-Request-Id": [
                        "UKSOUTH:20260310T160951Z:f4d69cdb-d05b-44a9-9a92-6eddb4d52e6e",
                    ],
                    "X-Content-Type-Options": ["nosniff"],
                    "Cache-Control": ["no-cache"],
                    "Content-Type": [
                        "application/json; charset=utf-8",
                    ],
                    "X-Ms-Ratelimit-Remaining-Subscription-Global-Writes": ["2999"],
                    "Strict-Transport-Security": [
                        "max-age=31536000; includeSubDomains",
                    ],
                    "Expires": ["-1"],
                    "X-Ms-Ratelimit-Remaining-Subscription-Writes": ["199"],
                    "X-Ms-Correlation-Request-Id": [
                        "f4d69cdb-d05b-44a9-9a92-6eddb4d52e6e",
                    ],
                    "X-Cache": ["CONFIG_NOCACHE"],
                    "Date": [
                        "Tue, 10 Mar 2026 16:09:50 GMT",
                    ],
                    "Content-Length": ["542"],
                    "X-Msedge-Ref": [
                        "Ref A: 3F20D18F061F48B0A5963AE25F0A156A Ref B: BL2AA2011003040 Ref C: 2026-03-10T16:09:50Z",
                    ],
                },
                Body: <*exported.nopClosingBytesReader | 0xc000848540>{
                    s: "{\"error\":{\"code\":\"InvalidTemplate\",\"message\":\"Deployment template validation failed: 'The following parameters were supplied, but do not correspond to any parameters defined in the template: 'tags'. The parameters defined in the template are: 'clusterName, identities, openshiftControlPlaneVersionId, openshiftNodePoolVersionId, persistTagValue, usePooledIdentities'. Please see https://aka.ms/arm-pass-parameter-values for usage details.'.\",\"additionalInfo\":[{\"type\":\"TemplateViolation\",\"info\":{\"lineNumber\":0,\"linePosition\":0,\"path\":\"\"}}]}}",
                    i: 0,
                },
                ContentLength: 542,
                TransferEncoding: nil,
                Close: false,
                Uncompress...

Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.

Learn more here: https://onsi.github.io/gomega/#adjusting-output

occurred
fail [github.com/Azure/ARO-HCP/test/e2e/gpu_nodepools_create_delete.go:100]: Unexpected error:
    <*fmt.wrapError | 0xc000848600>: 
    failed creating deployment "aro-hcp-demo" in resourcegroup="gpu-nodepools-NC4asT4v3-zhtprd": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourcegroups/gpu-nodepools-NC4asT4v3-zhtprd/providers/Microsoft.Resources/deployments/aro-hcp-demo
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidTemplate
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidTemplate",
        "message": "Deployment template validation failed: 'The following parameters were supplied, but do not correspond to any parameters defined in the template: 'tags'. The parameters defined in the template are: 'clusterName, identities, openshiftControlPlaneVersionId, openshiftNodePoolVersionId, persistTagValue, usePooledIdentities'. Please see https://aka.ms/arm-pass-parameter-values for usage details.'.",
        "additionalInfo": [
          {
            "type": "TemplateViolation",
            "info": {
              "lineNumber": 0,
              "linePosition": 0,
              "path": ""
            }
          }
        ]
      }
    }
    --------------------------------------------------------------------------------
    
    {
        msg: "failed creating deployment \"aro-hcp-demo\" in resourcegroup=\"gpu-nodepools-NC4asT4v3-zhtprd\": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourcegroups/gpu-nodepools-NC4asT4v3-zhtprd/providers/Microsoft.Resources/deployments/aro-hcp-demo\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: InvalidTemplate\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"InvalidTemplate\",\n    \"message\": \"Deployment template validation failed: 'The following parameters were supplied, but do not correspond to any parameters defined in the template: 'tags'. The parameters defined in the template are: 'clusterName, identities, openshiftControlPlaneVersionId, openshiftNodePoolVersionId, persistTagValue, usePooledIdentities'. Please see https://aka.ms/arm-pass-parameter-values for usage details.'.\",\n    \"additionalInfo\": [\n      {\n        \"type\": \"TemplateViolation\",\n        \"info\": {\n          \"lineNumber\": 0,\n          \"linePosition\": 0,\n          \"path\": \"\"\n        }\n      }\n    ]\n  }\n}\n--------------------------------------------------------------------------------\n",
        err: <*exported.ResponseError | 0xc00135a990>{
            ErrorCode: "InvalidTemplate",
            StatusCode: 400,
            RawResponse: {
                Status: "400 Bad Request",
                StatusCode: 400,
                Proto: "HTTP/2.0",
                ProtoMajor: 2,
                ProtoMinor: 0,
                Header: {
                    "Pragma": ["no-cache"],
                    "X-Ms-Failure-Cause": ["gateway"],
                    "X-Ms-Request-Id": [
                        "f4d69cdb-d05b-44a9-9a92-6eddb4d52e6e",
                    ],
                    "X-Ms-Routing-Request-Id": [
                        "UKSOUTH:20260310T160951Z:f4d69cdb-d05b-44a9-9a92-6eddb4d52e6e",
                    ],
                    "X-Content-Type-Options": ["nosniff"],
                    "Cache-Control": ["no-cache"],
                    "Content-Type": [
                        "application/json; charset=utf-8",
                    ],
                    "X-Ms-Ratelimit-Remaining-Subscription-Global-Writes": ["2999"],
                    "Strict-Transport-Security": [
                        "max-age=31536000; includeSubDomains",
                    ],
                    "Expires": ["-1"],
                    "X-Ms-Ratelimit-Remaining-Subscription-Writes": ["199"],
                    "X-Ms-Correlation-Request-Id": [
                        "f4d69cdb-d05b-44a9-9a92-6eddb4d52e6e",
                    ],
                    "X-Cache": ["CONFIG_NOCACHE"],
                    "Date": [
                        "Tue, 10 Mar 2026 16:09:50 GMT",
                    ],
                    "Content-Length": ["542"],
                    "X-Msedge-Ref": [
                        "Ref A: 3F20D18F061F48B0A5963AE25F0A156A Ref B: BL2AA2011003040 Ref C: 2026-03-10T16:09:50Z",
                    ],
                },
                Body: <*exported.nopClosingBytesReader | 0xc000848540>{
                    s: "{\"error\":{\"code\":\"InvalidTemplate\",\"message\":\"Deployment template validation failed: 'The following parameters were supplied, but do not correspond to any parameters defined in the template: 'tags'. The parameters defined in the template are: 'clusterName, identities, openshiftControlPlaneVersionId, openshiftNodePoolVersionId, persistTagValue, usePooledIdentities'. Please see https://aka.ms/arm-pass-parameter-values for usage details.'.\",\"additionalInfo\":[{\"type\":\"TemplateViolation\",\"info\":{\"lineNumber\":0,\"linePosition\":0,\"path\":\"\"}}]}}",
                    i: 0,
                },
                ContentLength: 542,
                TransferEncoding: nil,
                Close: false,
                Uncompress...

Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.

Learn more here: https://onsi.github.io/gomega/#adjusting-output

occurred

fail [github.com/Azure/ARO-HCP/test/e2e/gpu_nodepools_create_delete.go:100]: Unexpected error:
    <*fmt.wrapError | 0xc0011369c0>: 
    failed creating deployment "aro-hcp-demo" in resourcegroup="gpu-nodepools-NC4asT4v3-6t29q4": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourcegroups/gpu-nodepools-NC4asT4v3-6t29q4/providers/Microsoft.Resources/deployments/aro-hcp-demo
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidTemplate
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidTemplate",
        "message": "Deployment template validation failed: 'The following parameters were supplied, but do not correspond to any parameters defined in the template: 'tags'. The parameters defined in the template are: 'clusterName, identities, openshiftControlPlaneVersionId, openshiftNodePoolVersionId, persistTagValue, usePooledIdentities'. Please see https://aka.ms/arm-pass-parameter-values for usage details.'.",
        "additionalInfo": [
          {
            "type": "TemplateViolation",
            "info": {
              "lineNumber": 0,
              "linePosition": 0,
              "path": ""
            }
          }
        ]
      }
    }
    --------------------------------------------------------------------------------
    
    {
        msg: "failed creating deployment \"aro-hcp-demo\" in resourcegroup=\"gpu-nodepools-NC4asT4v3-6t29q4\": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourcegroups/gpu-nodepools-NC4asT4v3-6t29q4/providers/Microsoft.Resources/deployments/aro-hcp-demo\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: InvalidTemplate\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"InvalidTemplate\",\n    \"message\": \"Deployment template validation failed: 'The following parameters were supplied, but do not correspond to any parameters defined in the template: 'tags'. The parameters defined in the template are: 'clusterName, identities, openshiftControlPlaneVersionId, openshiftNodePoolVersionId, persistTagValue, usePooledIdentities'. Please see https://aka.ms/arm-pass-parameter-values for usage details.'.\",\n    \"additionalInfo\": [\n      {\n        \"type\": \"TemplateViolation\",\n        \"info\": {\n          \"lineNumber\": 0,\n          \"linePosition\": 0,\n          \"path\": \"\"\n        }\n      }\n    ]\n  }\n}\n--------------------------------------------------------------------------------\n",
        err: <*exported.ResponseError | 0xc000eb44e0>{
            ErrorCode: "InvalidTemplate",
            StatusCode: 400,
            RawResponse: {
                Status: "400 Bad Request",
                StatusCode: 400,
                Proto: "HTTP/2.0",
                ProtoMajor: 2,
                ProtoMinor: 0,
                Header: {
                    "X-Ms-Failure-Cause": ["gateway"],
                    "X-Ms-Ratelimit-Remaining-Subscription-Global-Writes": ["2999"],
                    "X-Ms-Ratelimit-Remaining-Subscription-Writes": ["199"],
                    "X-Ms-Correlation-Request-Id": [
                        "6f0859fb-712e-493d-b9f3-1feb834d30d5",
                    ],
                    "X-Msedge-Ref": [
                        "Ref A: 0385E8E79B674F5DB2E6A6AC40C9C59D Ref B: CHI261140506042 Ref C: 2026-03-10T11:15:34Z",
                    ],
                    "Cache-Control": ["no-cache"],
                    "Pragma": ["no-cache"],
                    "Content-Length": ["542"],
                    "Expires": ["-1"],
                    "X-Ms-Request-Id": [
                        "6f0859fb-712e-493d-b9f3-1feb834d30d5",
                    ],
                    "X-Content-Type-Options": ["nosniff"],
                    "X-Cache": ["CONFIG_NOCACHE"],
                    "Date": [
                        "Tue, 10 Mar 2026 11:15:34 GMT",
                    ],
                    "Content-Type": [
                        "application/json; charset=utf-8",
                    ],
                    "X-Ms-Routing-Request-Id": [
                        "UKSOUTH:20260310T111535Z:6f0859fb-712e-493d-b9f3-1feb834d30d5",
                    ],
                    "Strict-Transport-Security": [
                        "max-age=31536000; includeSubDomains",
                    ],
                },
                Body: <*exported.nopClosingBytesReader | 0xc001136900>{
                    s: "{\"error\":{\"code\":\"InvalidTemplate\",\"message\":\"Deployment template validation failed: 'The following parameters were supplied, but do not correspond to any parameters defined in the template: 'tags'. The parameters defined in the template are: 'clusterName, identities, openshiftControlPlaneVersionId, openshiftNodePoolVersionId, persistTagValue, usePooledIdentities'. Please see https://aka.ms/arm-pass-parameter-values for usage details.'.\",\"additionalInfo\":[{\"type\":\"TemplateViolation\",\"info\":{\"lineNumber\":0,\"linePosition\":0,\"path\":\"\"}}]}}",
                    i: 0,
                },
                ContentLength: 542,
                TransferEncoding: nil,
                Close: false,
                Uncompress...

Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.

Learn more here: https://onsi.github.io/gomega/#adjusting-output

occurred
fail [github.com/Azure/ARO-HCP/test/e2e/gpu_nodepools_create_delete.go:100]: Unexpected error:
    <*fmt.wrapError | 0xc0011369c0>: 
    failed creating deployment "aro-hcp-demo" in resourcegroup="gpu-nodepools-NC4asT4v3-6t29q4": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourcegroups/gpu-nodepools-NC4asT4v3-6t29q4/providers/Microsoft.Resources/deployments/aro-hcp-demo
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidTemplate
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidTemplate",
        "message": "Deployment template validation failed: 'The following parameters were supplied, but do not correspond to any parameters defined in the template: 'tags'. The parameters defined in the template are: 'clusterName, identities, openshiftControlPlaneVersionId, openshiftNodePoolVersionId, persistTagValue, usePooledIdentities'. Please see https://aka.ms/arm-pass-parameter-values for usage details.'.",
        "additionalInfo": [
          {
            "type": "TemplateViolation",
            "info": {
              "lineNumber": 0,
              "linePosition": 0,
              "path": ""
            }
          }
        ]
      }
    }
    --------------------------------------------------------------------------------
    
    {
        msg: "failed creating deployment \"aro-hcp-demo\" in resourcegroup=\"gpu-nodepools-NC4asT4v3-6t29q4\": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourcegroups/gpu-nodepools-NC4asT4v3-6t29q4/providers/Microsoft.Resources/deployments/aro-hcp-demo\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: InvalidTemplate\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"InvalidTemplate\",\n    \"message\": \"Deployment template validation failed: 'The following parameters were supplied, but do not correspond to any parameters defined in the template: 'tags'. The parameters defined in the template are: 'clusterName, identities, openshiftControlPlaneVersionId, openshiftNodePoolVersionId, persistTagValue, usePooledIdentities'. Please see https://aka.ms/arm-pass-parameter-values for usage details.'.\",\n    \"additionalInfo\": [\n      {\n        \"type\": \"TemplateViolation\",\n        \"info\": {\n          \"lineNumber\": 0,\n          \"linePosition\": 0,\n          \"path\": \"\"\n        }\n      }\n    ]\n  }\n}\n--------------------------------------------------------------------------------\n",
        err: <*exported.ResponseError | 0xc000eb44e0>{
            ErrorCode: "InvalidTemplate",
            StatusCode: 400,
            RawResponse: {
                Status: "400 Bad Request",
                StatusCode: 400,
                Proto: "HTTP/2.0",
                ProtoMajor: 2,
                ProtoMinor: 0,
                Header: {
                    "X-Ms-Failure-Cause": ["gateway"],
                    "X-Ms-Ratelimit-Remaining-Subscription-Global-Writes": ["2999"],
                    "X-Ms-Ratelimit-Remaining-Subscription-Writes": ["199"],
                    "X-Ms-Correlation-Request-Id": [
                        "6f0859fb-712e-493d-b9f3-1feb834d30d5",
                    ],
                    "X-Msedge-Ref": [
                        "Ref A: 0385E8E79B674F5DB2E6A6AC40C9C59D Ref B: CHI261140506042 Ref C: 2026-03-10T11:15:34Z",
                    ],
                    "Cache-Control": ["no-cache"],
                    "Pragma": ["no-cache"],
                    "Content-Length": ["542"],
                    "Expires": ["-1"],
                    "X-Ms-Request-Id": [
                        "6f0859fb-712e-493d-b9f3-1feb834d30d5",
                    ],
                    "X-Content-Type-Options": ["nosniff"],
                    "X-Cache": ["CONFIG_NOCACHE"],
                    "Date": [
                        "Tue, 10 Mar 2026 11:15:34 GMT",
                    ],
                    "Content-Type": [
                        "application/json; charset=utf-8",
                    ],
                    "X-Ms-Routing-Request-Id": [
                        "UKSOUTH:20260310T111535Z:6f0859fb-712e-493d-b9f3-1feb834d30d5",
                    ],
                    "Strict-Transport-Security": [
                        "max-age=31536000; includeSubDomains",
                    ],
                },
                Body: <*exported.nopClosingBytesReader | 0xc001136900>{
                    s: "{\"error\":{\"code\":\"InvalidTemplate\",\"message\":\"Deployment template validation failed: 'The following parameters were supplied, but do not correspond to any parameters defined in the template: 'tags'. The parameters defined in the template are: 'clusterName, identities, openshiftControlPlaneVersionId, openshiftNodePoolVersionId, persistTagValue, usePooledIdentities'. Please see https://aka.ms/arm-pass-parameter-values for usage details.'.\",\"additionalInfo\":[{\"type\":\"TemplateViolation\",\"info\":{\"lineNumber\":0,\"linePosition\":0,\"path\":\"\"}}]}}",
                    i: 0,
                },
                ContentLength: 542,
                TransferEncoding: nil,
                Close: false,
                Uncompress...

Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.

Learn more here: https://onsi.github.io/gomega/#adjusting-output

occurred

Contributing tests (1)

Failed At	Job	Test	Support
e2e	periodic-ci-Azure-ARO-HCP-main-periodic-integration-e2e-parallel	HCP Nodepools GPU instances creates and deletes vm type NC4asT4v3 in a single cluster	2

Affected runs (2)

Date (UTC)	Associated PR	Prow job
2026-03-10 15:24	n/a	2031390723315601408
2026-03-10 11:07	n/a	2031326084175237120

VerifyHostedControlPlaneZStreamUpgradeOnly(initial=4

full failure pattern:

VerifyHostedControlPlaneZStreamUpgradeOnly(initial=4

Signal:

e2e

none

Full failure examples (2)

fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:109]: Timed out after 2400.003s.
Expected success, but got an error:
    <*errors.joinError | 0xc001069d10>: 
    VerifyHostedControlPlaneZStreamUpgradeOnly(initial=4.20.15) failed: no version in clusterversion/version status.history is greater than initial "4.20.15"
    {
        errs: [
            <*fmt.wrapError | 0xc00052cbc0>{
                msg: "VerifyHostedControlPlaneZStreamUpgradeOnly(initial=4.20.15) failed: no version in clusterversion/version status.history is greater than initial \"4.20.15\"",
                err: <*errors.errorString | 0xc0008a6470>{
                    s: "no version in clusterversion/version status.history is greater than initial \"4.20.15\"",
                },
            },
        ],
    }
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:109]: Timed out after 2400.003s.
Expected success, but got an error:
    <*errors.joinError | 0xc001069d10>: 
    VerifyHostedControlPlaneZStreamUpgradeOnly(initial=4.20.15) failed: no version in clusterversion/version status.history is greater than initial "4.20.15"
    {
        errs: [
            <*fmt.wrapError | 0xc00052cbc0>{
                msg: "VerifyHostedControlPlaneZStreamUpgradeOnly(initial=4.20.15) failed: no version in clusterversion/version status.history is greater than initial \"4.20.15\"",
                err: <*errors.errorString | 0xc0008a6470>{
                    s: "no version in clusterversion/version status.history is greater than initial \"4.20.15\"",
                },
            },
        ],
    }

fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:109]: Timed out after 2400.007s.
Expected success, but got an error:
    <*errors.joinError | 0xc000f034d0>: 
    VerifyHostedControlPlaneZStreamUpgradeOnly(initial=4.20.15) failed: no version in clusterversion/version status.history is greater than initial "4.20.15"
    {
        errs: [
            <*fmt.wrapError | 0xc00051c6c0>{
                msg: "VerifyHostedControlPlaneZStreamUpgradeOnly(initial=4.20.15) failed: no version in clusterversion/version status.history is greater than initial \"4.20.15\"",
                err: <*errors.errorString | 0xc00073e8e0>{
                    s: "no version in clusterversion/version status.history is greater than initial \"4.20.15\"",
                },
            },
        ],
    }
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:109]: Timed out after 2400.007s.
Expected success, but got an error:
    <*errors.joinError | 0xc000f034d0>: 
    VerifyHostedControlPlaneZStreamUpgradeOnly(initial=4.20.15) failed: no version in clusterversion/version status.history is greater than initial "4.20.15"
    {
        errs: [
            <*fmt.wrapError | 0xc00051c6c0>{
                msg: "VerifyHostedControlPlaneZStreamUpgradeOnly(initial=4.20.15) failed: no version in clusterversion/version status.history is greater than initial \"4.20.15\"",
                err: <*errors.errorString | 0xc00073e8e0>{
                    s: "no version in clusterversion/version status.history is greater than initial \"4.20.15\"",
                },
            },
        ],
    }

Contributing tests (1)

Failed At	Job	Test	Support
e2e	periodic-ci-Azure-ARO-HCP-main-periodic-integration-e2e-parallel	Control plane automated z-stream upgrade with candidate channel should perform an automated control plane z-stream upgrade using the candidate channel	2

Affected runs (2)

Date (UTC)	Associated PR	Prow job
2026-03-12 22:14	n/a	2032218705995239424
2026-03-12 21:14	n/a	2032203554982203392

ERROR CODE: InvalidRequestContent; detail message node pool <nodepool> '4.20.15' must not be greater than Control Plane version '4.20.8'; provider Microsoft.RedHatOpenShift

full failure pattern:

ERROR CODE: InvalidRequestContent; detail message node pool <nodepool> '4.20.15' must not be greater than Control Plane version '4.20.8'; provider Microsoft.RedHatOpenShift

Signal:

e2e

none

Full failure examples (3)

fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create.go:118]: Unexpected error:
    <*fmt.wrapError | 0xc00171c6c0>: 
    failed to create NodePool np-1: failed starting nodepool creation "np-1" for cluster "basic-hcp-cluster" in resourcegroup="basic-cluster-g975c2": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/basic-cluster-g975c2/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/basic-hcp-cluster/nodePools/np-1
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Node pool version '4.20.15' must not be greater than Control Plane version '4.20.8'"
      }
    }
    --------------------------------------------------------------------------------
    
    {
        msg: "failed to create NodePool np-1: failed starting nodepool creation \"np-1\" for cluster \"basic-hcp-cluster\" in resourcegroup=\"basic-cluster-g975c2\": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/basic-cluster-g975c2/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/basic-hcp-cluster/nodePools/np-1\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: InvalidRequestContent\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"InvalidRequestContent\",\n    \"message\": \"Node pool version '4.20.15' must not be greater than Control Plane version '4.20.8'\"\n  }\n}\n--------------------------------------------------------------------------------\n",
        err: <*fmt.wrapError | 0xc00171c6a0>{
            msg: "failed starting nodepool creation \"np-1\" for cluster \"basic-hcp-cluster\" in resourcegroup=\"basic-cluster-g975c2\": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/basic-cluster-g975c2/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/basic-hcp-cluster/nodePools/np-1\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: InvalidRequestContent\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"InvalidRequestContent\",\n    \"message\": \"Node pool version '4.20.15' must not be greater than Control Plane version '4.20.8'\"\n  }\n}\n--------------------------------------------------------------------------------\n",
            err: <*exported.ResponseError | 0xc0009663c0>{
                ErrorCode: "InvalidRequestContent",
                StatusCode: 400,
                RawResponse: {
                    Status: "400 Bad Request",
                    StatusCode: 400,
                    Proto: "HTTP/2.0",
                    ProtoMajor: 2,
                    ProtoMinor: 0,
                    Header: {
                        "Content-Length": ["170"],
                        "X-Ms-Correlation-Request-Id": [
                            "cf3a6b30-b9f9-40f5-860b-db7a4db99729",
                        ],
                        "X-Cache": ["CONFIG_NOCACHE"],
                        "Expires": ["-1"],
                        "X-Ms-Ratelimit-Remaining-Subscription-Writes": ["199"],
                        "X-Ms-Routing-Request-Id": [
                            "UKSOUTH:20260312T231200Z:cf3a6b30-b9f9-40f5-860b-db7a4db99729",
                        ],
                        "Strict-Transport-Security": [
                            "max-age=31536000; includeSubDomains",
                        ],
                        "X-Msedge-Ref": [
                            "Ref A: 1334633E7ABC4C9D961015632B4669DE Ref B: CHI261140502060 Ref C: 2026-03-12T23:11:59Z",
                        ],
                        "Date": [
                            "Thu, 12 Mar 2026 23:11:59 GMT",
                        ],
                        "X-Envoy-Upstream-Service-Time": ["201"],
                        "X-Ms-Operation-Identifier": [
                            "appId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,tenantId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,objectId=c209f8df-52ae-48fb-98ea-380f58b04652/uksouth/67d0f8c8-def2-4e27-b339-fb62d5a3d936",
                        ],
                        "X-Ms-Ratelimit-Remaining-Subscription-Global-Writes": ["2999"],
                        "X-Content-Type-Options": ["nosniff"],
                        "Content-Type": ["application/json"],
                        "X-Ms-Error-Code": [
                            "InvalidRequestContent",
                        ],
                        "X-Ms-Request-Id": [
                            "f425ba51-7086-4447-8016-171717a6c5ab",
                    ...

Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.

Learn more here: https://onsi.github.io/gomega/#adjusting-output

occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create.go:118]: Unexpected error:
    <*fmt.wrapError | 0xc00171c6c0>: 
    failed to create NodePool np-1: failed starting nodepool creation "np-1" for cluster "basic-hcp-cluster" in resourcegroup="basic-cluster-g975c2": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/basic-cluster-g975c2/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/basic-hcp-cluster/nodePools/np-1
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Node pool version '4.20.15' must not be greater than Control Plane version '4.20.8'"
      }
    }
    --------------------------------------------------------------------------------
    
    {
        msg: "failed to create NodePool np-1: failed starting nodepool creation \"np-1\" for cluster \"basic-hcp-cluster\" in resourcegroup=\"basic-cluster-g975c2\": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/basic-cluster-g975c2/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/basic-hcp-cluster/nodePools/np-1\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: InvalidRequestContent\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"InvalidRequestContent\",\n    \"message\": \"Node pool version '4.20.15' must not be greater than Control Plane version '4.20.8'\"\n  }\n}\n--------------------------------------------------------------------------------\n",
        err: <*fmt.wrapError | 0xc00171c6a0>{
            msg: "failed starting nodepool creation \"np-1\" for cluster \"basic-hcp-cluster\" in resourcegroup=\"basic-cluster-g975c2\": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/basic-cluster-g975c2/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/basic-hcp-cluster/nodePools/np-1\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: InvalidRequestContent\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"InvalidRequestContent\",\n    \"message\": \"Node pool version '4.20.15' must not be greater than Control Plane version '4.20.8'\"\n  }\n}\n--------------------------------------------------------------------------------\n",
            err: <*exported.ResponseError | 0xc0009663c0>{
                ErrorCode: "InvalidRequestContent",
                StatusCode: 400,
                RawResponse: {
                    Status: "400 Bad Request",
                    StatusCode: 400,
                    Proto: "HTTP/2.0",
                    ProtoMajor: 2,
                    ProtoMinor: 0,
                    Header: {
                        "Content-Length": ["170"],
                        "X-Ms-Correlation-Request-Id": [
                            "cf3a6b30-b9f9-40f5-860b-db7a4db99729",
                        ],
                        "X-Cache": ["CONFIG_NOCACHE"],
                        "Expires": ["-1"],
                        "X-Ms-Ratelimit-Remaining-Subscription-Writes": ["199"],
                        "X-Ms-Routing-Request-Id": [
                            "UKSOUTH:20260312T231200Z:cf3a6b30-b9f9-40f5-860b-db7a4db99729",
                        ],
                        "Strict-Transport-Security": [
                            "max-age=31536000; includeSubDomains",
                        ],
                        "X-Msedge-Ref": [
                            "Ref A: 1334633E7ABC4C9D961015632B4669DE Ref B: CHI261140502060 Ref C: 2026-03-12T23:11:59Z",
                        ],
                        "Date": [
                            "Thu, 12 Mar 2026 23:11:59 GMT",
                        ],
                        "X-Envoy-Upstream-Service-Time": ["201"],
                        "X-Ms-Operation-Identifier": [
                            "appId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,tenantId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,objectId=c209f8df-52ae-48fb-98ea-380f58b04652/uksouth/67d0f8c8-def2-4e27-b339-fb62d5a3d936",
                        ],
                        "X-Ms-Ratelimit-Remaining-Subscription-Global-Writes": ["2999"],
                        "X-Content-Type-Options": ["nosniff"],
                        "Content-Type": ["application/json"],
                        "X-Ms-Error-Code": [
                            "InvalidRequestContent",
                        ],
                        "X-Ms-Request-Id": [
                            "f425ba51-7086-4447-8016-171717a6c5ab",
                    ...

Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.

Learn more here: https://onsi.github.io/gomega/#adjusting-output

occurred

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:114]: Unexpected error:
    <*fmt.wrapError | 0xc001408120>: 
    failed to create NodePool np-1: failed starting nodepool creation "np-1" for cluster "tls-endpoint-hcp-cluster" in resourcegroup="tls-endpoint-cluster-8t5md5": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/tls-endpoint-cluster-8t5md5/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/tls-endpoint-hcp-cluster/nodePools/np-1
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Node pool version '4.20.15' must not be greater than Control Plane version '4.20.8'"
      }
    }
    --------------------------------------------------------------------------------
    
    {
        msg: "failed to create NodePool np-1: failed starting nodepool creation \"np-1\" for cluster \"tls-endpoint-hcp-cluster\" in resourcegroup=\"tls-endpoint-cluster-8t5md5\": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/tls-endpoint-cluster-8t5md5/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/tls-endpoint-hcp-cluster/nodePools/np-1\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: InvalidRequestContent\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"InvalidRequestContent\",\n    \"message\": \"Node pool version '4.20.15' must not be greater than Control Plane version '4.20.8'\"\n  }\n}\n--------------------------------------------------------------------------------\n",
        err: <*fmt.wrapError | 0xc001408100>{
            msg: "failed starting nodepool creation \"np-1\" for cluster \"tls-endpoint-hcp-cluster\" in resourcegroup=\"tls-endpoint-cluster-8t5md5\": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/tls-endpoint-cluster-8t5md5/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/tls-endpoint-hcp-cluster/nodePools/np-1\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: InvalidRequestContent\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"InvalidRequestContent\",\n    \"message\": \"Node pool version '4.20.15' must not be greater than Control Plane version '4.20.8'\"\n  }\n}\n--------------------------------------------------------------------------------\n",
            err: <*exported.ResponseError | 0xc000c90fc0>{
                ErrorCode: "InvalidRequestContent",
                StatusCode: 400,
                RawResponse: {
                    Status: "400 Bad Request",
                    StatusCode: 400,
                    Proto: "HTTP/2.0",
                    ProtoMajor: 2,
                    ProtoMinor: 0,
                    Header: {
                        "X-Envoy-Upstream-Service-Time": ["2649"],
                        "X-Ms-Ratelimit-Remaining-Subscription-Global-Writes": ["2999"],
                        "X-Ms-Routing-Request-Id": [
                            "UKSOUTH:20260312T231012Z:014d9526-d846-4eb1-91c9-ca8a5c50bfbd",
                        ],
                        "X-Msedge-Ref": [
                            "Ref A: 8080CB14631643F1BD10EDD83999FADD Ref B: CHI261140513054 Ref C: 2026-03-12T23:10:10Z",
                        ],
                        "Pragma": ["no-cache"],
                        "Content-Length": ["170"],
                        "Expires": ["-1"],
                        "Strict-Transport-Security": [
                            "max-age=31536000; includeSubDomains",
                        ],
                        "X-Ms-Operation-Identifier": [
                            "appId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,tenantId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,objectId=c209f8df-52ae-48fb-98ea-380f58b04652/uksouth/9e990a7a-bc1c-4fa5-84e6-6d350f3c8cfe",
                        ],
                        "X-Content-Type-Options": ["nosniff"],
                        "X-Cache": ["CONFIG_NOCACHE"],
                        "Date": [
                            "Thu, 12 Mar 2026 23:10:12 GMT",
                        ],
                        "X-Ms-Error-Code": [
                            "InvalidRequestContent",
                        ],
                        "X-Ms-Ratelimit-Remaining-Subscription-Writes": ["199"],
                        "X-Ms-Correlation-Request-Id": [
                            "014d9526-d846-4eb1-91c9-ca8a5c50bfbd",
                        ],
                        "Cache-Control": ["no-cache"],
                        "Content-Ty...

Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.

Learn more here: https://onsi.github.io/gomega/#adjusting-output

occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:114]: Unexpected error:
    <*fmt.wrapError | 0xc001408120>: 
    failed to create NodePool np-1: failed starting nodepool creation "np-1" for cluster "tls-endpoint-hcp-cluster" in resourcegroup="tls-endpoint-cluster-8t5md5": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/tls-endpoint-cluster-8t5md5/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/tls-endpoint-hcp-cluster/nodePools/np-1
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Node pool version '4.20.15' must not be greater than Control Plane version '4.20.8'"
      }
    }
    --------------------------------------------------------------------------------
    
    {
        msg: "failed to create NodePool np-1: failed starting nodepool creation \"np-1\" for cluster \"tls-endpoint-hcp-cluster\" in resourcegroup=\"tls-endpoint-cluster-8t5md5\": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/tls-endpoint-cluster-8t5md5/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/tls-endpoint-hcp-cluster/nodePools/np-1\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: InvalidRequestContent\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"InvalidRequestContent\",\n    \"message\": \"Node pool version '4.20.15' must not be greater than Control Plane version '4.20.8'\"\n  }\n}\n--------------------------------------------------------------------------------\n",
        err: <*fmt.wrapError | 0xc001408100>{
            msg: "failed starting nodepool creation \"np-1\" for cluster \"tls-endpoint-hcp-cluster\" in resourcegroup=\"tls-endpoint-cluster-8t5md5\": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/tls-endpoint-cluster-8t5md5/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/tls-endpoint-hcp-cluster/nodePools/np-1\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: InvalidRequestContent\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"InvalidRequestContent\",\n    \"message\": \"Node pool version '4.20.15' must not be greater than Control Plane version '4.20.8'\"\n  }\n}\n--------------------------------------------------------------------------------\n",
            err: <*exported.ResponseError | 0xc000c90fc0>{
                ErrorCode: "InvalidRequestContent",
                StatusCode: 400,
                RawResponse: {
                    Status: "400 Bad Request",
                    StatusCode: 400,
                    Proto: "HTTP/2.0",
                    ProtoMajor: 2,
                    ProtoMinor: 0,
                    Header: {
                        "X-Envoy-Upstream-Service-Time": ["2649"],
                        "X-Ms-Ratelimit-Remaining-Subscription-Global-Writes": ["2999"],
                        "X-Ms-Routing-Request-Id": [
                            "UKSOUTH:20260312T231012Z:014d9526-d846-4eb1-91c9-ca8a5c50bfbd",
                        ],
                        "X-Msedge-Ref": [
                            "Ref A: 8080CB14631643F1BD10EDD83999FADD Ref B: CHI261140513054 Ref C: 2026-03-12T23:10:10Z",
                        ],
                        "Pragma": ["no-cache"],
                        "Content-Length": ["170"],
                        "Expires": ["-1"],
                        "Strict-Transport-Security": [
                            "max-age=31536000; includeSubDomains",
                        ],
                        "X-Ms-Operation-Identifier": [
                            "appId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,tenantId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,objectId=c209f8df-52ae-48fb-98ea-380f58b04652/uksouth/9e990a7a-bc1c-4fa5-84e6-6d350f3c8cfe",
                        ],
                        "X-Content-Type-Options": ["nosniff"],
                        "X-Cache": ["CONFIG_NOCACHE"],
                        "Date": [
                            "Thu, 12 Mar 2026 23:10:12 GMT",
                        ],
                        "X-Ms-Error-Code": [
                            "InvalidRequestContent",
                        ],
                        "X-Ms-Ratelimit-Remaining-Subscription-Writes": ["199"],
                        "X-Ms-Correlation-Request-Id": [
                            "014d9526-d846-4eb1-91c9-ca8a5c50bfbd",
                        ],
                        "Cache-Control": ["no-cache"],
                        "Content-Ty...

Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.

Learn more here: https://onsi.github.io/gomega/#adjusting-output

occurred

fail [github.com/Azure/ARO-HCP/test/e2e/simple_negative_cases.go:102]: Unexpected error:
    <*fmt.wrapError | 0xc00087e580>: 
    failed to create NodePool np-1: failed starting nodepool creation "np-1" for cluster "negative-tests-cluster" in resourcegroup="rg-negative-tests-mq56rt": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-negative-tests-mq56rt/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/negative-tests-cluster/nodePools/np-1
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Node pool version '4.20.15' must not be greater than Control Plane version '4.20.8'"
      }
    }
    --------------------------------------------------------------------------------
    
    {
        msg: "failed to create NodePool np-1: failed starting nodepool creation \"np-1\" for cluster \"negative-tests-cluster\" in resourcegroup=\"rg-negative-tests-mq56rt\": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-negative-tests-mq56rt/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/negative-tests-cluster/nodePools/np-1\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: InvalidRequestContent\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"InvalidRequestContent\",\n    \"message\": \"Node pool version '4.20.15' must not be greater than Control Plane version '4.20.8'\"\n  }\n}\n--------------------------------------------------------------------------------\n",
        err: <*fmt.wrapError | 0xc00087e560>{
            msg: "failed starting nodepool creation \"np-1\" for cluster \"negative-tests-cluster\" in resourcegroup=\"rg-negative-tests-mq56rt\": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-negative-tests-mq56rt/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/negative-tests-cluster/nodePools/np-1\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: InvalidRequestContent\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"InvalidRequestContent\",\n    \"message\": \"Node pool version '4.20.15' must not be greater than Control Plane version '4.20.8'\"\n  }\n}\n--------------------------------------------------------------------------------\n",
            err: <*exported.ResponseError | 0xc000e0e090>{
                ErrorCode: "InvalidRequestContent",
                StatusCode: 400,
                RawResponse: {
                    Status: "400 Bad Request",
                    StatusCode: 400,
                    Proto: "HTTP/2.0",
                    ProtoMajor: 2,
                    ProtoMinor: 0,
                    Header: {
                        "Cache-Control": ["no-cache"],
                        "Content-Type": ["application/json"],
                        "X-Ms-Request-Id": [
                            "95277bb3-59de-4af9-a6be-d794c22ffea6",
                        ],
                        "X-Ms-Ratelimit-Remaining-Subscription-Writes": ["199"],
                        "X-Content-Type-Options": ["nosniff"],
                        "X-Cache": ["CONFIG_NOCACHE"],
                        "X-Msedge-Ref": [
                            "Ref A: DECDB7E93FD643FFB79BD2D9D72D81DF Ref B: CHI261140510025 Ref C: 2026-03-12T22:52:58Z",
                        ],
                        "Pragma": ["no-cache"],
                        "Content-Length": ["170"],
                        "Expires": ["-1"],
                        "X-Ms-Routing-Request-Id": [
                            "UKSOUTH:20260312T225305Z:9b9678c6-264f-423d-bafc-7733fb8aa855",
                        ],
                        "X-Ms-Error-Code": [
                            "InvalidRequestContent",
                        ],
                        "X-Envoy-Upstream-Service-Time": ["7627"],
                        "X-Ms-Operation-Identifier": [
                            "appId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,tenantId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,objectId=c209f8df-52ae-48fb-98ea-380f58b04652/uksouth/a50444fe-08f0-4f32-b20a-20c19b8c9e0e",
                        ],
                        "X-Ms-Ratelimit-Remaining-Subscription-Global-Writes": ["2999"],
                        "X-Ms-Correlation-Request-Id": [
                            "9b9678c6-264f-423d-bafc-7733fb8aa855",
                        ],
                        "Strict-Transport-Security": [
                            "max-age=31536000; includeSubDomains",
  ...

Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.

Learn more here: https://onsi.github.io/gomega/#adjusting-output

occurred
fail [github.com/Azure/ARO-HCP/test/e2e/simple_negative_cases.go:102]: Unexpected error:
    <*fmt.wrapError | 0xc00087e580>: 
    failed to create NodePool np-1: failed starting nodepool creation "np-1" for cluster "negative-tests-cluster" in resourcegroup="rg-negative-tests-mq56rt": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-negative-tests-mq56rt/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/negative-tests-cluster/nodePools/np-1
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Node pool version '4.20.15' must not be greater than Control Plane version '4.20.8'"
      }
    }
    --------------------------------------------------------------------------------
    
    {
        msg: "failed to create NodePool np-1: failed starting nodepool creation \"np-1\" for cluster \"negative-tests-cluster\" in resourcegroup=\"rg-negative-tests-mq56rt\": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-negative-tests-mq56rt/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/negative-tests-cluster/nodePools/np-1\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: InvalidRequestContent\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"InvalidRequestContent\",\n    \"message\": \"Node pool version '4.20.15' must not be greater than Control Plane version '4.20.8'\"\n  }\n}\n--------------------------------------------------------------------------------\n",
        err: <*fmt.wrapError | 0xc00087e560>{
            msg: "failed starting nodepool creation \"np-1\" for cluster \"negative-tests-cluster\" in resourcegroup=\"rg-negative-tests-mq56rt\": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-negative-tests-mq56rt/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/negative-tests-cluster/nodePools/np-1\n--------------------------------------------------------------------------------\nRESPONSE 400: 400 Bad Request\nERROR CODE: InvalidRequestContent\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"InvalidRequestContent\",\n    \"message\": \"Node pool version '4.20.15' must not be greater than Control Plane version '4.20.8'\"\n  }\n}\n--------------------------------------------------------------------------------\n",
            err: <*exported.ResponseError | 0xc000e0e090>{
                ErrorCode: "InvalidRequestContent",
                StatusCode: 400,
                RawResponse: {
                    Status: "400 Bad Request",
                    StatusCode: 400,
                    Proto: "HTTP/2.0",
                    ProtoMajor: 2,
                    ProtoMinor: 0,
                    Header: {
                        "Cache-Control": ["no-cache"],
                        "Content-Type": ["application/json"],
                        "X-Ms-Request-Id": [
                            "95277bb3-59de-4af9-a6be-d794c22ffea6",
                        ],
                        "X-Ms-Ratelimit-Remaining-Subscription-Writes": ["199"],
                        "X-Content-Type-Options": ["nosniff"],
                        "X-Cache": ["CONFIG_NOCACHE"],
                        "X-Msedge-Ref": [
                            "Ref A: DECDB7E93FD643FFB79BD2D9D72D81DF Ref B: CHI261140510025 Ref C: 2026-03-12T22:52:58Z",
                        ],
                        "Pragma": ["no-cache"],
                        "Content-Length": ["170"],
                        "Expires": ["-1"],
                        "X-Ms-Routing-Request-Id": [
                            "UKSOUTH:20260312T225305Z:9b9678c6-264f-423d-bafc-7733fb8aa855",
                        ],
                        "X-Ms-Error-Code": [
                            "InvalidRequestContent",
                        ],
                        "X-Envoy-Upstream-Service-Time": ["7627"],
                        "X-Ms-Operation-Identifier": [
                            "appId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,tenantId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,objectId=c209f8df-52ae-48fb-98ea-380f58b04652/uksouth/a50444fe-08f0-4f32-b20a-20c19b8c9e0e",
                        ],
                        "X-Ms-Ratelimit-Remaining-Subscription-Global-Writes": ["2999"],
                        "X-Ms-Correlation-Request-Id": [
                            "9b9678c6-264f-423d-bafc-7733fb8aa855",
                        ],
                        "Strict-Transport-Security": [
                            "max-age=31536000; includeSubDomains",
  ...

Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.

Learn more here: https://onsi.github.io/gomega/#adjusting-output

occurred

Contributing tests (9)

Failed At	Job	Test	Support
e2e	periodic-ci-Azure-ARO-HCP-main-periodic-integration-e2e-parallel	Authorized CIDRs Connectivity should allow API access only from authorized VM IP	1
e2e	periodic-ci-Azure-ARO-HCP-main-periodic-integration-e2e-parallel	Customer should be able to create a cluster with an external auth config and get the external auth config	1
e2e	periodic-ci-Azure-ARO-HCP-main-periodic-integration-e2e-parallel	Customer should be able to create a cluster with default autoscaling and a nodepool with autoscaling enabled	1
e2e	periodic-ci-Azure-ARO-HCP-main-periodic-integration-e2e-parallel	Customer should be able to create an HCP cluster	1
e2e	periodic-ci-Azure-ARO-HCP-main-periodic-integration-e2e-parallel	Customer should be able to create an HCP cluster and custom node pool osDisk size	1
e2e	periodic-ci-Azure-ARO-HCP-main-periodic-integration-e2e-parallel	Customer should be able to create an HCP cluster with custom autoscaling	1
e2e	periodic-ci-Azure-ARO-HCP-main-periodic-integration-e2e-parallel	Customer should be able to create node pools with ARM64-based VMs	1
e2e	periodic-ci-Azure-ARO-HCP-main-periodic-integration-e2e-parallel	Customer should create an HCP cluster and validate TLS certificates	1
e2e	periodic-ci-Azure-ARO-HCP-main-periodic-integration-e2e-parallel	Customer should not be able to perform various invalid operations on cluster resources	1

Affected runs (1)

Date (UTC)	Associated PR	Prow job
2026-03-12 22:14	n/a	2032218705995239424

ERROR CODE: InternalServerError; context ; provider Microsoft.Network

full failure pattern:

ERROR CODE: InternalServerError; context ; provider Microsoft.Network

Signal:

e2e

none

Full failure examples (2)

fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:93]: Unexpected error:
    <*fmt.wrapError | 0xc000c720a0>: 
    failed to create HCP cluster ea-cluster: failed waiting for cluster="ea-cluster" in resourcegroup="external-auth-cluster-qkwf4p" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0b05f658-bbb6-452d-b845-92443ec2c619
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0b05f658-bbb6-452d-b845-92443ec2c619",
      "name": "0b05f658-bbb6-452d-b845-92443ec2c619",
      "status": "Failed",
      "startTime": "2026-03-11T03:36:35.705780329Z",
      "endTime": "2026-03-11T03:38:07.574131728Z",
      "error": {
        "code": "InternalServerError",
        "message": "GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Network/locations/uksouth/operations/de3f3e34-ee55-4c5b-bdc7-7cbf3170faf2\n--------------------------------------------------------------------------------\nRESPONSE 404: 404 Not Found\nERROR CODE UNAVAILABLE\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"message\": \"Operation de3f3e34-ee55-4c5b-bdc7-7cbf3170faf2 not found.\",\n    \"details\": []\n  }\n}\n--------------------------------------------------------------------------------\n"
      }
    }
    --------------------------------------------------------------------------------
    
    {
        msg: "failed to create HCP cluster ea-cluster: failed waiting for cluster=\"ea-cluster\" in resourcegroup=\"external-auth-cluster-qkwf4p\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0b05f658-bbb6-452d-b845-92443ec2c619\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n  \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0b05f658-bbb6-452d-b845-92443ec2c619\",\n  \"name\": \"0b05f658-bbb6-452d-b845-92443ec2c619\",\n  \"status\": \"Failed\",\n  \"startTime\": \"2026-03-11T03:36:35.705780329Z\",\n  \"endTime\": \"2026-03-11T03:38:07.574131728Z\",\n  \"error\": {\n    \"code\": \"InternalServerError\",\n    \"message\": \"GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Network/locations/uksouth/operations/de3f3e34-ee55-4c5b-bdc7-7cbf3170faf2\\n--------------------------------------------------------------------------------\\nRESPONSE 404: 404 Not Found\\nERROR CODE UNAVAILABLE\\n--------------------------------------------------------------------------------\\n{\\n  \\\"error\\\": {\\n    \\\"message\\\": \\\"Operation de3f3e34-ee55-4c5b-bdc7-7cbf3170faf2 not found.\\\",\\n    \\\"details\\\": []\\n  }\\n}\\n--------------------------------------------------------------------------------\\n\"\n  }\n}\n--------------------------------------------------------------------------------\n",
        err: <*fmt.wrapError | 0xc000c72080>{
            msg: "failed waiting for cluster=\"ea-cluster\" in resourcegroup=\"external-auth-cluster-qkwf4p\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0b05f658-bbb6-452d-b845-92443ec2c619\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n  \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0b05f658-bbb6-452d-b845-92443ec2c619\",\n  \"name\": \"0b05f658-bbb6-452d-b845-92443ec2c619\",\n  \"status\": \"Failed\",\n  \"startTime\": \"2026-03-11T03:36:35.705780329Z\",\n  \"endTime\": \"2026-03-11T03:38:07.574131728Z\",\n  \"error\": {\n    \"code\": \"InternalServerError\",\n    \"message\": \"GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Network/locations/uksouth/operations/de3f3e34-ee55-4c5b-bdc7-7cbf3170faf2\\n--------------------------------------------------------------------------------\\nRESPONSE 404: 404 Not Found\\nERROR CODE UNAVAILABLE\\n--------------------------------------------------------------------------------\\n{\\n  \\\"error\\\": {\\n    \\\"message\\\": \\\"Operation de3f3e34-ee55-4c5b-bdc7-7cbf3170faf2 not found.\\\",\\n    \\\"details\\\": []\\n  }\\n}\\n--------------------------------------------------------------------------------\\n\"\n  }\n}\n--------------------------------------------------------------------------------\n",
            err: <*exported.ResponseError | 0xc00144fd10>{
                ErrorCode: "InternalServerError",
                StatusCode: 200,
                RawResponse: {
                    Status: "200 OK",
                    StatusCode: 200,
                    Proto: "HTTP/2.0",
                    ProtoMajor: 2,
                    ProtoMinor: 0,
                    Header: {
                        "X-Ms-Ratelimit-Remaining-Subscription-Reads": ["249"],
...

Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.

Learn more here: https://onsi.github.io/gomega/#adjusting-output

occurred
fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:93]: Unexpected error:
    <*fmt.wrapError | 0xc000c720a0>: 
    failed to create HCP cluster ea-cluster: failed waiting for cluster="ea-cluster" in resourcegroup="external-auth-cluster-qkwf4p" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0b05f658-bbb6-452d-b845-92443ec2c619
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0b05f658-bbb6-452d-b845-92443ec2c619",
      "name": "0b05f658-bbb6-452d-b845-92443ec2c619",
      "status": "Failed",
      "startTime": "2026-03-11T03:36:35.705780329Z",
      "endTime": "2026-03-11T03:38:07.574131728Z",
      "error": {
        "code": "InternalServerError",
        "message": "GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Network/locations/uksouth/operations/de3f3e34-ee55-4c5b-bdc7-7cbf3170faf2\n--------------------------------------------------------------------------------\nRESPONSE 404: 404 Not Found\nERROR CODE UNAVAILABLE\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"message\": \"Operation de3f3e34-ee55-4c5b-bdc7-7cbf3170faf2 not found.\",\n    \"details\": []\n  }\n}\n--------------------------------------------------------------------------------\n"
      }
    }
    --------------------------------------------------------------------------------
    
    {
        msg: "failed to create HCP cluster ea-cluster: failed waiting for cluster=\"ea-cluster\" in resourcegroup=\"external-auth-cluster-qkwf4p\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0b05f658-bbb6-452d-b845-92443ec2c619\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n  \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0b05f658-bbb6-452d-b845-92443ec2c619\",\n  \"name\": \"0b05f658-bbb6-452d-b845-92443ec2c619\",\n  \"status\": \"Failed\",\n  \"startTime\": \"2026-03-11T03:36:35.705780329Z\",\n  \"endTime\": \"2026-03-11T03:38:07.574131728Z\",\n  \"error\": {\n    \"code\": \"InternalServerError\",\n    \"message\": \"GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Network/locations/uksouth/operations/de3f3e34-ee55-4c5b-bdc7-7cbf3170faf2\\n--------------------------------------------------------------------------------\\nRESPONSE 404: 404 Not Found\\nERROR CODE UNAVAILABLE\\n--------------------------------------------------------------------------------\\n{\\n  \\\"error\\\": {\\n    \\\"message\\\": \\\"Operation de3f3e34-ee55-4c5b-bdc7-7cbf3170faf2 not found.\\\",\\n    \\\"details\\\": []\\n  }\\n}\\n--------------------------------------------------------------------------------\\n\"\n  }\n}\n--------------------------------------------------------------------------------\n",
        err: <*fmt.wrapError | 0xc000c72080>{
            msg: "failed waiting for cluster=\"ea-cluster\" in resourcegroup=\"external-auth-cluster-qkwf4p\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0b05f658-bbb6-452d-b845-92443ec2c619\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n  \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0b05f658-bbb6-452d-b845-92443ec2c619\",\n  \"name\": \"0b05f658-bbb6-452d-b845-92443ec2c619\",\n  \"status\": \"Failed\",\n  \"startTime\": \"2026-03-11T03:36:35.705780329Z\",\n  \"endTime\": \"2026-03-11T03:38:07.574131728Z\",\n  \"error\": {\n    \"code\": \"InternalServerError\",\n    \"message\": \"GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Network/locations/uksouth/operations/de3f3e34-ee55-4c5b-bdc7-7cbf3170faf2\\n--------------------------------------------------------------------------------\\nRESPONSE 404: 404 Not Found\\nERROR CODE UNAVAILABLE\\n--------------------------------------------------------------------------------\\n{\\n  \\\"error\\\": {\\n    \\\"message\\\": \\\"Operation de3f3e34-ee55-4c5b-bdc7-7cbf3170faf2 not found.\\\",\\n    \\\"details\\\": []\\n  }\\n}\\n--------------------------------------------------------------------------------\\n\"\n  }\n}\n--------------------------------------------------------------------------------\n",
            err: <*exported.ResponseError | 0xc00144fd10>{
                ErrorCode: "InternalServerError",
                StatusCode: 200,
                RawResponse: {
                    Status: "200 OK",
                    StatusCode: 200,
                    Proto: "HTTP/2.0",
                    ProtoMajor: 2,
                    ProtoMinor: 0,
                    Header: {
                        "X-Ms-Ratelimit-Remaining-Subscription-Reads": ["249"],
...

Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.

Learn more here: https://onsi.github.io/gomega/#adjusting-output

occurred

fail [github.com/Azure/ARO-HCP/test/e2e/arm64_nodepool.go:90]: Unexpected error:
    <*fmt.wrapError | 0xc0000d9940>: 
    failed to create HCP cluster arm64-vm-hcp-cluster: failed waiting for cluster="arm64-vm-hcp-cluster" in resourcegroup="arm64-vm-cluster-6h5tqt" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/5c49f2a2-a1d3-45dc-b6e8-265bbbe3eadb
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/5c49f2a2-a1d3-45dc-b6e8-265bbbe3eadb",
      "name": "5c49f2a2-a1d3-45dc-b6e8-265bbbe3eadb",
      "status": "Failed",
      "startTime": "2026-03-11T03:36:37.014555279Z",
      "endTime": "2026-03-11T03:38:07.990052924Z",
      "error": {
        "code": "InternalServerError",
        "message": "GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Network/locations/uksouth/operations/bcedbd48-06f7-4d82-a965-ffd4f43ad6ae\n--------------------------------------------------------------------------------\nRESPONSE 404: 404 Not Found\nERROR CODE UNAVAILABLE\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"message\": \"Operation bcedbd48-06f7-4d82-a965-ffd4f43ad6ae not found.\",\n    \"details\": []\n  }\n}\n--------------------------------------------------------------------------------\n"
      }
    }
    --------------------------------------------------------------------------------
    
    {
        msg: "failed to create HCP cluster arm64-vm-hcp-cluster: failed waiting for cluster=\"arm64-vm-hcp-cluster\" in resourcegroup=\"arm64-vm-cluster-6h5tqt\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/5c49f2a2-a1d3-45dc-b6e8-265bbbe3eadb\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n  \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/5c49f2a2-a1d3-45dc-b6e8-265bbbe3eadb\",\n  \"name\": \"5c49f2a2-a1d3-45dc-b6e8-265bbbe3eadb\",\n  \"status\": \"Failed\",\n  \"startTime\": \"2026-03-11T03:36:37.014555279Z\",\n  \"endTime\": \"2026-03-11T03:38:07.990052924Z\",\n  \"error\": {\n    \"code\": \"InternalServerError\",\n    \"message\": \"GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Network/locations/uksouth/operations/bcedbd48-06f7-4d82-a965-ffd4f43ad6ae\\n--------------------------------------------------------------------------------\\nRESPONSE 404: 404 Not Found\\nERROR CODE UNAVAILABLE\\n--------------------------------------------------------------------------------\\n{\\n  \\\"error\\\": {\\n    \\\"message\\\": \\\"Operation bcedbd48-06f7-4d82-a965-ffd4f43ad6ae not found.\\\",\\n    \\\"details\\\": []\\n  }\\n}\\n--------------------------------------------------------------------------------\\n\"\n  }\n}\n--------------------------------------------------------------------------------\n",
        err: <*fmt.wrapError | 0xc0000d9900>{
            msg: "failed waiting for cluster=\"arm64-vm-hcp-cluster\" in resourcegroup=\"arm64-vm-cluster-6h5tqt\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/5c49f2a2-a1d3-45dc-b6e8-265bbbe3eadb\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n  \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/5c49f2a2-a1d3-45dc-b6e8-265bbbe3eadb\",\n  \"name\": \"5c49f2a2-a1d3-45dc-b6e8-265bbbe3eadb\",\n  \"status\": \"Failed\",\n  \"startTime\": \"2026-03-11T03:36:37.014555279Z\",\n  \"endTime\": \"2026-03-11T03:38:07.990052924Z\",\n  \"error\": {\n    \"code\": \"InternalServerError\",\n    \"message\": \"GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Network/locations/uksouth/operations/bcedbd48-06f7-4d82-a965-ffd4f43ad6ae\\n--------------------------------------------------------------------------------\\nRESPONSE 404: 404 Not Found\\nERROR CODE UNAVAILABLE\\n--------------------------------------------------------------------------------\\n{\\n  \\\"error\\\": {\\n    \\\"message\\\": \\\"Operation bcedbd48-06f7-4d82-a965-ffd4f43ad6ae not found.\\\",\\n    \\\"details\\\": []\\n  }\\n}\\n--------------------------------------------------------------------------------\\n\"\n  }\n}\n--------------------------------------------------------------------------------\n",
            err: <*exported.ResponseError | 0xc000f006c0>{
                ErrorCode: "InternalServerError",
                StatusCode: 200,
                RawResponse: {
                    Status: "200 OK",
                    StatusCode: 200,
                    Proto: "HTTP/2.0",
                    ProtoMajor: 2,
                    ProtoMinor: 0,
                    Header: {
                        "X-Ms-Routing-Request-Id": [
       ...

Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.

Learn more here: https://onsi.github.io/gomega/#adjusting-output

occurred
fail [github.com/Azure/ARO-HCP/test/e2e/arm64_nodepool.go:90]: Unexpected error:
    <*fmt.wrapError | 0xc0000d9940>: 
    failed to create HCP cluster arm64-vm-hcp-cluster: failed waiting for cluster="arm64-vm-hcp-cluster" in resourcegroup="arm64-vm-cluster-6h5tqt" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/5c49f2a2-a1d3-45dc-b6e8-265bbbe3eadb
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/5c49f2a2-a1d3-45dc-b6e8-265bbbe3eadb",
      "name": "5c49f2a2-a1d3-45dc-b6e8-265bbbe3eadb",
      "status": "Failed",
      "startTime": "2026-03-11T03:36:37.014555279Z",
      "endTime": "2026-03-11T03:38:07.990052924Z",
      "error": {
        "code": "InternalServerError",
        "message": "GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Network/locations/uksouth/operations/bcedbd48-06f7-4d82-a965-ffd4f43ad6ae\n--------------------------------------------------------------------------------\nRESPONSE 404: 404 Not Found\nERROR CODE UNAVAILABLE\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"message\": \"Operation bcedbd48-06f7-4d82-a965-ffd4f43ad6ae not found.\",\n    \"details\": []\n  }\n}\n--------------------------------------------------------------------------------\n"
      }
    }
    --------------------------------------------------------------------------------
    
    {
        msg: "failed to create HCP cluster arm64-vm-hcp-cluster: failed waiting for cluster=\"arm64-vm-hcp-cluster\" in resourcegroup=\"arm64-vm-cluster-6h5tqt\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/5c49f2a2-a1d3-45dc-b6e8-265bbbe3eadb\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n  \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/5c49f2a2-a1d3-45dc-b6e8-265bbbe3eadb\",\n  \"name\": \"5c49f2a2-a1d3-45dc-b6e8-265bbbe3eadb\",\n  \"status\": \"Failed\",\n  \"startTime\": \"2026-03-11T03:36:37.014555279Z\",\n  \"endTime\": \"2026-03-11T03:38:07.990052924Z\",\n  \"error\": {\n    \"code\": \"InternalServerError\",\n    \"message\": \"GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Network/locations/uksouth/operations/bcedbd48-06f7-4d82-a965-ffd4f43ad6ae\\n--------------------------------------------------------------------------------\\nRESPONSE 404: 404 Not Found\\nERROR CODE UNAVAILABLE\\n--------------------------------------------------------------------------------\\n{\\n  \\\"error\\\": {\\n    \\\"message\\\": \\\"Operation bcedbd48-06f7-4d82-a965-ffd4f43ad6ae not found.\\\",\\n    \\\"details\\\": []\\n  }\\n}\\n--------------------------------------------------------------------------------\\n\"\n  }\n}\n--------------------------------------------------------------------------------\n",
        err: <*fmt.wrapError | 0xc0000d9900>{
            msg: "failed waiting for cluster=\"arm64-vm-hcp-cluster\" in resourcegroup=\"arm64-vm-cluster-6h5tqt\" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/5c49f2a2-a1d3-45dc-b6e8-265bbbe3eadb\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n  \"id\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/5c49f2a2-a1d3-45dc-b6e8-265bbbe3eadb\",\n  \"name\": \"5c49f2a2-a1d3-45dc-b6e8-265bbbe3eadb\",\n  \"status\": \"Failed\",\n  \"startTime\": \"2026-03-11T03:36:37.014555279Z\",\n  \"endTime\": \"2026-03-11T03:38:07.990052924Z\",\n  \"error\": {\n    \"code\": \"InternalServerError\",\n    \"message\": \"GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Network/locations/uksouth/operations/bcedbd48-06f7-4d82-a965-ffd4f43ad6ae\\n--------------------------------------------------------------------------------\\nRESPONSE 404: 404 Not Found\\nERROR CODE UNAVAILABLE\\n--------------------------------------------------------------------------------\\n{\\n  \\\"error\\\": {\\n    \\\"message\\\": \\\"Operation bcedbd48-06f7-4d82-a965-ffd4f43ad6ae not found.\\\",\\n    \\\"details\\\": []\\n  }\\n}\\n--------------------------------------------------------------------------------\\n\"\n  }\n}\n--------------------------------------------------------------------------------\n",
            err: <*exported.ResponseError | 0xc000f006c0>{
                ErrorCode: "InternalServerError",
                StatusCode: 200,
                RawResponse: {
                    Status: "200 OK",
                    StatusCode: 200,
                    Proto: "HTTP/2.0",
                    ProtoMajor: 2,
                    ProtoMinor: 0,
                    Header: {
                        "X-Ms-Routing-Request-Id": [
       ...

Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.

Learn more here: https://onsi.github.io/gomega/#adjusting-output

occurred

Contributing tests (2)

Failed At	Job	Test	Support
e2e	periodic-ci-Azure-ARO-HCP-main-periodic-integration-e2e-parallel	Customer should be able to create a cluster with an external auth config and get the external auth config	1
e2e	periodic-ci-Azure-ARO-HCP-main-periodic-integration-e2e-parallel	Customer should be able to create node pools with ARM64-based VMs	1

Affected runs (1)

Date (UTC)	Associated PR	Prow job
2026-03-11 03:22	n/a	2031571552545280000

Interrupted by User

full failure pattern:

Interrupted by User

Signal:

e2e

none

Full failure examples (1)

fail [:0]: Interrupted by User
fail [:0]: Interrupted by User

Contributing tests (1)

Failed At	Job	Test	Support
e2e	periodic-ci-Azure-ARO-HCP-main-periodic-integration-e2e-parallel	Authorized CIDRs Connectivity should allow API access only from authorized VM IP	1

Affected runs (1)

Date (UTC)	Associated PR	Prow job
2026-03-11 00:05	n/a	2031521952853659648

ERROR CODE: Conflict; detail message Cannot create resource while parent resource is provisioning; provider Microsoft.RedHatOpenShift

full failure pattern:

ERROR CODE: Conflict; detail message Cannot create resource while parent resource is provisioning; provider Microsoft.RedHatOpenShift

Signal:

e2e

none

Full failure examples (1)

fail [github.com/Azure/ARO-HCP/test/e2e/arm64_nodepool.go:109]: Unexpected error:
    <*fmt.wrapError | 0xc0007b0fa0>: 
    failed to create NodePool arm64-vm-np-1: failed starting nodepool creation "arm64-vm-np-1" for cluster "arm64-vm-hcp-cluster" in resourcegroup="arm64-vm-cluster-8h7xd7": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/arm64-vm-cluster-8h7xd7/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/arm64-vm-hcp-cluster/nodePools/arm64-vm-np-1
    --------------------------------------------------------------------------------
    RESPONSE 409: 409 Conflict
    ERROR CODE: Conflict
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "Conflict",
        "message": "Cannot create resource while parent resource is provisioning",
        "target": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/arm64-vm-cluster-8h7xd7/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/arm64-vm-hcp-cluster/nodePools/arm64-vm-np-1"
      }
    }
    --------------------------------------------------------------------------------
    
    {
        msg: "failed to create NodePool arm64-vm-np-1: failed starting nodepool creation \"arm64-vm-np-1\" for cluster \"arm64-vm-hcp-cluster\" in resourcegroup=\"arm64-vm-cluster-8h7xd7\": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/arm64-vm-cluster-8h7xd7/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/arm64-vm-hcp-cluster/nodePools/arm64-vm-np-1\n--------------------------------------------------------------------------------\nRESPONSE 409: 409 Conflict\nERROR CODE: Conflict\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"Conflict\",\n    \"message\": \"Cannot create resource while parent resource is provisioning\",\n    \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/arm64-vm-cluster-8h7xd7/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/arm64-vm-hcp-cluster/nodePools/arm64-vm-np-1\"\n  }\n}\n--------------------------------------------------------------------------------\n",
        err: <*fmt.wrapError | 0xc0007b0f60>{
            msg: "failed starting nodepool creation \"arm64-vm-np-1\" for cluster \"arm64-vm-hcp-cluster\" in resourcegroup=\"arm64-vm-cluster-8h7xd7\": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/arm64-vm-cluster-8h7xd7/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/arm64-vm-hcp-cluster/nodePools/arm64-vm-np-1\n--------------------------------------------------------------------------------\nRESPONSE 409: 409 Conflict\nERROR CODE: Conflict\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"Conflict\",\n    \"message\": \"Cannot create resource while parent resource is provisioning\",\n    \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/arm64-vm-cluster-8h7xd7/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/arm64-vm-hcp-cluster/nodePools/arm64-vm-np-1\"\n  }\n}\n--------------------------------------------------------------------------------\n",
            err: <*exported.ResponseError | 0xc0015e0990>{
                ErrorCode: "Conflict",
                StatusCode: 409,
                RawResponse: {
                    Status: "409 Conflict",
                    StatusCode: 409,
                    Proto: "HTTP/2.0",
                    ProtoMajor: 2,
                    ProtoMinor: 0,
                    Header: {
                        "X-Msedge-Ref": [
                            "Ref A: 1A43493D763743078F50D58A7107C67A Ref B: CHI261140511036 Ref C: 2026-03-09T15:39:54Z",
                        ],
                        "Content-Type": ["application/json"],
                        "X-Ms-Ratelimit-Remaining-Subscription-Writes": ["199"],
                        "Content-Length": ["348"],
                        "X-Ms-Request-Id": [
                            "b47e62cf-b184-4859-bc06-a546313b427d",
                        ],
                        "X-Ms-Routing-Request-Id": [
                            "UKSOUTH:20260309T153955Z:71f1c6be-0050-4562-ae53-b90c10721c0e",
                        ],
                        "X-Content-Type-Options": ["nosniff"],
                        "Cache-Control": ["no-cache"],
                        "Pragma": ["no-cache"],
                        "X-Envoy-Upstream-Service-Time": ["169"],
                        "X-Ms-Correlation-Request-Id": [
                            "71f1c6be-0050-4562-ae53-b90c10721c0e",
                        ],
                        "Date": [
                            "Mon, 09 Mar 2026 15:39:54 GMT",
                        ],
                        "Expires": ["-1"],
                        "X-Ms-Error-Code": ["Conflict"],
                        "X-Ms-Operation-Identifier": [
                            "appId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,tenantId=64dc69e4-d083-49fc-9569-ebece...

Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.

Learn more here: https://onsi.github.io/gomega/#adjusting-output

occurred
fail [github.com/Azure/ARO-HCP/test/e2e/arm64_nodepool.go:109]: Unexpected error:
    <*fmt.wrapError | 0xc0007b0fa0>: 
    failed to create NodePool arm64-vm-np-1: failed starting nodepool creation "arm64-vm-np-1" for cluster "arm64-vm-hcp-cluster" in resourcegroup="arm64-vm-cluster-8h7xd7": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/arm64-vm-cluster-8h7xd7/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/arm64-vm-hcp-cluster/nodePools/arm64-vm-np-1
    --------------------------------------------------------------------------------
    RESPONSE 409: 409 Conflict
    ERROR CODE: Conflict
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "Conflict",
        "message": "Cannot create resource while parent resource is provisioning",
        "target": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/arm64-vm-cluster-8h7xd7/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/arm64-vm-hcp-cluster/nodePools/arm64-vm-np-1"
      }
    }
    --------------------------------------------------------------------------------
    
    {
        msg: "failed to create NodePool arm64-vm-np-1: failed starting nodepool creation \"arm64-vm-np-1\" for cluster \"arm64-vm-hcp-cluster\" in resourcegroup=\"arm64-vm-cluster-8h7xd7\": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/arm64-vm-cluster-8h7xd7/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/arm64-vm-hcp-cluster/nodePools/arm64-vm-np-1\n--------------------------------------------------------------------------------\nRESPONSE 409: 409 Conflict\nERROR CODE: Conflict\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"Conflict\",\n    \"message\": \"Cannot create resource while parent resource is provisioning\",\n    \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/arm64-vm-cluster-8h7xd7/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/arm64-vm-hcp-cluster/nodePools/arm64-vm-np-1\"\n  }\n}\n--------------------------------------------------------------------------------\n",
        err: <*fmt.wrapError | 0xc0007b0f60>{
            msg: "failed starting nodepool creation \"arm64-vm-np-1\" for cluster \"arm64-vm-hcp-cluster\" in resourcegroup=\"arm64-vm-cluster-8h7xd7\": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/arm64-vm-cluster-8h7xd7/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/arm64-vm-hcp-cluster/nodePools/arm64-vm-np-1\n--------------------------------------------------------------------------------\nRESPONSE 409: 409 Conflict\nERROR CODE: Conflict\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"Conflict\",\n    \"message\": \"Cannot create resource while parent resource is provisioning\",\n    \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/arm64-vm-cluster-8h7xd7/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/arm64-vm-hcp-cluster/nodePools/arm64-vm-np-1\"\n  }\n}\n--------------------------------------------------------------------------------\n",
            err: <*exported.ResponseError | 0xc0015e0990>{
                ErrorCode: "Conflict",
                StatusCode: 409,
                RawResponse: {
                    Status: "409 Conflict",
                    StatusCode: 409,
                    Proto: "HTTP/2.0",
                    ProtoMajor: 2,
                    ProtoMinor: 0,
                    Header: {
                        "X-Msedge-Ref": [
                            "Ref A: 1A43493D763743078F50D58A7107C67A Ref B: CHI261140511036 Ref C: 2026-03-09T15:39:54Z",
                        ],
                        "Content-Type": ["application/json"],
                        "X-Ms-Ratelimit-Remaining-Subscription-Writes": ["199"],
                        "Content-Length": ["348"],
                        "X-Ms-Request-Id": [
                            "b47e62cf-b184-4859-bc06-a546313b427d",
                        ],
                        "X-Ms-Routing-Request-Id": [
                            "UKSOUTH:20260309T153955Z:71f1c6be-0050-4562-ae53-b90c10721c0e",
                        ],
                        "X-Content-Type-Options": ["nosniff"],
                        "Cache-Control": ["no-cache"],
                        "Pragma": ["no-cache"],
                        "X-Envoy-Upstream-Service-Time": ["169"],
                        "X-Ms-Correlation-Request-Id": [
                            "71f1c6be-0050-4562-ae53-b90c10721c0e",
                        ],
                        "Date": [
                            "Mon, 09 Mar 2026 15:39:54 GMT",
                        ],
                        "Expires": ["-1"],
                        "X-Ms-Error-Code": ["Conflict"],
                        "X-Ms-Operation-Identifier": [
                            "appId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,tenantId=64dc69e4-d083-49fc-9569-ebece...

Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.

Learn more here: https://onsi.github.io/gomega/#adjusting-output

occurred

Contributing tests (1)

Failed At	Job	Test	Support
e2e	periodic-ci-Azure-ARO-HCP-main-periodic-integration-e2e-parallel	Customer should be able to create node pools with ARM64-based VMs	1

Affected runs (1)

Date (UTC)	Associated PR	Prow job
2026-03-09 15:26	n/a	2031028912292630528

Admin credential 1 should fail after revocation, last error: <nil>

full failure pattern:

Admin credential 1 should fail after revocation, last error: <nil>

Signal:

e2e

none

Full failure examples (1)

fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:222]: Admin credential 1 should fail after revocation, last error: <nil>
Unexpected error:
    <context.deadlineExceededError>: 
    context deadline exceeded
    {}
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:222]: Admin credential 1 should fail after revocation, last error: <nil>
Unexpected error:
    <context.deadlineExceededError>: 
    context deadline exceeded
    {}
occurred

Contributing tests (1)

Failed At	Job	Test	Support
e2e	periodic-ci-Azure-ARO-HCP-main-periodic-integration-e2e-parallel	Customer should be able to test admin credentials before cluster ready, then full admin credential lifecycle	1

Affected runs (1)

Date (UTC)	Associated PR	Prow job
2026-03-14 17:26	n/a	2032871090371432448

Cluster provisioning failed

full failure pattern:

Cluster provisioning failed

Signal:

e2e

none

Full failure examples (1)

fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:152]: Cluster provisioning failed
fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:152]: Cluster provisioning failed

Contributing tests (1)

Failed At	Job	Test	Support
e2e	periodic-ci-Azure-ARO-HCP-main-periodic-integration-e2e-parallel	Customer should be able to test admin credentials before cluster ready, then full admin credential lifecycle	1

Affected runs (1)

Date (UTC)	Associated PR	Prow job
2026-03-11 03:22	n/a	2031571552545280000

ERROR CODE: Conflict; detail message Resource is already deleting; provider Microsoft.RedHatOpenShift

full failure pattern:

ERROR CODE: Conflict; detail message Resource is already deleting; provider Microsoft.RedHatOpenShift

Signal:

e2e

none

Full failure examples (1)

fail [github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:243]: Unexpected error:
    <*errors.joinError | 0xc000d78048>: 
    failed to cleanup resource group: at least one hcp cluster failed to delete: DELETE https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/cluster-listing-sqdgm5/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/list-test-cluster-vzqchl
    --------------------------------------------------------------------------------
    RESPONSE 409: 409 Conflict
    ERROR CODE: Conflict
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "Conflict",
        "message": "Resource is already deleting",
        "target": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/cluster-listing-sqdgm5/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/list-test-cluster-vzqchl"
      }
    }
    --------------------------------------------------------------------------------
    
    {
        errs: [
            <*fmt.wrapError | 0xc001509a00>{
                msg: "failed to cleanup resource group: at least one hcp cluster failed to delete: DELETE https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/cluster-listing-sqdgm5/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/list-test-cluster-vzqchl\n--------------------------------------------------------------------------------\nRESPONSE 409: 409 Conflict\nERROR CODE: Conflict\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"Conflict\",\n    \"message\": \"Resource is already deleting\",\n    \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/cluster-listing-sqdgm5/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/list-test-cluster-vzqchl\"\n  }\n}\n--------------------------------------------------------------------------------\n",
                err: <*fmt.wrapError | 0xc0015099e0>{
                    msg: "at least one hcp cluster failed to delete: DELETE https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/cluster-listing-sqdgm5/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/list-test-cluster-vzqchl\n--------------------------------------------------------------------------------\nRESPONSE 409: 409 Conflict\nERROR CODE: Conflict\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"Conflict\",\n    \"message\": \"Resource is already deleting\",\n    \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/cluster-listing-sqdgm5/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/list-test-cluster-vzqchl\"\n  }\n}\n--------------------------------------------------------------------------------\n",
                    err: <*exported.ResponseError | 0xc0009cc510>{
                        ErrorCode: "Conflict",
                        StatusCode: 409,
                        RawResponse: {
                            Status: "409 Conflict",
                            StatusCode: 409,
                            Proto: "HTTP/2.0",
                            ProtoMajor: 2,
                            ProtoMinor: 0,
                            Header: {
                                "Content-Length": ["295"],
                                "X-Ms-Ratelimit-Remaining-Subscription-Global-Deletes": ["3000"],
                                "X-Ms-Correlation-Request-Id": [
                                    "36e4392a-0d35-4147-8a9f-6300ad30a25c",
                                ],
                                "X-Ms-Routing-Request-Id": [
                                    "UKSOUTH:20260309T050748Z:36e4392a-0d35-4147-8a9f-6300ad30a25c",
                                ],
                                "Cache-Control": ["no-cache"],
                                "Expires": ["-1"],
                                "X-Ms-Request-Id": [
                                    "9240eef9-4396-4070-b424-2e847f902062",
                                ],
                                "X-Envoy-Upstream-Service-Time": ["32460"],
                                "X-Ms-Ratelimit-Remaining-Subscription-Deletes": ["200"],
                                "X-Cache": ["CONFIG_NOCACHE"],
                                "Date": [
                                    "Mon, 09 Mar 2026 05:07:47 GMT",
                                ],
                                "Content-Type": ["application/json"],
                                "X-Ms-Error-Code": ["Conflict"],
                                "X-Ms-Operation-Identifier": [
                                    "appId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,tenantId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,objectId=c209f8df-52ae-48fb-98ea-380f58b04652/uksouth/1795cff2-640d-4155-ab23-dd61fa932a34",
                                ],
  ...

Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.

Learn more here: https://onsi.github.io/gomega/#adjusting-output

occurred
fail [github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:243]: Unexpected error:
    <*errors.joinError | 0xc000d78048>: 
    failed to cleanup resource group: at least one hcp cluster failed to delete: DELETE https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/cluster-listing-sqdgm5/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/list-test-cluster-vzqchl
    --------------------------------------------------------------------------------
    RESPONSE 409: 409 Conflict
    ERROR CODE: Conflict
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "Conflict",
        "message": "Resource is already deleting",
        "target": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/cluster-listing-sqdgm5/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/list-test-cluster-vzqchl"
      }
    }
    --------------------------------------------------------------------------------
    
    {
        errs: [
            <*fmt.wrapError | 0xc001509a00>{
                msg: "failed to cleanup resource group: at least one hcp cluster failed to delete: DELETE https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/cluster-listing-sqdgm5/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/list-test-cluster-vzqchl\n--------------------------------------------------------------------------------\nRESPONSE 409: 409 Conflict\nERROR CODE: Conflict\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"Conflict\",\n    \"message\": \"Resource is already deleting\",\n    \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/cluster-listing-sqdgm5/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/list-test-cluster-vzqchl\"\n  }\n}\n--------------------------------------------------------------------------------\n",
                err: <*fmt.wrapError | 0xc0015099e0>{
                    msg: "at least one hcp cluster failed to delete: DELETE https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/cluster-listing-sqdgm5/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/list-test-cluster-vzqchl\n--------------------------------------------------------------------------------\nRESPONSE 409: 409 Conflict\nERROR CODE: Conflict\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"Conflict\",\n    \"message\": \"Resource is already deleting\",\n    \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/cluster-listing-sqdgm5/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/list-test-cluster-vzqchl\"\n  }\n}\n--------------------------------------------------------------------------------\n",
                    err: <*exported.ResponseError | 0xc0009cc510>{
                        ErrorCode: "Conflict",
                        StatusCode: 409,
                        RawResponse: {
                            Status: "409 Conflict",
                            StatusCode: 409,
                            Proto: "HTTP/2.0",
                            ProtoMajor: 2,
                            ProtoMinor: 0,
                            Header: {
                                "Content-Length": ["295"],
                                "X-Ms-Ratelimit-Remaining-Subscription-Global-Deletes": ["3000"],
                                "X-Ms-Correlation-Request-Id": [
                                    "36e4392a-0d35-4147-8a9f-6300ad30a25c",
                                ],
                                "X-Ms-Routing-Request-Id": [
                                    "UKSOUTH:20260309T050748Z:36e4392a-0d35-4147-8a9f-6300ad30a25c",
                                ],
                                "Cache-Control": ["no-cache"],
                                "Expires": ["-1"],
                                "X-Ms-Request-Id": [
                                    "9240eef9-4396-4070-b424-2e847f902062",
                                ],
                                "X-Envoy-Upstream-Service-Time": ["32460"],
                                "X-Ms-Ratelimit-Remaining-Subscription-Deletes": ["200"],
                                "X-Cache": ["CONFIG_NOCACHE"],
                                "Date": [
                                    "Mon, 09 Mar 2026 05:07:47 GMT",
                                ],
                                "Content-Type": ["application/json"],
                                "X-Ms-Error-Code": ["Conflict"],
                                "X-Ms-Operation-Identifier": [
                                    "appId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,tenantId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,objectId=c209f8df-52ae-48fb-98ea-380f58b04652/uksouth/1795cff2-640d-4155-ab23-dd61fa932a34",
                                ],
  ...

Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.

Learn more here: https://onsi.github.io/gomega/#adjusting-output

occurred

Contributing tests (1)

Failed At	Job	Test	Support
e2e	periodic-ci-Azure-ARO-HCP-main-periodic-integration-e2e-parallel	Customer should be able to list HCP clusters without node pools at both subscription and resource group levels	1

Affected runs (1)

Date (UTC)	Associated PR	Prow job
2026-03-09 04:24	n/a	2030862218051981312

ERROR CODE: DeploymentFailed; detail code InvalidRequestContent; detail message node pool <nodepool> '4.20.15' must not be greater than Control Plane version '4.20.8'

full failure pattern:

ERROR CODE: DeploymentFailed; detail code InvalidRequestContent; detail message node pool <nodepool> '4.20.15' must not be greater than Control Plane version '4.20.8'

Signal:

e2e

none

Full failure examples (1)

fail [github.com/Azure/ARO-HCP/test/e2e/gpu_nodepools_create_delete.go:100]: Unexpected error:
    <*fmt.wrapError | 0xc000fc6740>: 
    failed waiting for deployment "aro-hcp-demo" in resourcegroup="gpu-nodepools-NC4asT4v3-w6hmz9" to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourcegroups/gpu-nodepools-NC4asT4v3-w6hmz9/providers/Microsoft.Resources/deployments/aro-hcp-demo/operationStatuses/08584282515392752534
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: DeploymentFailed
    --------------------------------------------------------------------------------
    {
      "status": "Failed",
      "error": {
        "code": "DeploymentFailed",
        "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.",
        "details": [
          {
            "code": "Conflict",
            "message": "{\r\n  \"status\": \"Failed\",\r\n  \"error\": {\r\n    \"code\": \"ResourceDeploymentFailure\",\r\n    \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n    \"details\": [\r\n      {\r\n        \"code\": \"DeploymentFailed\",\r\n        \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/gpu-nodepools-NC4asT4v3-w6hmz9/providers/Microsoft.Resources/deployments/nodepool-1\",\r\n        \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n        \"details\": [\r\n          {\r\n            \"code\": \"InvalidRequestContent\",\r\n            \"message\": \"Node pool version '4.20.15' must not be greater than Control Plane version '4.20.8'\"\r\n          }\r\n        ]\r\n      }\r\n    ]\r\n  }\r\n}"
          }
        ]
      }
    }
    --------------------------------------------------------------------------------
    
    {
        msg: "failed waiting for deployment \"aro-hcp-demo\" in resourcegroup=\"gpu-nodepools-NC4asT4v3-w6hmz9\" to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourcegroups/gpu-nodepools-NC4asT4v3-w6hmz9/providers/Microsoft.Resources/deployments/aro-hcp-demo/operationStatuses/08584282515392752534\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: DeploymentFailed\n--------------------------------------------------------------------------------\n{\n  \"status\": \"Failed\",\n  \"error\": {\n    \"code\": \"DeploymentFailed\",\n    \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\n    \"details\": [\n      {\n        \"code\": \"Conflict\",\n        \"message\": \"{\\r\\n  \\\"status\\\": \\\"Failed\\\",\\r\\n  \\\"error\\\": {\\r\\n    \\\"code\\\": \\\"ResourceDeploymentFailure\\\",\\r\\n    \\\"message\\\": \\\"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\\\",\\r\\n    \\\"details\\\": [\\r\\n      {\\r\\n        \\\"code\\\": \\\"DeploymentFailed\\\",\\r\\n        \\\"target\\\": \\\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/gpu-nodepools-NC4asT4v3-w6hmz9/providers/Microsoft.Resources/deployments/nodepool-1\\\",\\r\\n        \\\"message\\\": \\\"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\\\",\\r\\n        \\\"details\\\": [\\r\\n          {\\r\\n            \\\"code\\\": \\\"InvalidRequestContent\\\",\\r\\n            \\\"message\\\": \\\"Node pool version '4.20.15' must not be greater than Control Plane version '4.20.8'\\\"\\r\\n          }\\r\\n        ]\\r\\n      }\\r\\n    ]\\r\\n  }\\r\\n}\"\n      }\n    ]\n  }\n}\n--------------------------------------------------------------------------------\n",
        err: <*exported.ResponseError | 0xc001500e10>{
            ErrorCode: "DeploymentFailed",
            StatusCode: 200,
            RawResponse: {
                Status: "200 OK",
                StatusCode: 200,
                Proto: "HTTP/2.0",
                ProtoMajor: 2,
                ProtoMinor: 0,
                Header: {
                    "Pragma": ["no-cache"],
                    "Content-Length": ["1243"],
                    "X-Ms-Request-Id": [
                        "0d231ad8-8318-4e3a-920c-9541e2affd70",
                    ],
                    "Cache-Control": ["no-cache"],
                    "X-Ms-Correlation-Request-Id": [
                        "0d231ad8-8318-4e3a-920c-9541e2affd70",
                    ],
                    "X-Cache": ["CONFIG_NOCACHE"],
                    "Content-Type": [
                        "application/json; charset=utf-8",
                    ],
                    "X-Ms-Ratelimit-Remaining-Subscription-Global-Reads": ["3748"],
                    "X-Ms-Routing-Request-Id": [
                        "UKSOUTH:20260312T225347Z:0d231ad8-8318-4e3a-920c-9541e2affd70",
                    ],
                    "Expires": ["-1"],
                    "X-Ms-Ratelimit-Remaining-Subscription-Reads": ["249"],
                    "Strict-Transport-Security": [
                        "max-age=31536000; includeSubDomains",
                    ],
                    "X-Content-Type-Options": ["nosniff"],
                    "X-Msedge-Ref": [
                        "Ref A: 4733BC488DE8424898A45EF16542E708 Ref B: CH1AA2020603045 Ref C: 2026-03-12T22:53:47Z",
                    ],
                    "Date": [
                        "Thu, 12 Mar 2026 22:53:46 GMT",
                    ],
                },
                Body: <*exported.nopClosingBytesReader | 0xc000fc6620>{
                    s:...

Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.

Learn more here: https://onsi.github.io/gomega/#adjusting-output

occurred
fail [github.com/Azure/ARO-HCP/test/e2e/gpu_nodepools_create_delete.go:100]: Unexpected error:
    <*fmt.wrapError | 0xc000fc6740>: 
    failed waiting for deployment "aro-hcp-demo" in resourcegroup="gpu-nodepools-NC4asT4v3-w6hmz9" to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourcegroups/gpu-nodepools-NC4asT4v3-w6hmz9/providers/Microsoft.Resources/deployments/aro-hcp-demo/operationStatuses/08584282515392752534
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: DeploymentFailed
    --------------------------------------------------------------------------------
    {
      "status": "Failed",
      "error": {
        "code": "DeploymentFailed",
        "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.",
        "details": [
          {
            "code": "Conflict",
            "message": "{\r\n  \"status\": \"Failed\",\r\n  \"error\": {\r\n    \"code\": \"ResourceDeploymentFailure\",\r\n    \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n    \"details\": [\r\n      {\r\n        \"code\": \"DeploymentFailed\",\r\n        \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/gpu-nodepools-NC4asT4v3-w6hmz9/providers/Microsoft.Resources/deployments/nodepool-1\",\r\n        \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n        \"details\": [\r\n          {\r\n            \"code\": \"InvalidRequestContent\",\r\n            \"message\": \"Node pool version '4.20.15' must not be greater than Control Plane version '4.20.8'\"\r\n          }\r\n        ]\r\n      }\r\n    ]\r\n  }\r\n}"
          }
        ]
      }
    }
    --------------------------------------------------------------------------------
    
    {
        msg: "failed waiting for deployment \"aro-hcp-demo\" in resourcegroup=\"gpu-nodepools-NC4asT4v3-w6hmz9\" to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourcegroups/gpu-nodepools-NC4asT4v3-w6hmz9/providers/Microsoft.Resources/deployments/aro-hcp-demo/operationStatuses/08584282515392752534\n--------------------------------------------------------------------------------\nRESPONSE 200: 200 OK\nERROR CODE: DeploymentFailed\n--------------------------------------------------------------------------------\n{\n  \"status\": \"Failed\",\n  \"error\": {\n    \"code\": \"DeploymentFailed\",\n    \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\n    \"details\": [\n      {\n        \"code\": \"Conflict\",\n        \"message\": \"{\\r\\n  \\\"status\\\": \\\"Failed\\\",\\r\\n  \\\"error\\\": {\\r\\n    \\\"code\\\": \\\"ResourceDeploymentFailure\\\",\\r\\n    \\\"message\\\": \\\"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\\\",\\r\\n    \\\"details\\\": [\\r\\n      {\\r\\n        \\\"code\\\": \\\"DeploymentFailed\\\",\\r\\n        \\\"target\\\": \\\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/gpu-nodepools-NC4asT4v3-w6hmz9/providers/Microsoft.Resources/deployments/nodepool-1\\\",\\r\\n        \\\"message\\\": \\\"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\\\",\\r\\n        \\\"details\\\": [\\r\\n          {\\r\\n            \\\"code\\\": \\\"InvalidRequestContent\\\",\\r\\n            \\\"message\\\": \\\"Node pool version '4.20.15' must not be greater than Control Plane version '4.20.8'\\\"\\r\\n          }\\r\\n        ]\\r\\n      }\\r\\n    ]\\r\\n  }\\r\\n}\"\n      }\n    ]\n  }\n}\n--------------------------------------------------------------------------------\n",
        err: <*exported.ResponseError | 0xc001500e10>{
            ErrorCode: "DeploymentFailed",
            StatusCode: 200,
            RawResponse: {
                Status: "200 OK",
                StatusCode: 200,
                Proto: "HTTP/2.0",
                ProtoMajor: 2,
                ProtoMinor: 0,
                Header: {
                    "Pragma": ["no-cache"],
                    "Content-Length": ["1243"],
                    "X-Ms-Request-Id": [
                        "0d231ad8-8318-4e3a-920c-9541e2affd70",
                    ],
                    "Cache-Control": ["no-cache"],
                    "X-Ms-Correlation-Request-Id": [
                        "0d231ad8-8318-4e3a-920c-9541e2affd70",
                    ],
                    "X-Cache": ["CONFIG_NOCACHE"],
                    "Content-Type": [
                        "application/json; charset=utf-8",
                    ],
                    "X-Ms-Ratelimit-Remaining-Subscription-Global-Reads": ["3748"],
                    "X-Ms-Routing-Request-Id": [
                        "UKSOUTH:20260312T225347Z:0d231ad8-8318-4e3a-920c-9541e2affd70",
                    ],
                    "Expires": ["-1"],
                    "X-Ms-Ratelimit-Remaining-Subscription-Reads": ["249"],
                    "Strict-Transport-Security": [
                        "max-age=31536000; includeSubDomains",
                    ],
                    "X-Content-Type-Options": ["nosniff"],
                    "X-Msedge-Ref": [
                        "Ref A: 4733BC488DE8424898A45EF16542E708 Ref B: CH1AA2020603045 Ref C: 2026-03-12T22:53:47Z",
                    ],
                    "Date": [
                        "Thu, 12 Mar 2026 22:53:46 GMT",
                    ],
                },
                Body: <*exported.nopClosingBytesReader | 0xc000fc6620>{
                    s:...

Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.

Learn more here: https://onsi.github.io/gomega/#adjusting-output

occurred

Contributing tests (1)

Failed At	Job	Test	Support
e2e	periodic-ci-Azure-ARO-HCP-main-periodic-integration-e2e-parallel	HCP Nodepools GPU instances creates and deletes vm type NC4asT4v3 in a single cluster	1

Affected runs (1)

Date (UTC)	Associated PR	Prow job
2026-03-12 22:14	n/a	2032218705995239424

ERROR CODE: InternalServerError; context ; provider Microsoft.RedHatOpenShift

full failure pattern:

ERROR CODE: InternalServerError; context ; provider Microsoft.RedHatOpenShift

Signal:

e2e

none

Full failure examples (1)

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:185]: Unexpected error:
    <*fmt.wrapError | 0xc0012ecd00>: 
    failed to start credential request: POST https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-9wh75v/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cidr-connectivity-test/requestAdminCredential
    --------------------------------------------------------------------------------
    RESPONSE 500: 500 Internal Server Error
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InternalServerError",
        "message": "Internal server error."
      }
    }
    --------------------------------------------------------------------------------
    
    {
        msg: "failed to start credential request: POST https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-9wh75v/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cidr-connectivity-test/requestAdminCredential\n--------------------------------------------------------------------------------\nRESPONSE 500: 500 Internal Server Error\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"InternalServerError\",\n    \"message\": \"Internal server error.\"\n  }\n}\n--------------------------------------------------------------------------------\n",
        err: <*exported.ResponseError | 0xc00130de60>{
            ErrorCode: "InternalServerError",
            StatusCode: 500,
            RawResponse: {
                Status: "500 Internal Server Error",
                StatusCode: 500,
                Proto: "HTTP/2.0",
                ProtoMajor: 2,
                ProtoMinor: 0,
                Header: {
                    "Pragma": ["no-cache"],
                    "Content-Type": ["application/json"],
                    "Expires": ["-1"],
                    "X-Ms-Request-Id": [
                        "aded1fbf-5095-4f14-957e-7e4810dfa7b7",
                    ],
                    "X-Envoy-Upstream-Service-Time": ["1537"],
                    "Cache-Control": ["no-cache"],
                    "X-Ms-Error-Code": [
                        "InternalServerError",
                    ],
                    "X-Ms-Failure-Cause": ["service"],
                    "X-Ms-Operation-Identifier": [
                        "appId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,tenantId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,objectId=c209f8df-52ae-48fb-98ea-380f58b04652/uksouth/ffa8e515-a5e8-40aa-9ef9-9df55097710b",
                    ],
                    "X-Ms-Ratelimit-Remaining-Subscription-Writes": ["199"],
                    "X-Content-Type-Options": ["nosniff"],
                    "X-Ms-Ratelimit-Remaining-Subscription-Global-Writes": ["2999"],
                    "X-Ms-Routing-Request-Id": [
                        "UKSOUTH:20260309T155315Z:1e0a595f-913b-42ed-b06c-d8a52395722d",
                    ],
                    "X-Msedge-Ref": [
                        "Ref A: D4B3120337FD4E1A835EB7502426C822 Ref B: CH1AA2020603037 Ref C: 2026-03-09T15:53:13Z",
                    ],
                    "Date": [
                        "Mon, 09 Mar 2026 15:53:14 GMT",
                    ],
                    "Content-Length": ["107"],
                    "X-Ms-Correlation-Request-Id": [
                        "1e0a595f-913b-42ed-b06c-d8a52395722d",
                    ],
                    "Strict-Transport-Security": [
                        "max-age=31536000; includeSubDomains",
                    ],
                    "X-Cache": ["CONFIG_NOCACHE"],
                },
                Body: <*exported.nopClosingBytesReader | 0xc0012ecc40>{
                    s: [123, 10, 32, 32, 32, 32, 34, 101, 114, 114, 111, 114, 34, 58, 32, 123, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 99, 111, 100, 101, 34, 58, 32, 34, 73, 110, 116, 101, 114, 110, 97, 108, 83, 101, 114, 118, 101, 114, 69, 114, 114, 111, 114, 34, 44, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 109, 101, 115, 115, 97, 103, 101, 34, 58, 32, 34, 73, 110, 116, 101, 114, 110, 97, 108, 32, 115, 101, 114, 118, 101, 114, 32, 101, 114, 114, 111, 114, 46, 34, 10, 32, 32, 32, 32, 125, 10, 125],
                    i: 0,
                },
                ContentLength: 107,
                TransferEncoding: nil,
                Close: false,
                Uncompressed: false,
                Trailer: nil,
                Request: {
                    Method: "POST",
                    URL: {
                        Scheme: "https",
                        Opaque: "",
                        User: nil,
                        H...

Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.

Learn more here: https://onsi.github.io/gomega/#adjusting-output

occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:185]: Unexpected error:
    <*fmt.wrapError | 0xc0012ecd00>: 
    failed to start credential request: POST https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-9wh75v/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cidr-connectivity-test/requestAdminCredential
    --------------------------------------------------------------------------------
    RESPONSE 500: 500 Internal Server Error
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InternalServerError",
        "message": "Internal server error."
      }
    }
    --------------------------------------------------------------------------------
    
    {
        msg: "failed to start credential request: POST https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-9wh75v/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cidr-connectivity-test/requestAdminCredential\n--------------------------------------------------------------------------------\nRESPONSE 500: 500 Internal Server Error\nERROR CODE: InternalServerError\n--------------------------------------------------------------------------------\n{\n  \"error\": {\n    \"code\": \"InternalServerError\",\n    \"message\": \"Internal server error.\"\n  }\n}\n--------------------------------------------------------------------------------\n",
        err: <*exported.ResponseError | 0xc00130de60>{
            ErrorCode: "InternalServerError",
            StatusCode: 500,
            RawResponse: {
                Status: "500 Internal Server Error",
                StatusCode: 500,
                Proto: "HTTP/2.0",
                ProtoMajor: 2,
                ProtoMinor: 0,
                Header: {
                    "Pragma": ["no-cache"],
                    "Content-Type": ["application/json"],
                    "Expires": ["-1"],
                    "X-Ms-Request-Id": [
                        "aded1fbf-5095-4f14-957e-7e4810dfa7b7",
                    ],
                    "X-Envoy-Upstream-Service-Time": ["1537"],
                    "Cache-Control": ["no-cache"],
                    "X-Ms-Error-Code": [
                        "InternalServerError",
                    ],
                    "X-Ms-Failure-Cause": ["service"],
                    "X-Ms-Operation-Identifier": [
                        "appId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,tenantId=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX,objectId=c209f8df-52ae-48fb-98ea-380f58b04652/uksouth/ffa8e515-a5e8-40aa-9ef9-9df55097710b",
                    ],
                    "X-Ms-Ratelimit-Remaining-Subscription-Writes": ["199"],
                    "X-Content-Type-Options": ["nosniff"],
                    "X-Ms-Ratelimit-Remaining-Subscription-Global-Writes": ["2999"],
                    "X-Ms-Routing-Request-Id": [
                        "UKSOUTH:20260309T155315Z:1e0a595f-913b-42ed-b06c-d8a52395722d",
                    ],
                    "X-Msedge-Ref": [
                        "Ref A: D4B3120337FD4E1A835EB7502426C822 Ref B: CH1AA2020603037 Ref C: 2026-03-09T15:53:13Z",
                    ],
                    "Date": [
                        "Mon, 09 Mar 2026 15:53:14 GMT",
                    ],
                    "Content-Length": ["107"],
                    "X-Ms-Correlation-Request-Id": [
                        "1e0a595f-913b-42ed-b06c-d8a52395722d",
                    ],
                    "Strict-Transport-Security": [
                        "max-age=31536000; includeSubDomains",
                    ],
                    "X-Cache": ["CONFIG_NOCACHE"],
                },
                Body: <*exported.nopClosingBytesReader | 0xc0012ecc40>{
                    s: [123, 10, 32, 32, 32, 32, 34, 101, 114, 114, 111, 114, 34, 58, 32, 123, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 99, 111, 100, 101, 34, 58, 32, 34, 73, 110, 116, 101, 114, 110, 97, 108, 83, 101, 114, 118, 101, 114, 69, 114, 114, 111, 114, 34, 44, 10, 32, 32, 32, 32, 32, 32, 32, 32, 34, 109, 101, 115, 115, 97, 103, 101, 34, 58, 32, 34, 73, 110, 116, 101, 114, 110, 97, 108, 32, 115, 101, 114, 118, 101, 114, 32, 101, 114, 114, 111, 114, 46, 34, 10, 32, 32, 32, 32, 125, 10, 125],
                    i: 0,
                },
                ContentLength: 107,
                TransferEncoding: nil,
                Close: false,
                Uncompressed: false,
                Trailer: nil,
                Request: {
                    Method: "POST",
                    URL: {
                        Scheme: "https",
                        Opaque: "",
                        User: nil,
                        H...

Gomega truncated this representation as it exceeds 'format.MaxLength'.
Consider having the object provide a custom 'GomegaStringer' representation
or adjust the parameters in Gomega's 'format' package.

Learn more here: https://onsi.github.io/gomega/#adjusting-output

occurred

Contributing tests (1)

Failed At	Job	Test	Support
e2e	periodic-ci-Azure-ARO-HCP-main-periodic-integration-e2e-parallel	Authorized CIDRs Connectivity should allow API access only from authorized VM IP	1

Affected runs (1)

Date (UTC)	Associated PR	Prow job
2026-03-09 15:26	n/a	2031028912292630528

Should receive valid HTTP status code from authorized IP

full failure pattern:

Should receive valid HTTP status code from authorized IP

Signal:

e2e

none

Full failure examples (1)

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:168]: Timed out after 120.002s.
The function passed to Eventually failed at /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:167 with:
Should receive valid HTTP status code from authorized IP
Expected
    <string>: 000
to match regular expression
    <string>: ^[2-5][0-9][0-9]$
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:168]: Timed out after 120.002s.
The function passed to Eventually failed at /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:167 with:
Should receive valid HTTP status code from authorized IP
Expected
    <string>: 000
to match regular expression
    <string>: ^[2-5][0-9][0-9]$

Contributing tests (1)

Failed At	Job	Test	Support
e2e	periodic-ci-Azure-ARO-HCP-main-periodic-integration-e2e-parallel	Authorized CIDRs Connectivity should allow API access only from authorized VM IP	1

Affected runs (1)

Date (UTC)	Associated PR	Prow job
2026-03-12 13:21	n/a	2032084631825682432

Timed out after 120.345s.

full failure pattern:

Timed out after 120.345s.

Signal:

e2e

none

Full failure examples (1)

fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:168]: Timed out after 120.345s.
The function passed to Eventually failed at /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:162 with:
Unexpected error:
    <context.deadlineExceededError>: 
    context deadline exceeded
    {}
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:168]: Timed out after 120.345s.
The function passed to Eventually failed at /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:162 with:
Unexpected error:
    <context.deadlineExceededError>: 
    context deadline exceeded
    {}
occurred

Contributing tests (1)

Failed At	Job	Test	Support
e2e	periodic-ci-Azure-ARO-HCP-main-periodic-integration-e2e-parallel	Authorized CIDRs Connectivity should allow API access only from authorized VM IP	1

Affected runs (1)

Date (UTC)	Associated PR	Prow job
2026-03-14 00:12	n/a	2032610884349595648

client rate limiter Wait returned an error: context deadline exceeded

full failure pattern:

client rate limiter Wait returned an error: context deadline exceeded

Signal:

e2e

none

Full failure examples (1)

fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:222]: Admin credential 1 should fail after revocation, last error: client rate limiter Wait returned an error: context deadline exceeded
Unexpected error:
    <context.deadlineExceededError>: 
    context deadline exceeded
    {}
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:222]: Admin credential 1 should fail after revocation, last error: client rate limiter Wait returned an error: context deadline exceeded
Unexpected error:
    <context.deadlineExceededError>: 
    context deadline exceeded
    {}
occurred

Contributing tests (1)

Failed At	Job	Test	Support
e2e	periodic-ci-Azure-ARO-HCP-main-periodic-integration-e2e-parallel	Customer should be able to test admin credentials before cluster ready, then full admin credential lifecycle	1

Affected runs (1)

Date (UTC)	Associated PR	Prow job
2026-03-15 05:21	n/a	2033051021734842368