CIHealth Run Log

Environment: STG

Time (UTC)	Runtime	Job	Region	Failed at	Result	PR	Failed tests	Details
23:00:14	1h 55m 50s	periodic-ci-Azure-ARO-HCP-main-periodic-stage-e2e-parallel-ocp-nightly post-goodmerged PR No additional run metadata captured	uksouth	e2e	failed	n/a	13	e2e (13) ERROR CODE: InternalServerError; detail message [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated; provider Microsoft.RedHatOpenShift 23:00:14 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP nightly channel for 5.1 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:123]: HCP cluster rg-nightly-5-1-wllchtq6tspw/cluster-nightly-5-1-xqrkdk should provision Unexpected error: <fmt.wrapError \| 0xc000614980>: failed to create HCP cluster cluster-nightly-5-1-xqrkdk: failed waiting for cluster="cluster-nightly-5-1-xqrkdk" in resourcegroup="rg-nightly-5-1-wllchtq6tspw" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/8d90a29e-3743-4f19-9012-64f6e3adf3af -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/8d90a29e-3743-4f19-9012-64f6e3adf3af", "name": "8d90a29e-3743-4f19-9012-64f6e3adf3af", "status": "Failed", "startTime": "2026-09-24T23:29:29.214562248Z", "endTime": "2026-09-24T23:58:29.739929622Z", "error": { "code": "InternalServerError", "message": "[cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:123]: HCP cluster rg-nightly-5-1-wllchtq6tspw/cluster-nightly-5-1-xqrkdk should provision Unexpected error: <fmt.wrapError \| 0xc000614980>: failed to create HCP cluster cluster-nightly-5-1-xqrkdk: failed waiting for cluster="cluster-nightly-5-1-xqrkdk" in resourcegroup="rg-nightly-5-1-wllchtq6tspw" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/8d90a29e-3743-4f19-9012-64f6e3adf3af -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/8d90a29e-3743-4f19-9012-64f6e3adf3af", "name": "8d90a29e-3743-4f19-9012-64f6e3adf3af", "status": "Failed", "startTime": "2026-09-24T23:29:29.214562248Z", "endTime": "2026-09-24T23:58:29.739929622Z", "error": { "code": "InternalServerError", "message": "[cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; [servingCABundle] ServingCABundle not yet populated" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message cluster creation did not complete before the deadline; provider Microsoft.RedHatOpenShift 23:00:14 UTC · failed at e2e · test Customer should upgrade and update a nodepool from 4.20.z to 4.21.zLatest · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:146]: failed to create HCP cluster np-version-upgrade-cluster-cf4l7z with version 4.21.0-0.nightly-multi-2026-09-24-033657 Unexpected error: <fmt.wrapError \| 0xc00142c760>: failed to create HCP cluster np-version-upgrade-cluster-cf4l7z: failed waiting for cluster="np-version-upgrade-cluster-cf4l7z" in resourcegroup="rg-np-version-upgrade-cf4l7z-c8gcs9n4rck7" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/58578399-7a04-4d7e-9fff-65b3ebeb1ba4 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/58578399-7a04-4d7e-9fff-65b3ebeb1ba4", "name": "58578399-7a04-4d7e-9fff-65b3ebeb1ba4", "status": "Failed", "startTime": "2026-09-24T23:42:40.568397466Z", "endTime": "2026-09-25T00:11:49.734192333Z", "error": { "code": "InternalServerError", "message": "cluster creation did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:146]: failed to create HCP cluster np-version-upgrade-cluster-cf4l7z with version 4.21.0-0.nightly-multi-2026-09-24-033657 Unexpected error: <fmt.wrapError \| 0xc00142c760>: failed to create HCP cluster np-version-upgrade-cluster-cf4l7z: failed waiting for cluster="np-version-upgrade-cluster-cf4l7z" in resourcegroup="rg-np-version-upgrade-cf4l7z-c8gcs9n4rck7" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/58578399-7a04-4d7e-9fff-65b3ebeb1ba4 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/58578399-7a04-4d7e-9fff-65b3ebeb1ba4", "name": "58578399-7a04-4d7e-9fff-65b3ebeb1ba4", "status": "Failed", "startTime": "2026-09-24T23:42:40.568397466Z", "endTime": "2026-09-25T00:11:49.734192333Z", "error": { "code": "InternalServerError", "message": "cluster creation did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InvalidRequestContent; detail message Invalid value: "4.21": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is "nightly"; provider Microsoft.RedHatOpenShift 23:00:14 UTC · failed at e2e · test Customer should be able to create an HCP cluster with custom autoscaling and update the autoscaling configuration · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:173]: failed to update cluster "autoscaling-hcp-cluster" autoscaling configuration Unexpected error: <exported.ResponseError \| 0xc0019711d0>: PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/autoscaling-cluster-7dp4spk72w92/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/autoscaling-hcp-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"4.21\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"", "target": "properties.version.id" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:173]: failed to update cluster "autoscaling-hcp-cluster" autoscaling configuration Unexpected error: <exported.ResponseError \| 0xc0019711d0>: PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/autoscaling-cluster-7dp4spk72w92/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/autoscaling-hcp-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"4.21\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"", "target": "properties.version.id" } } -------------------------------------------------------------------------------- ... occurred VerifyKubeAPIServerServerVersionUpgraded failed: kube-apiserver ServerVersion not updated (unchanged from pre-upgrade) 23:00:14 UTC · failed at e2e · test Customer should be able to successfully upgrade control plane minor version from 5.0 minor to 5.1 minor · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:176]: Timed out after 2700.002s. Expected success, but got an error: <errors.joinError \| 0xc0016823c0>: VerifyKubeAPIServerServerVersionUpgraded failed: kube-apiserver ServerVersion not updated (unchanged from pre-upgrade) VerifyHostedControlPlaneYStreamUpgrade(previousMinor=5.0.0-0.nightly-multi-2026-09-24-142420, targetMinor=5.1.0-0.nightly-multi-2026-08-10-113053) failed: clusterversion status.history has no version in target minor "5.1.0-0.nightly-multi-2026-08-10-113053" ... fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:176]: Timed out after 2700.002s. Expected success, but got an error: <errors.joinError \| 0xc0016823c0>: VerifyKubeAPIServerServerVersionUpgraded failed: kube-apiserver ServerVersion not updated (unchanged from pre-upgrade) VerifyHostedControlPlaneYStreamUpgrade(previousMinor=5.0.0-0.nightly-multi-2026-09-24-142420, targetMinor=5.1.0-0.nightly-multi-2026-08-10-113053) failed: clusterversion status.history has no version in target minor "5.1.0-0.nightly-multi-2026-08-10-113053" ... ERROR CODE: InvalidRequestContent; detail message Invalid value: "4.21": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is "nightly"; provider Microsoft.RedHatOpenShift 23:00:14 UTC · failed at e2e · test Authorized CIDRs Connectivity should allow API access only from authorized VM IP · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:408]: failed to begin updating cluster "cidr-connectivity-test" authorized CIDRs Unexpected error: <exported.ResponseError \| 0xc001566b10>: PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-z76xpv9lv7mg/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cidr-connectivity-test -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"4.21\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"", "target": "properties.version.id" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:408]: failed to begin updating cluster "cidr-connectivity-test" authorized CIDRs Unexpected error: <exported.ResponseError \| 0xc001566b10>: PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-z76xpv9lv7mg/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cidr-connectivity-test -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"4.21\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"", "target": "properties.version.id" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InvalidRequestContent; detail message Invalid value: "<ocp-nightly>": must be at least 4.20.8; provider Microsoft.RedHatOpenShift 23:00:14 UTC · failed at e2e · test Customer should upgrade a nodepool skipping one minor version (+2) from 4.20.z to 4.22.zLatest · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:459]: failed to create nodepool nps-420-422 at version 4.20.0-0.nightly-multi-2026-09-23-090319 Unexpected error: <fmt.wrapError \| 0xc000f5c1c0>: failed to create NodePool nps-420-422: failed starting nodepool creation "nps-420-422" for cluster "np-skip-minor-k5htqv" in resourcegroup="rg-np-skip-minor-k5htqv-87tsfgngkbxz": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-skip-minor-k5htqv-87tsfgngkbxz/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-skip-minor-k5htqv/nodePools/nps-420-422 -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"4.20.0-0.nightly-multi-2026-09-23-090319\": must be at least 4.20.8", "target": "properties.version.id" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:459]: failed to create nodepool nps-420-422 at version 4.20.0-0.nightly-multi-2026-09-23-090319 Unexpected error: <fmt.wrapError \| 0xc000f5c1c0>: failed to create NodePool nps-420-422: failed starting nodepool creation "nps-420-422" for cluster "np-skip-minor-k5htqv" in resourcegroup="rg-np-skip-minor-k5htqv-87tsfgngkbxz": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-skip-minor-k5htqv-87tsfgngkbxz/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-skip-minor-k5htqv/nodePools/nps-420-422 -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"4.20.0-0.nightly-multi-2026-09-23-090319\": must be at least 4.20.8", "target": "properties.version.id" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InvalidRequestContent; detail message Invalid value: "4.21": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is "nightly"; provider Microsoft.RedHatOpenShift 23:00:14 UTC · failed at e2e · test Customer should be able to create a cluster and node pools with aggregated advanced features · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:275]: failed to begin cluster "agg-cluster" authorized CIDR update Unexpected error: <exported.ResponseError \| 0xc0021e3bc0>: PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/feature-aggregation-cccvsc896tfj/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/agg-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"4.21\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"", "target": "properties.version.id" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:275]: failed to begin cluster "agg-cluster" authorized CIDR update Unexpected error: <exported.ResponseError \| 0xc0021e3bc0>: PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/feature-aggregation-cccvsc896tfj/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/agg-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"4.21\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"", "target": "properties.version.id" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InvalidRequestContent; detail message Invalid value: "4.21": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is "nightly"; provider Microsoft.RedHatOpenShift 23:00:14 UTC · failed at e2e · test Customer should be able to create an HCP cluster and manage ImageDigestMirrors · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/idms_lifecycle.go:175]: failed to update cluster with second ImageDigestMirror set Unexpected error: <exported.ResponseError \| 0xc0011064b0>: PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/idms-6qptj87wv4cv/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/idms-e2e-hcp-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"4.21\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"", "target": "properties.version.id" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/idms_lifecycle.go:175]: failed to update cluster with second ImageDigestMirror set Unexpected error: <exported.ResponseError \| 0xc0011064b0>: PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/idms-6qptj87wv4cv/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/idms-e2e-hcp-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"4.21\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"", "target": "properties.version.id" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InvalidRequestContent; detail message Invalid value: "<ocp-nightly>": must be at least 4.20.8; provider Microsoft.RedHatOpenShift 23:00:14 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP nightly channel for 4.20 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-g6wkqd" for cluster "cluster-nightly-4-20-g6wkqd" Unexpected error: <fmt.wrapError \| 0xc000926600>: failed to create NodePool np-g6wkqd: failed starting nodepool creation "np-g6wkqd" for cluster "cluster-nightly-4-20-g6wkqd" in resourcegroup="rg-nightly-4-20-2j2bp6ng4gdp": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-nightly-4-20-2j2bp6ng4gdp/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-nightly-4-20-g6wkqd/nodePools/np-g6wkqd -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"4.20.0-0.nightly-multi-2026-09-23-090319\": must be at least 4.20.8", "target": "properties.version.id" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-g6wkqd" for cluster "cluster-nightly-4-20-g6wkqd" Unexpected error: <fmt.wrapError \| 0xc000926600>: failed to create NodePool np-g6wkqd: failed starting nodepool creation "np-g6wkqd" for cluster "cluster-nightly-4-20-g6wkqd" in resourcegroup="rg-nightly-4-20-2j2bp6ng4gdp": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-nightly-4-20-2j2bp6ng4gdp/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-nightly-4-20-g6wkqd/nodePools/np-g6wkqd -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"4.20.0-0.nightly-multi-2026-09-23-090319\": must be at least 4.20.8", "target": "properties.version.id" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InvalidRequestContent; detail message Invalid value: "<ocp-nightly>": must be at least 4.20.8; provider Microsoft.RedHatOpenShift 23:00:14 UTC · failed at e2e · test Customer should downgrade a nodepool to a lower minor version from 4.22.zLatest to 4.20.zLatest · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:766]: failed to update nodepool npdg-422-420 to downgrade target 4.20.0-0.nightly-multi-2026-09-23-090319 Unexpected error: <fmt.wrapError \| 0xc00151a140>: failed to start nodepool "npdg-422-420" update in cluster "np-dg-minor-4pqnxp" resourcegroup="rg-np-dg-minor-4pqnxp-4lnmhzl52tvp": PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-dg-minor-4pqnxp-4lnmhzl52tvp/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-dg-minor-4pqnxp/nodePools/npdg-422-420 -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"4.20.0-0.nightly-multi-2026-09-23-090319\": must be at least 4.20.8", "target": "properties.version.id" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:766]: failed to update nodepool npdg-422-420 to downgrade target 4.20.0-0.nightly-multi-2026-09-23-090319 Unexpected error: <fmt.wrapError \| 0xc00151a140>: failed to start nodepool "npdg-422-420" update in cluster "np-dg-minor-4pqnxp" resourcegroup="rg-np-dg-minor-4pqnxp-4lnmhzl52tvp": PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-dg-minor-4pqnxp-4lnmhzl52tvp/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-dg-minor-4pqnxp/nodePools/npdg-422-420 -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"4.20.0-0.nightly-multi-2026-09-23-090319\": must be at least 4.20.8", "target": "properties.version.id" } } -------------------------------------------------------------------------------- ... occurred storage account log verification failed 23:00:14 UTC · failed at e2e · test Customer should be able to forward control plane logs to a storage account and Event Hub via shoebox diagnostic settings · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/shoebox_logs.go:353]: failed to verify shoebox logs reached storage account shoebox6npjq8 and Event Hub shoebox-eh for cluster /subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/shoebox-cluster-g4qtcfdblkfs/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/shoebox-hcp-cluster Unexpected error: <errors.errorString \| 0xc000d18d20>: storage account log verification failed ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/shoebox_logs.go:353]: failed to verify shoebox logs reached storage account shoebox6npjq8 and Event Hub shoebox-eh for cluster /subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/shoebox-cluster-g4qtcfdblkfs/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/shoebox-hcp-cluster Unexpected error: <errors.errorString \| 0xc000d18d20>: storage account log verification failed ... occurred ERROR CODE: InvalidRequestContent; detail message Invalid value: "4.21": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is "nightly"; provider Microsoft.RedHatOpenShift 23:00:14 UTC · failed at e2e · test Update HCPOpenShiftCluster Positive creates a cluster and updates tags with a PATCH request · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:193]: failed to update HCP cluster tags via PATCH Unexpected error: <exported.ResponseError \| 0xc0013a3320>: PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-tags-cgk7n47ncfl4/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-tags-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"4.21\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"", "target": "properties.version.id" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:193]: failed to update HCP cluster tags via PATCH Unexpected error: <exported.ResponseError \| 0xc0013a3320>: PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-tags-cgk7n47ncfl4/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-tags-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"4.21\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"", "target": "properties.version.id" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message node pool creation did not complete before the deadline; provider Microsoft.RedHatOpenShift 23:00:14 UTC · failed at e2e · test Customer should be able to create an HCP cluster and custom node pool osDisk size · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_nodepool_osdisk.go:96]: failed to create node pool "nodepool-128GiB" with 128GiB osDisk Unexpected error: <fmt.wrapError \| 0xc000d07ae0>: failed to create NodePool nodepool-128GiB: failed waiting for nodepool="nodepool-128GiB" for cluster "hcp-cluster-np-128" in resourcegroup="clusternp128-xxwmzws6kxxc" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/85216ecc-6f15-4ee4-9411-ab10183b2cec -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/85216ecc-6f15-4ee4-9411-ab10183b2cec", "name": "85216ecc-6f15-4ee4-9411-ab10183b2cec", "status": "Failed", "startTime": "2026-09-24T23:13:56.752561651Z", "endTime": "2026-09-24T23:32:59.664565124Z", "error": { "code": "InternalServerError", "message": "node pool creation did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_nodepool_osdisk.go:96]: failed to create node pool "nodepool-128GiB" with 128GiB osDisk Unexpected error: <fmt.wrapError \| 0xc000d07ae0>: failed to create NodePool nodepool-128GiB: failed waiting for nodepool="nodepool-128GiB" for cluster "hcp-cluster-np-128" in resourcegroup="clusternp128-xxwmzws6kxxc" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/85216ecc-6f15-4ee4-9411-ab10183b2cec -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/85216ecc-6f15-4ee4-9411-ab10183b2cec", "name": "85216ecc-6f15-4ee4-9411-ab10183b2cec", "status": "Failed", "startTime": "2026-09-24T23:13:56.752561651Z", "endTime": "2026-09-24T23:32:59.664565124Z", "error": { "code": "InternalServerError", "message": "node pool creation did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred
20:07:48	31m 42s	branch-ci-Azure-ARO-HCP-main-e2e-stage-e2e-parallel post-goodmerged PR No additional run metadata captured	westus3	e2e	failed	n/a	59	e2e (59) ERROR CODE: LocationNotAvailableForResourceType; detail message The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'.; provider Microsoft.RedHatOpenShift 20:07:48 UTC · failed at e2e · test Customer should upgrade and update a nodepool from 4.21.z to 4.21.zLatest · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:146]: failed to create HCP cluster np-version-upgrade-cluster-przs8v with version 4.21.34 Unexpected error: <fmt.wrapError \| 0xc000736620>: failed to create HCP cluster np-version-upgrade-cluster-przs8v: failed starting cluster creation "np-version-upgrade-cluster-przs8v" in resourcegroup="rg-np-version-upgrade-przs8v-vc4ptfztdpvg": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-version-upgrade-przs8v-vc4ptfztdpvg/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-version-upgrade-cluster-przs8v -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:146]: failed to create HCP cluster np-version-upgrade-cluster-przs8v with version 4.21.34 Unexpected error: <fmt.wrapError \| 0xc000736620>: failed to create HCP cluster np-version-upgrade-cluster-przs8v: failed starting cluster creation "np-version-upgrade-cluster-przs8v" in resourcegroup="rg-np-version-upgrade-przs8v-vc4ptfztdpvg": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-version-upgrade-przs8v-vc4ptfztdpvg/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-version-upgrade-cluster-przs8v -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: LocationNotAvailableForResourceType; detail message The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'.; provider Microsoft.RedHatOpenShift 20:07:48 UTC · failed at e2e · test Create HCPOpenShiftCluster with Private KeyVault should create a cluster with private keyvault using v20251223preview API · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:99]: failed to create HCP cluster "private-kv-cluster" with private keyvault Unexpected error: <fmt.wrapError \| 0xc0015650a0>: failed starting cluster creation "private-kv-cluster" in resourcegroup="private-keyvault-8qxdjfjb7mb2": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/private-keyvault-8qxdjfjb7mb2/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/private-kv-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:99]: failed to create HCP cluster "private-kv-cluster" with private keyvault Unexpected error: <fmt.wrapError \| 0xc0015650a0>: failed starting cluster creation "private-kv-cluster" in resourcegroup="private-keyvault-8qxdjfjb7mb2": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/private-keyvault-8qxdjfjb7mb2/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/private-kv-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: LocationNotAvailableForResourceType; detail message The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'.; provider Microsoft.RedHatOpenShift 20:07:48 UTC · failed at e2e · test Customer should be able to successfully upgrade control plane minor version from 4.20 minor to 4.21 minor · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:133]: failed to create HCP cluster "cp-ystream-upgrade-4-21-cskmgv" at install version 4.20 Unexpected error: <fmt.wrapError \| 0xc000a70420>: failed to create HCP cluster cp-ystream-upgrade-4-21-cskmgv: failed starting cluster creation "cp-ystream-upgrade-4-21-cskmgv" in resourcegroup="rg-cp-ystream-upgrade-4-21-4sk4lx7gbhw6": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cp-ystream-upgrade-4-21-4sk4lx7gbhw6/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cp-ystream-upgrade-4-21-cskmgv -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:133]: failed to create HCP cluster "cp-ystream-upgrade-4-21-cskmgv" at install version 4.20 Unexpected error: <fmt.wrapError \| 0xc000a70420>: failed to create HCP cluster cp-ystream-upgrade-4-21-cskmgv: failed starting cluster creation "cp-ystream-upgrade-4-21-cskmgv" in resourcegroup="rg-cp-ystream-upgrade-4-21-4sk4lx7gbhw6": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cp-ystream-upgrade-4-21-4sk4lx7gbhw6/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cp-ystream-upgrade-4-21-cskmgv -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: LocationNotAvailableForResourceType; detail message The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'.; provider Microsoft.RedHatOpenShift 20:07:48 UTC · failed at e2e · test Customer should downgrade a nodepool to a lower minor version from 4.22.zLatest to 4.20.zLatest · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:720]: failed to create HCP cluster Unexpected error: <fmt.wrapError \| 0xc0015a88e0>: failed to create HCP cluster np-dg-minor-zvqbvr: failed starting cluster creation "np-dg-minor-zvqbvr" in resourcegroup="rg-np-dg-minor-zvqbvr-b979m4957htq": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-dg-minor-zvqbvr-b979m4957htq/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-dg-minor-zvqbvr -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:720]: failed to create HCP cluster Unexpected error: <fmt.wrapError \| 0xc0015a88e0>: failed to create HCP cluster np-dg-minor-zvqbvr: failed starting cluster creation "np-dg-minor-zvqbvr" in resourcegroup="rg-np-dg-minor-zvqbvr-b979m4957htq": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-dg-minor-zvqbvr-b979m4957htq/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-dg-minor-zvqbvr -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: NoRegisteredProviderFound; detail message No registered resource provider found for location 'westus3' and API version '2024-06-10-preview' for type 'locations/hcpOpenShiftVersions'. The supported api-versions are '2026-09-01-preview, 2026-06-30-preview, 2025-12-23-preview, 2024-06-10-preview'. The supported locations are 'switzerlandnorth, canadacentral, australiaeast, westeurope, centralindia, brazilsouth, eastus2, uksouth'.; provider Microsoft.RedHatOpenShift 20:07:48 UTC · failed at e2e · test Customer should be able to list available HCP OpenShift versions and validate response content · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_versions.go:43]: failed to list HCP OpenShift versions Unexpected error: <exported.ResponseError \| 0xc0017589f0>: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOpenShiftVersions -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: NoRegisteredProviderFound -------------------------------------------------------------------------------- { "error": { "code": "NoRegisteredProviderFound", "message": "No registered resource provider found for location 'westus3' and API version '2024-06-10-preview' for type 'locations/hcpOpenShiftVersions'. The supported api-versions are '2026-09-01-preview, 2026-06-30-preview, 2025-12-23-preview, 2024-06-10-preview'. The supported locations are 'switzerlandnorth, canadacentral, australiaeast, westeurope, centralindia, brazilsouth, eastus2, uksouth'." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_versions.go:43]: failed to list HCP OpenShift versions Unexpected error: <exported.ResponseError \| 0xc0017589f0>: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westus3/hcpOpenShiftVersions -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: NoRegisteredProviderFound -------------------------------------------------------------------------------- { "error": { "code": "NoRegisteredProviderFound", "message": "No registered resource provider found for location 'westus3' and API version '2024-06-10-preview' for type 'locations/hcpOpenShiftVersions'. The supported api-versions are '2026-09-01-preview, 2026-06-30-preview, 2025-12-23-preview, 2024-06-10-preview'. The supported locations are 'switzerlandnorth, canadacentral, australiaeast, westeurope, centralindia, brazilsouth, eastus2, uksouth'." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: LocationNotAvailableForResourceType; detail message The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'.; provider Microsoft.RedHatOpenShift 20:07:48 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.21 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:123]: HCP cluster rg-candidate-4-21-4t8l4tgxv9hz/cluster-candidate-4-21-2n8mmh should provision Unexpected error: <fmt.wrapError \| 0xc00092c560>: failed to create HCP cluster cluster-candidate-4-21-2n8mmh: failed starting cluster creation "cluster-candidate-4-21-2n8mmh" in resourcegroup="rg-candidate-4-21-4t8l4tgxv9hz": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-4-21-4t8l4tgxv9hz/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-candidate-4-21-2n8mmh -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:123]: HCP cluster rg-candidate-4-21-4t8l4tgxv9hz/cluster-candidate-4-21-2n8mmh should provision Unexpected error: <fmt.wrapError \| 0xc00092c560>: failed to create HCP cluster cluster-candidate-4-21-2n8mmh: failed starting cluster creation "cluster-candidate-4-21-2n8mmh" in resourcegroup="rg-candidate-4-21-4t8l4tgxv9hz": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-4-21-4t8l4tgxv9hz/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-candidate-4-21-2n8mmh -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: LocationNotAvailableForResourceType; detail message The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'.; provider Microsoft.RedHatOpenShift 20:07:48 UTC · failed at e2e · test Customer should be able to create an HCP cluster and manage pull secrets · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_pullsecret.go:113]: failed to create HCP cluster for pull secret test Unexpected error: <fmt.wrapError \| 0xc0011587a0>: failed to create HCP cluster pullsecret-hcp-cluster: failed starting cluster creation "pullsecret-hcp-cluster" in resourcegroup="pullsecret-test-dkc6688q7l7z": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/pullsecret-test-dkc6688q7l7z/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/pullsecret-hcp-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_pullsecret.go:113]: failed to create HCP cluster for pull secret test Unexpected error: <fmt.wrapError \| 0xc0011587a0>: failed to create HCP cluster pullsecret-hcp-cluster: failed starting cluster creation "pullsecret-hcp-cluster" in resourcegroup="pullsecret-test-dkc6688q7l7z": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/pullsecret-test-dkc6688q7l7z/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/pullsecret-hcp-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: LocationNotAvailableForResourceType; detail message The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'.; provider Microsoft.RedHatOpenShift 20:07:48 UTC · failed at e2e · test Customer should be able to use workload identity via the cluster OIDC issuer URL · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/oidc_issuer_workload_identity.go:227]: failed to create HCP cluster oidc-wi-cluster Unexpected error: <fmt.wrapError \| 0xc000c722a0>: failed to create HCP cluster oidc-wi-cluster: failed starting cluster creation "oidc-wi-cluster" in resourcegroup="oidc-wi-fbqvn4k6lw4t": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/oidc-wi-fbqvn4k6lw4t/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/oidc-wi-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/oidc_issuer_workload_identity.go:227]: failed to create HCP cluster oidc-wi-cluster Unexpected error: <fmt.wrapError \| 0xc000c722a0>: failed to create HCP cluster oidc-wi-cluster: failed starting cluster creation "oidc-wi-cluster" in resourcegroup="oidc-wi-fbqvn4k6lw4t": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/oidc-wi-fbqvn4k6lw4t/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/oidc-wi-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: LocationNotAvailableForResourceType; detail message The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'.; provider Microsoft.RedHatOpenShift 20:07:48 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 5.0 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:123]: HCP cluster rg-candidate-5-0-hpp6c5sdhdrk/cluster-candidate-5-0-9n8h68 should provision Unexpected error: <fmt.wrapError \| 0xc000fc0320>: failed to create HCP cluster cluster-candidate-5-0-9n8h68: failed starting cluster creation "cluster-candidate-5-0-9n8h68" in resourcegroup="rg-candidate-5-0-hpp6c5sdhdrk": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-5-0-hpp6c5sdhdrk/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-candidate-5-0-9n8h68 -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:123]: HCP cluster rg-candidate-5-0-hpp6c5sdhdrk/cluster-candidate-5-0-9n8h68 should provision Unexpected error: <fmt.wrapError \| 0xc000fc0320>: failed to create HCP cluster cluster-candidate-5-0-9n8h68: failed starting cluster creation "cluster-candidate-5-0-9n8h68" in resourcegroup="rg-candidate-5-0-hpp6c5sdhdrk": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-5-0-hpp6c5sdhdrk/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-candidate-5-0-9n8h68 -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: LocationNotAvailableForResourceType; detail message The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'.; provider Microsoft.RedHatOpenShift 20:07:48 UTC · failed at e2e · test Customer should create a cluster using v20260901preview API and verify cluster health · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_v20260901preview.go:86]: failed to create HCP cluster "v20260901" via v20260901preview Unexpected error: <fmt.wrapError \| 0xc000f87400>: failed to create HCP cluster v20260901: failed starting cluster creation "v20260901" in resourcegroup="v20260901preview-r58424j4l8b9": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/v20260901preview-r58424j4l8b9/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/v20260901 -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_v20260901preview.go:86]: failed to create HCP cluster "v20260901" via v20260901preview Unexpected error: <fmt.wrapError \| 0xc000f87400>: failed to create HCP cluster v20260901: failed starting cluster creation "v20260901" in resourcegroup="v20260901preview-r58424j4l8b9": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/v20260901preview-r58424j4l8b9/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/v20260901 -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: LocationNotAvailableForResourceType; detail message The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'.; provider Microsoft.RedHatOpenShift 20:07:48 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 4.22 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:123]: failed to create HCP cluster "cluster-zstream-4-22-gbqkd2" with version 4.22.14 on candidate channel Unexpected error: <fmt.wrapError \| 0xc00149a920>: failed to create HCP cluster cluster-zstream-4-22-gbqkd2: failed starting cluster creation "cluster-zstream-4-22-gbqkd2" in resourcegroup="rg-zstream-upgrade-4-22-nqb6nvgsb7ch": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-22-nqb6nvgsb7ch/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-zstream-4-22-gbqkd2 -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:123]: failed to create HCP cluster "cluster-zstream-4-22-gbqkd2" with version 4.22.14 on candidate channel Unexpected error: <fmt.wrapError \| 0xc00149a920>: failed to create HCP cluster cluster-zstream-4-22-gbqkd2: failed starting cluster creation "cluster-zstream-4-22-gbqkd2" in resourcegroup="rg-zstream-upgrade-4-22-nqb6nvgsb7ch": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-22-nqb6nvgsb7ch/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-zstream-4-22-gbqkd2 -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: LocationNotAvailableForResourceType; detail message The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'.; provider Microsoft.RedHatOpenShift 20:07:48 UTC · failed at e2e · test Update HCPOpenShiftCluster Negative creates a cluster and fails to update its name with a PATCH request · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:84]: failed to create HCP cluster for patch-name test Unexpected error: <fmt.wrapError \| 0xc000886a80>: failed to create HCP cluster patch-name-cluster: failed starting cluster creation "patch-name-cluster" in resourcegroup="patch-name-5wbvtl2mnmqn": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-name-5wbvtl2mnmqn/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-name-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:84]: failed to create HCP cluster for patch-name test Unexpected error: <fmt.wrapError \| 0xc000886a80>: failed to create HCP cluster patch-name-cluster: failed starting cluster creation "patch-name-cluster" in resourcegroup="patch-name-5wbvtl2mnmqn": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-name-5wbvtl2mnmqn/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-name-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: LocationNotAvailableForResourceType; detail message The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'.; provider Microsoft.RedHatOpenShift 20:07:48 UTC · failed at e2e · test Customer should be able to successfully upgrade control plane minor version from 4.22 minor to 4.23 minor · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:133]: failed to create HCP cluster "cp-ystream-upgrade-4-23-qxkzz7" at install version 4.22 Unexpected error: <fmt.wrapError \| 0xc000a75ba0>: failed to create HCP cluster cp-ystream-upgrade-4-23-qxkzz7: failed starting cluster creation "cp-ystream-upgrade-4-23-qxkzz7" in resourcegroup="rg-cp-ystream-upgrade-4-23-dzbljwj42bk9": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cp-ystream-upgrade-4-23-dzbljwj42bk9/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cp-ystream-upgrade-4-23-qxkzz7 -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:133]: failed to create HCP cluster "cp-ystream-upgrade-4-23-qxkzz7" at install version 4.22 Unexpected error: <fmt.wrapError \| 0xc000a75ba0>: failed to create HCP cluster cp-ystream-upgrade-4-23-qxkzz7: failed starting cluster creation "cp-ystream-upgrade-4-23-qxkzz7" in resourcegroup="rg-cp-ystream-upgrade-4-23-dzbljwj42bk9": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cp-ystream-upgrade-4-23-dzbljwj42bk9/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cp-ystream-upgrade-4-23-qxkzz7 -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: LocationNotAvailableForResourceType; detail message The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'.; provider Microsoft.RedHatOpenShift 20:07:48 UTC · failed at e2e · test Update HCPOpenShiftCluster Positive creates a cluster and updates tags with a PATCH request · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:161]: failed to create HCP cluster for patch-tags test Unexpected error: <fmt.wrapError \| 0xc000706ce0>: failed to create HCP cluster patch-tags-cluster: failed starting cluster creation "patch-tags-cluster" in resourcegroup="patch-tags-qj5qhprdvftt": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-tags-qj5qhprdvftt/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-tags-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:161]: failed to create HCP cluster for patch-tags test Unexpected error: <fmt.wrapError \| 0xc000706ce0>: failed to create HCP cluster patch-tags-cluster: failed starting cluster creation "patch-tags-cluster" in resourcegroup="patch-tags-qj5qhprdvftt": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-tags-qj5qhprdvftt/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-tags-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: LocationNotAvailableForResourceType; detail message The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'.; provider Microsoft.RedHatOpenShift 20:07:48 UTC · failed at e2e · test Customer should be able to delete a nodepool · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_delete.go:81]: failed to create HCP cluster np-delete-hcp-cluster Unexpected error: <fmt.wrapError \| 0xc001011720>: failed to create HCP cluster np-delete-hcp-cluster: failed starting cluster creation "np-delete-hcp-cluster" in resourcegroup="nodepool-delete-8x4ndpfmx4ct": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/nodepool-delete-8x4ndpfmx4ct/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-delete-hcp-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_delete.go:81]: failed to create HCP cluster np-delete-hcp-cluster Unexpected error: <fmt.wrapError \| 0xc001011720>: failed to create HCP cluster np-delete-hcp-cluster: failed starting cluster creation "np-delete-hcp-cluster" in resourcegroup="nodepool-delete-8x4ndpfmx4ct": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/nodepool-delete-8x4ndpfmx4ct/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-delete-hcp-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: LocationNotAvailableForResourceType; detail message The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'.; provider Microsoft.RedHatOpenShift 20:07:48 UTC · failed at e2e · test Customer should be able to create node pools with ARM64-based VMs · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/arm64_nodepool.go:88]: failed to create HCP cluster "arm64-vm-hcp-cluster" Unexpected error: <fmt.wrapError \| 0xc000fe31e0>: failed to create HCP cluster arm64-vm-hcp-cluster: failed starting cluster creation "arm64-vm-hcp-cluster" in resourcegroup="arm64-vm-cluster-jvstdp6449kc": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/arm64-vm-cluster-jvstdp6449kc/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/arm64-vm-hcp-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/arm64_nodepool.go:88]: failed to create HCP cluster "arm64-vm-hcp-cluster" Unexpected error: <fmt.wrapError \| 0xc000fe31e0>: failed to create HCP cluster arm64-vm-hcp-cluster: failed starting cluster creation "arm64-vm-hcp-cluster" in resourcegroup="arm64-vm-cluster-jvstdp6449kc": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/arm64-vm-cluster-jvstdp6449kc/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/arm64-vm-hcp-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: LocationNotAvailableForResourceType; detail message The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'.; provider Microsoft.RedHatOpenShift 20:07:48 UTC · failed at e2e · test Customer should upgrade and update a nodepool from 4.20.z to 4.20.zLatest · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:146]: failed to create HCP cluster np-version-upgrade-cluster-twdz9c with version 4.20.39 Unexpected error: <fmt.wrapError \| 0xc0001eedc0>: failed to create HCP cluster np-version-upgrade-cluster-twdz9c: failed starting cluster creation "np-version-upgrade-cluster-twdz9c" in resourcegroup="rg-np-version-upgrade-twdz9c-bdc7bcn8g67z": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-version-upgrade-twdz9c-bdc7bcn8g67z/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-version-upgrade-cluster-twdz9c -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:146]: failed to create HCP cluster np-version-upgrade-cluster-twdz9c with version 4.20.39 Unexpected error: <fmt.wrapError \| 0xc0001eedc0>: failed to create HCP cluster np-version-upgrade-cluster-twdz9c: failed starting cluster creation "np-version-upgrade-cluster-twdz9c" in resourcegroup="rg-np-version-upgrade-twdz9c-bdc7bcn8g67z": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-version-upgrade-twdz9c-bdc7bcn8g67z/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-version-upgrade-cluster-twdz9c -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: LocationNotAvailableForResourceType; detail message The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'.; provider Microsoft.RedHatOpenShift 20:07:48 UTC · failed at e2e · test Nodepool Ephemeral OS Disk should create a nodepool with ephemeral OS disk when autoRepair is enabled · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_ephemeral_osdisk.go:84]: failed to create HCP cluster ephemeral-disk Unexpected error: <fmt.wrapError \| 0xc000b7e680>: failed to create HCP cluster ephemeral-disk: failed starting cluster creation "ephemeral-disk" in resourcegroup="ephemeral-osdisk-d5k7vz9fgmp5": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/ephemeral-osdisk-d5k7vz9fgmp5/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/ephemeral-disk -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_ephemeral_osdisk.go:84]: failed to create HCP cluster ephemeral-disk Unexpected error: <fmt.wrapError \| 0xc000b7e680>: failed to create HCP cluster ephemeral-disk: failed starting cluster creation "ephemeral-disk" in resourcegroup="ephemeral-osdisk-d5k7vz9fgmp5": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/ephemeral-osdisk-d5k7vz9fgmp5/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/ephemeral-disk -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: LocationNotAvailableForResourceType; detail message The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'.; provider Microsoft.RedHatOpenShift 20:07:48 UTC · failed at e2e · test Customer should be able to create a cluster with default autoscaling and a nodepool with autoscaling enabled up to replica limits · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:102]: failed to create HCP cluster np-autoscale-cluster Unexpected error: <fmt.wrapError \| 0xc0010837c0>: failed to create HCP cluster np-autoscale-cluster: failed starting cluster creation "np-autoscale-cluster" in resourcegroup="np-autoscaling-d8bbt9k992fx": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/np-autoscaling-d8bbt9k992fx/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-autoscale-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:102]: failed to create HCP cluster np-autoscale-cluster Unexpected error: <fmt.wrapError \| 0xc0010837c0>: failed to create HCP cluster np-autoscale-cluster: failed starting cluster creation "np-autoscale-cluster" in resourcegroup="np-autoscaling-d8bbt9k992fx": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/np-autoscaling-d8bbt9k992fx/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-autoscale-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: LocationNotAvailableForResourceType; detail message The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'.; provider Microsoft.RedHatOpenShift 20:07:48 UTC · failed at e2e · test Customer should be able to successfully upgrade control plane minor version from 4.21 minor to 4.22 minor · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:133]: failed to create HCP cluster "cp-ystream-upgrade-4-22-bk8phf" at install version 4.21 Unexpected error: <fmt.wrapError \| 0xc000fdf180>: failed to create HCP cluster cp-ystream-upgrade-4-22-bk8phf: failed starting cluster creation "cp-ystream-upgrade-4-22-bk8phf" in resourcegroup="rg-cp-ystream-upgrade-4-22-wzs2hgm7sj86": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cp-ystream-upgrade-4-22-wzs2hgm7sj86/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cp-ystream-upgrade-4-22-bk8phf -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:133]: failed to create HCP cluster "cp-ystream-upgrade-4-22-bk8phf" at install version 4.21 Unexpected error: <fmt.wrapError \| 0xc000fdf180>: failed to create HCP cluster cp-ystream-upgrade-4-22-bk8phf: failed starting cluster creation "cp-ystream-upgrade-4-22-bk8phf" in resourcegroup="rg-cp-ystream-upgrade-4-22-wzs2hgm7sj86": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cp-ystream-upgrade-4-22-wzs2hgm7sj86/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cp-ystream-upgrade-4-22-bk8phf -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: LocationNotAvailableForResourceType; detail message The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'.; provider Microsoft.RedHatOpenShift 20:07:48 UTC · failed at e2e · test Customer should create an HCP cluster and validate TLS certificates · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:96]: failed to create HCP cluster for TLS endpoint test Unexpected error: <fmt.wrapError \| 0xc000c456a0>: failed to create HCP cluster tls-endpoint-hcp-cluster: failed starting cluster creation "tls-endpoint-hcp-cluster" in resourcegroup="tls-endpoint-cluster-8nq9pchcqchg": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/tls-endpoint-cluster-8nq9pchcqchg/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/tls-endpoint-hcp-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:96]: failed to create HCP cluster for TLS endpoint test Unexpected error: <fmt.wrapError \| 0xc000c456a0>: failed to create HCP cluster tls-endpoint-hcp-cluster: failed starting cluster creation "tls-endpoint-hcp-cluster" in resourcegroup="tls-endpoint-cluster-8nq9pchcqchg": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/tls-endpoint-cluster-8nq9pchcqchg/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/tls-endpoint-hcp-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: LocationNotAvailableForResourceType; detail message The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'.; provider Microsoft.RedHatOpenShift 20:07:48 UTC · failed at e2e · test FIPS Mode Support with v2026-09-01 API should create an HCP cluster with FIPS mode enabled via cryptoRestrictions property · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_fips_mode.go:87]: failed to create HCP cluster "fips-enabled-cluster" with cryptoRestrictions set to FIPS Unexpected error: <fmt.wrapError \| 0xc000fdf860>: failed starting cluster creation "fips-enabled-cluster" in resourcegroup="fips-enabled-4xdxfr96vsdr": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/fips-enabled-4xdxfr96vsdr/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/fips-enabled-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_fips_mode.go:87]: failed to create HCP cluster "fips-enabled-cluster" with cryptoRestrictions set to FIPS Unexpected error: <fmt.wrapError \| 0xc000fdf860>: failed starting cluster creation "fips-enabled-cluster" in resourcegroup="fips-enabled-4xdxfr96vsdr": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/fips-enabled-4xdxfr96vsdr/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/fips-enabled-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: LocationNotAvailableForResourceType; detail message The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'.; provider Microsoft.RedHatOpenShift 20:07:48 UTC · failed at e2e · test Authorized CIDRs Connectivity should allow API access only from authorized VM IP · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:111]: failed to create HCP cluster "cidr-connectivity-test" with authorized CIDRs Unexpected error: <fmt.wrapError \| 0xc000f47340>: failed to create HCP cluster cidr-connectivity-test: failed starting cluster creation "cidr-connectivity-test" in resourcegroup="e2e-cidr-connectivity-s75gnvmnmmhn": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-s75gnvmnmmhn/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cidr-connectivity-test -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:111]: failed to create HCP cluster "cidr-connectivity-test" with authorized CIDRs Unexpected error: <fmt.wrapError \| 0xc000f47340>: failed to create HCP cluster cidr-connectivity-test: failed starting cluster creation "cidr-connectivity-test" in resourcegroup="e2e-cidr-connectivity-s75gnvmnmmhn": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-s75gnvmnmmhn/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cidr-connectivity-test -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: LocationNotAvailableForResourceType; detail message The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'.; provider Microsoft.RedHatOpenShift 20:07:48 UTC · failed at e2e · test Customer should be able to create an HCP cluster and custom node pool osDisk size · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_nodepool_osdisk.go:80]: failed to create HCP cluster "hcp-cluster-np-128" Unexpected error: <fmt.wrapError \| 0xc001121380>: failed to create HCP cluster hcp-cluster-np-128: failed starting cluster creation "hcp-cluster-np-128" in resourcegroup="clusternp128-c25p8tntx7z6": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/clusternp128-c25p8tntx7z6/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/hcp-cluster-np-128 -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_nodepool_osdisk.go:80]: failed to create HCP cluster "hcp-cluster-np-128" Unexpected error: <fmt.wrapError \| 0xc001121380>: failed to create HCP cluster hcp-cluster-np-128: failed starting cluster creation "hcp-cluster-np-128" in resourcegroup="clusternp128-c25p8tntx7z6": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/clusternp128-c25p8tntx7z6/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/hcp-cluster-np-128 -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: LocationNotAvailableForResourceType; detail message The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'.; provider Microsoft.RedHatOpenShift 20:07:48 UTC · failed at e2e · test Customer should upgrade and update a nodepool from 4.20.z to 4.21.zLatest · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:146]: failed to create HCP cluster np-version-upgrade-cluster-mgpwxh with version 4.21.34 Unexpected error: <fmt.wrapError \| 0xc001978d80>: failed to create HCP cluster np-version-upgrade-cluster-mgpwxh: failed starting cluster creation "np-version-upgrade-cluster-mgpwxh" in resourcegroup="rg-np-version-upgrade-mgpwxh-62rzgbpsmm8q": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-version-upgrade-mgpwxh-62rzgbpsmm8q/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-version-upgrade-cluster-mgpwxh -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:146]: failed to create HCP cluster np-version-upgrade-cluster-mgpwxh with version 4.21.34 Unexpected error: <fmt.wrapError \| 0xc001978d80>: failed to create HCP cluster np-version-upgrade-cluster-mgpwxh: failed starting cluster creation "np-version-upgrade-cluster-mgpwxh" in resourcegroup="rg-np-version-upgrade-mgpwxh-62rzgbpsmm8q": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-version-upgrade-mgpwxh-62rzgbpsmm8q/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-version-upgrade-cluster-mgpwxh -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: LocationNotAvailableForResourceType; detail message The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'.; provider Microsoft.RedHatOpenShift 20:07:48 UTC · failed at e2e · test Customer should be able to lifecycle and confirm external auth on a cluster · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_list_and_verify.go:87]: failed to create HCP cluster for external auth list test Unexpected error: <fmt.wrapError \| 0xc000825b40>: failed to create HCP cluster ea-listtblh6f: failed starting cluster creation "ea-listtblh6f" in resourcegroup="ea-list-vc7smgt572nt": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/ea-list-vc7smgt572nt/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/ea-listtblh6f -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_list_and_verify.go:87]: failed to create HCP cluster for external auth list test Unexpected error: <fmt.wrapError \| 0xc000825b40>: failed to create HCP cluster ea-listtblh6f: failed starting cluster creation "ea-listtblh6f" in resourcegroup="ea-list-vc7smgt572nt": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/ea-list-vc7smgt572nt/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/ea-listtblh6f -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: LocationNotAvailableForResourceType; detail message The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'.; provider Microsoft.RedHatOpenShift 20:07:48 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 4.20 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:123]: failed to create HCP cluster "cluster-zstream-4-20-5zxmn6" with version 4.20.38 on candidate channel Unexpected error: <fmt.wrapError \| 0xc000e7afc0>: failed to create HCP cluster cluster-zstream-4-20-5zxmn6: failed starting cluster creation "cluster-zstream-4-20-5zxmn6" in resourcegroup="rg-zstream-upgrade-4-20-87qdz9hglgx4": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-20-87qdz9hglgx4/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-zstream-4-20-5zxmn6 -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:123]: failed to create HCP cluster "cluster-zstream-4-20-5zxmn6" with version 4.20.38 on candidate channel Unexpected error: <fmt.wrapError \| 0xc000e7afc0>: failed to create HCP cluster cluster-zstream-4-20-5zxmn6: failed starting cluster creation "cluster-zstream-4-20-5zxmn6" in resourcegroup="rg-zstream-upgrade-4-20-87qdz9hglgx4": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-20-87qdz9hglgx4/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-zstream-4-20-5zxmn6 -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: LocationNotAvailableForResourceType; detail message The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'.; provider Microsoft.RedHatOpenShift 20:07:48 UTC · failed at e2e · test Customer should be able to create a HCP cluster and use cilium CNI plugin · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_cni_cilium.go:88]: failed to create HCP cluster "cilium-cl" without CNI Unexpected error: <fmt.wrapError \| 0xc000fe4b00>: failed to create HCP cluster cilium-cl: failed starting cluster creation "cilium-cl" in resourcegroup="cni-cilium-jjx6z2lgm28f": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/cni-cilium-jjx6z2lgm28f/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cilium-cl -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_cni_cilium.go:88]: failed to create HCP cluster "cilium-cl" without CNI Unexpected error: <fmt.wrapError \| 0xc000fe4b00>: failed to create HCP cluster cilium-cl: failed starting cluster creation "cilium-cl" in resourcegroup="cni-cilium-jjx6z2lgm28f": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/cni-cilium-jjx6z2lgm28f/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cilium-cl -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: LocationNotAvailableForResourceType; detail message The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'.; provider Microsoft.RedHatOpenShift 20:07:48 UTC · failed at e2e · test Customer should be able to issue the default ingress serving certificate from Let's Encrypt using cert-manager with ACME DNS-01 challenges · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_ingress_cert_letsencrypt.go:124]: failed to create HCP cluster Unexpected error: <fmt.wrapError \| 0xc000e037e0>: failed to create HCP cluster ingress-cert-le: failed starting cluster creation "ingress-cert-le" in resourcegroup="ingress-cert-le-dnt66gjnv8ns": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/ingress-cert-le-dnt66gjnv8ns/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/ingress-cert-le -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_ingress_cert_letsencrypt.go:124]: failed to create HCP cluster Unexpected error: <fmt.wrapError \| 0xc000e037e0>: failed to create HCP cluster ingress-cert-le: failed starting cluster creation "ingress-cert-le" in resourcegroup="ingress-cert-le-dnt66gjnv8ns": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/ingress-cert-le-dnt66gjnv8ns/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/ingress-cert-le -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: LocationNotAvailableForResourceType; detail message The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'.; provider Microsoft.RedHatOpenShift 20:07:48 UTC · failed at e2e · test Customer should downgrade a nodepool version z-stream downgrade from 4.21.zLatest to 4.21.zPrevious · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:580]: failed to create HCP cluster Unexpected error: <fmt.wrapError \| 0xc0001cd6e0>: failed to create HCP cluster np-downgrade-75gbkn: failed starting cluster creation "np-downgrade-75gbkn" in resourcegroup="rg-np-downgrade-75gbkn-lrrpbz298xcd": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-downgrade-75gbkn-lrrpbz298xcd/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-downgrade-75gbkn -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:580]: failed to create HCP cluster Unexpected error: <fmt.wrapError \| 0xc0001cd6e0>: failed to create HCP cluster np-downgrade-75gbkn: failed starting cluster creation "np-downgrade-75gbkn" in resourcegroup="rg-np-downgrade-75gbkn-lrrpbz298xcd": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-downgrade-75gbkn-lrrpbz298xcd/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-downgrade-75gbkn -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: LocationNotAvailableForResourceType; detail message The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'.; provider Microsoft.RedHatOpenShift 20:07:48 UTC · failed at e2e · test Customer should not be able to reuse subnets and NSGs between clusters · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_nsg_subnet_reuse.go:107]: failed to create seed HCP cluster basic-cluster Unexpected error: <fmt.wrapError \| 0xc000a68fc0>: failed to create HCP cluster basic-cluster: failed starting cluster creation "basic-cluster" in resourcegroup="rg-cluster-nsg-subnet-reuse-w7ldgfwkn8pz": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cluster-nsg-subnet-reuse-w7ldgfwkn8pz/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/basic-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_nsg_subnet_reuse.go:107]: failed to create seed HCP cluster basic-cluster Unexpected error: <fmt.wrapError \| 0xc000a68fc0>: failed to create HCP cluster basic-cluster: failed starting cluster creation "basic-cluster" in resourcegroup="rg-cluster-nsg-subnet-reuse-w7ldgfwkn8pz": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cluster-nsg-subnet-reuse-w7ldgfwkn8pz/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/basic-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: LocationNotAvailableForResourceType; detail message The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'.; provider Microsoft.RedHatOpenShift 20:07:48 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 4.21 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:123]: failed to create HCP cluster "cluster-zstream-4-21-gwg2rx" with version 4.21.33 on candidate channel Unexpected error: <fmt.wrapError \| 0xc000e86f00>: failed to create HCP cluster cluster-zstream-4-21-gwg2rx: failed starting cluster creation "cluster-zstream-4-21-gwg2rx" in resourcegroup="rg-zstream-upgrade-4-21-sdrclvvxd89m": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-21-sdrclvvxd89m/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-zstream-4-21-gwg2rx -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:123]: failed to create HCP cluster "cluster-zstream-4-21-gwg2rx" with version 4.21.33 on candidate channel Unexpected error: <fmt.wrapError \| 0xc000e86f00>: failed to create HCP cluster cluster-zstream-4-21-gwg2rx: failed starting cluster creation "cluster-zstream-4-21-gwg2rx" in resourcegroup="rg-zstream-upgrade-4-21-sdrclvvxd89m": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-21-sdrclvvxd89m/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-zstream-4-21-gwg2rx -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: LocationNotAvailableForResourceType; detail message The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'.; provider Microsoft.RedHatOpenShift 20:07:48 UTC · failed at e2e · test Customer should create a cluster with private KAS and verify API server is only reachable from VNet while ingress remains public · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:91]: failed to create HCP cluster "private-kas" with private KAS Unexpected error: <fmt.wrapError \| 0xc000881a00>: failed to create HCP cluster private-kas: failed starting cluster creation "private-kas" in resourcegroup="private-kas-8m56dcvx9lzw": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/private-kas-8m56dcvx9lzw/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/private-kas -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:91]: failed to create HCP cluster "private-kas" with private KAS Unexpected error: <fmt.wrapError \| 0xc000881a00>: failed to create HCP cluster private-kas: failed starting cluster creation "private-kas" in resourcegroup="private-kas-8m56dcvx9lzw": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/private-kas-8m56dcvx9lzw/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/private-kas -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: LocationNotAvailableForResourceType; detail message The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'.; provider Microsoft.RedHatOpenShift 20:07:48 UTC · failed at e2e · test Customer should be able to create several HCP clusters in their customer resource group, but not in the same managed resource group · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:111]: failed to begin creation of first cluster "basic-hcp-cluster" Unexpected error: <fmt.wrapError \| 0xc000e74100>: failed starting cluster creation "basic-hcp-cluster" in resourcegroup="customer-rg-fw89pw968mp7": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/customer-rg-fw89pw968mp7/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/basic-hcp-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:111]: failed to begin creation of first cluster "basic-hcp-cluster" Unexpected error: <fmt.wrapError \| 0xc000e74100>: failed starting cluster creation "basic-hcp-cluster" in resourcegroup="customer-rg-fw89pw968mp7": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/customer-rg-fw89pw968mp7/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/basic-hcp-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: LocationNotAvailableForResourceType; detail message The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'.; provider Microsoft.RedHatOpenShift 20:07:48 UTC · failed at e2e · test Customer should be able to create an HCP cluster with Image Registry not present · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/image_registry_cluster_create.go:89]: failed to create HCP cluster with image registry disabled Unexpected error: <fmt.wrapError \| 0xc000a798c0>: failed to create HCP cluster disabled-image-registry-hcp-cluster: failed starting cluster creation "disabled-image-registry-hcp-cluster" in resourcegroup="disabled-image-registry-dnzscjfgzjmg": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/disabled-image-registry-dnzscjfgzjmg/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/disabled-image-registry-hcp-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/image_registry_cluster_create.go:89]: failed to create HCP cluster with image registry disabled Unexpected error: <fmt.wrapError \| 0xc000a798c0>: failed to create HCP cluster disabled-image-registry-hcp-cluster: failed starting cluster creation "disabled-image-registry-hcp-cluster" in resourcegroup="disabled-image-registry-dnzscjfgzjmg": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/disabled-image-registry-dnzscjfgzjmg/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/disabled-image-registry-hcp-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: LocationNotAvailableForResourceType; detail message The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'.; provider Microsoft.RedHatOpenShift 20:07:48 UTC · failed at e2e · test Customer should be able to create a no-CNI private cluster with a private key vault, a nodepool and install cilium CNI successfully · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:107]: failed to create HCP cluster "cilium-cluster" with no CNI and private etcd Unexpected error: <fmt.wrapError \| 0xc000b70d40>: failed starting cluster creation "cilium-cluster" in resourcegroup="complex-cilium-kv-nkm54tdkjl42": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/complex-cilium-kv-nkm54tdkjl42/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cilium-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:107]: failed to create HCP cluster "cilium-cluster" with no CNI and private etcd Unexpected error: <fmt.wrapError \| 0xc000b70d40>: failed starting cluster creation "cilium-cluster" in resourcegroup="complex-cilium-kv-nkm54tdkjl42": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/complex-cilium-kv-nkm54tdkjl42/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cilium-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: LocationNotAvailableForResourceType; detail message The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'.; provider Microsoft.RedHatOpenShift 20:07:48 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.22 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:123]: HCP cluster rg-candidate-4-22-2hgvnr5nrxqw/cluster-candidate-4-22-kqvcgg should provision Unexpected error: <fmt.wrapError \| 0xc000bece20>: failed to create HCP cluster cluster-candidate-4-22-kqvcgg: failed starting cluster creation "cluster-candidate-4-22-kqvcgg" in resourcegroup="rg-candidate-4-22-2hgvnr5nrxqw": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-4-22-2hgvnr5nrxqw/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-candidate-4-22-kqvcgg -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:123]: HCP cluster rg-candidate-4-22-2hgvnr5nrxqw/cluster-candidate-4-22-kqvcgg should provision Unexpected error: <fmt.wrapError \| 0xc000bece20>: failed to create HCP cluster cluster-candidate-4-22-kqvcgg: failed starting cluster creation "cluster-candidate-4-22-kqvcgg" in resourcegroup="rg-candidate-4-22-2hgvnr5nrxqw": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-4-22-2hgvnr5nrxqw/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-candidate-4-22-kqvcgg -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: LocationNotAvailableForResourceType; detail message The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'.; provider Microsoft.RedHatOpenShift 20:07:48 UTC · failed at e2e · test Customer should be able to cordon, drain, and uncordon a node in an HCP cluster · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_cordon.go:90]: failed to create HCP cluster cordon-hcp-cluster Unexpected error: <fmt.wrapError \| 0xc000bc4c20>: failed to create HCP cluster cordon-hcp-cluster: failed starting cluster creation "cordon-hcp-cluster" in resourcegroup="rg-cordon-test-q4fkwqfnvpzh": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cordon-test-q4fkwqfnvpzh/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cordon-hcp-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_cordon.go:90]: failed to create HCP cluster cordon-hcp-cluster Unexpected error: <fmt.wrapError \| 0xc000bc4c20>: failed to create HCP cluster cordon-hcp-cluster: failed starting cluster creation "cordon-hcp-cluster" in resourcegroup="rg-cordon-test-q4fkwqfnvpzh": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cordon-test-q4fkwqfnvpzh/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cordon-hcp-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: LocationNotAvailableForResourceType; detail message The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'.; provider Microsoft.RedHatOpenShift 20:07:48 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.20 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:123]: HCP cluster rg-candidate-4-20-jv7lbnk8wtqv/cluster-candidate-4-20-z76tdl should provision Unexpected error: <fmt.wrapError \| 0xc0017160e0>: failed to create HCP cluster cluster-candidate-4-20-z76tdl: failed starting cluster creation "cluster-candidate-4-20-z76tdl" in resourcegroup="rg-candidate-4-20-jv7lbnk8wtqv": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-4-20-jv7lbnk8wtqv/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-candidate-4-20-z76tdl -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:123]: HCP cluster rg-candidate-4-20-jv7lbnk8wtqv/cluster-candidate-4-20-z76tdl should provision Unexpected error: <fmt.wrapError \| 0xc0017160e0>: failed to create HCP cluster cluster-candidate-4-20-z76tdl: failed starting cluster creation "cluster-candidate-4-20-z76tdl" in resourcegroup="rg-candidate-4-20-jv7lbnk8wtqv": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-4-20-jv7lbnk8wtqv/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-candidate-4-20-z76tdl -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: LocationNotAvailableForResourceType; detail message The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'.; provider Microsoft.RedHatOpenShift 20:07:48 UTC · failed at e2e · test Customer should be able to update node pool labels and taints · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_labels_taints.go:92]: failed to create HCP cluster cluster-np-labels-taints Unexpected error: <fmt.wrapError \| 0xc0002196c0>: failed to create HCP cluster cluster-np-labels-taints: failed starting cluster creation "cluster-np-labels-taints" in resourcegroup="rg-np-labels-taints-4vpw2jpcb2x9": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-labels-taints-4vpw2jpcb2x9/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-np-labels-taints -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_labels_taints.go:92]: failed to create HCP cluster cluster-np-labels-taints Unexpected error: <fmt.wrapError \| 0xc0002196c0>: failed to create HCP cluster cluster-np-labels-taints: failed starting cluster creation "cluster-np-labels-taints" in resourcegroup="rg-np-labels-taints-4vpw2jpcb2x9": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-labels-taints-4vpw2jpcb2x9/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-np-labels-taints -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: LocationNotAvailableForResourceType; detail message The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'.; provider Microsoft.RedHatOpenShift 20:07:48 UTC · failed at e2e · test Customer should not be able to perform various invalid operations on cluster resources · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/simple_negative_cases.go:110]: failed to create HCP cluster negative-tests-cluster Unexpected error: <fmt.wrapError \| 0xc000da3060>: failed to create HCP cluster negative-tests-cluster: failed starting cluster creation "negative-tests-cluster" in resourcegroup="rg-negative-tests-58qzvqwc5m7v": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-negative-tests-58qzvqwc5m7v/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/negative-tests-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/simple_negative_cases.go:110]: failed to create HCP cluster negative-tests-cluster Unexpected error: <fmt.wrapError \| 0xc000da3060>: failed to create HCP cluster negative-tests-cluster: failed starting cluster creation "negative-tests-cluster" in resourcegroup="rg-negative-tests-58qzvqwc5m7v": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-negative-tests-58qzvqwc5m7v/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/negative-tests-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: LocationNotAvailableForResourceType; detail message The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'.; provider Microsoft.RedHatOpenShift 20:07:48 UTC · failed at e2e · test Customer should be able to forward control plane logs to a storage account and Event Hub via shoebox diagnostic settings · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/shoebox_logs.go:272]: failed to create HCP cluster shoebox-hcp-cluster Unexpected error: <fmt.wrapError \| 0xc0011e97c0>: failed to create HCP cluster shoebox-hcp-cluster: failed starting cluster creation "shoebox-hcp-cluster" in resourcegroup="shoebox-cluster-794wg6jp9b8f": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/shoebox-cluster-794wg6jp9b8f/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/shoebox-hcp-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/shoebox_logs.go:272]: failed to create HCP cluster shoebox-hcp-cluster Unexpected error: <fmt.wrapError \| 0xc0011e97c0>: failed to create HCP cluster shoebox-hcp-cluster: failed starting cluster creation "shoebox-hcp-cluster" in resourcegroup="shoebox-cluster-794wg6jp9b8f": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/shoebox-cluster-794wg6jp9b8f/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/shoebox-hcp-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: LocationNotAvailableForResourceType; detail message The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'.; provider Microsoft.RedHatOpenShift 20:07:48 UTC · failed at e2e · test Customer should respect cluster-wide node limits with nodepool autoscaling · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:291]: failed to create HCP cluster node-limit-cluster with MaxNodesTotal limit Unexpected error: <fmt.wrapError \| 0xc000c87ba0>: failed to create HCP cluster node-limit-cluster: failed starting cluster creation "node-limit-cluster" in resourcegroup="node-limit-test-7cj42nbzch5r": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/node-limit-test-7cj42nbzch5r/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/node-limit-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:291]: failed to create HCP cluster node-limit-cluster with MaxNodesTotal limit Unexpected error: <fmt.wrapError \| 0xc000c87ba0>: failed to create HCP cluster node-limit-cluster: failed starting cluster creation "node-limit-cluster" in resourcegroup="node-limit-test-7cj42nbzch5r": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/node-limit-test-7cj42nbzch5r/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/node-limit-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: LocationNotAvailableForResourceType; detail message The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'.; provider Microsoft.RedHatOpenShift 20:07:48 UTC · failed at e2e · test Customer should be able to create an HCP cluster and manage ImageDigestMirrors · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/idms_lifecycle.go:113]: failed to create HCP cluster with ImageDigestMirrors Unexpected error: <fmt.wrapError \| 0xc001780ce0>: failed to create HCP cluster idms-e2e-hcp-cluster: failed starting cluster creation "idms-e2e-hcp-cluster" in resourcegroup="idms-b2v8ts8xj59w": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/idms-b2v8ts8xj59w/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/idms-e2e-hcp-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/idms_lifecycle.go:113]: failed to create HCP cluster with ImageDigestMirrors Unexpected error: <fmt.wrapError \| 0xc001780ce0>: failed to create HCP cluster idms-e2e-hcp-cluster: failed starting cluster creation "idms-e2e-hcp-cluster" in resourcegroup="idms-b2v8ts8xj59w": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/idms-b2v8ts8xj59w/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/idms-e2e-hcp-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: LocationNotAvailableForResourceType; detail message The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'.; provider Microsoft.RedHatOpenShift 20:07:48 UTC · failed at e2e · test Customer should be able to create a cluster with an external auth config and get the external auth config · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:94]: failed to create HCP cluster for external auth test Unexpected error: <fmt.wrapError \| 0xc000e02ca0>: failed to create HCP cluster ea-cluster: failed starting cluster creation "ea-cluster" in resourcegroup="external-auth-cluster-7cp2xxm744zd": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/external-auth-cluster-7cp2xxm744zd/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/ea-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:94]: failed to create HCP cluster for external auth test Unexpected error: <fmt.wrapError \| 0xc000e02ca0>: failed to create HCP cluster ea-cluster: failed starting cluster creation "ea-cluster" in resourcegroup="external-auth-cluster-7cp2xxm744zd": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/external-auth-cluster-7cp2xxm744zd/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/ea-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: LocationNotAvailableForResourceType; detail message The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'.; provider Microsoft.RedHatOpenShift 20:07:48 UTC · failed at e2e · test Customer should create a cluster with private ingress using v20260901preview and verify the ingress is internal · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_ingress.go:87]: failed to create HCP cluster "private-ingress" with private ingress Unexpected error: <fmt.wrapError \| 0xc000b6e7c0>: failed to create HCP cluster private-ingress: failed starting cluster creation "private-ingress" in resourcegroup="private-ingress-sh2zd6mpft2k": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/private-ingress-sh2zd6mpft2k/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/private-ingress -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_ingress.go:87]: failed to create HCP cluster "private-ingress" with private ingress Unexpected error: <fmt.wrapError \| 0xc000b6e7c0>: failed to create HCP cluster private-ingress: failed starting cluster creation "private-ingress" in resourcegroup="private-ingress-sh2zd6mpft2k": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/private-ingress-sh2zd6mpft2k/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/private-ingress -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: LocationNotAvailableForResourceType; detail message The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'.; provider Microsoft.RedHatOpenShift 20:07:48 UTC · failed at e2e · test Customer should be able to create an HCP cluster with back-level version 4.20 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:152]: failed to create HCP cluster version 4.20 Unexpected error: <fmt.wrapError \| 0xc00163f5c0>: failed starting cluster creation "cluster-ver-4-20" in resourcegroup="rg-cluster-back-version-dnkpdfkrzkfr": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cluster-back-version-dnkpdfkrzkfr/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-ver-4-20 -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:152]: failed to create HCP cluster version 4.20 Unexpected error: <fmt.wrapError \| 0xc00163f5c0>: failed starting cluster creation "cluster-ver-4-20" in resourcegroup="rg-cluster-back-version-dnkpdfkrzkfr": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cluster-back-version-dnkpdfkrzkfr/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-ver-4-20 -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: LocationNotAvailableForResourceType; detail message The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'.; provider Microsoft.RedHatOpenShift 20:07:48 UTC · failed at e2e · test Customer should be able to delete an HCP cluster whose managed identities were deleted first · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delete_missing_identities.go:113]: failed to create HCP cluster "missing-mi-hcp-cluster" Unexpected error: <fmt.wrapError \| 0xc000fcef00>: failed to create HCP cluster missing-mi-hcp-cluster: failed starting cluster creation "missing-mi-hcp-cluster" in resourcegroup="delete-mi-rg-nvddkl6rpvcr": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/delete-mi-rg-nvddkl6rpvcr/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/missing-mi-hcp-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delete_missing_identities.go:113]: failed to create HCP cluster "missing-mi-hcp-cluster" Unexpected error: <fmt.wrapError \| 0xc000fcef00>: failed to create HCP cluster missing-mi-hcp-cluster: failed starting cluster creation "missing-mi-hcp-cluster" in resourcegroup="delete-mi-rg-nvddkl6rpvcr": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/delete-mi-rg-nvddkl6rpvcr/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/missing-mi-hcp-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: LocationNotAvailableForResourceType; detail message The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'.; provider Microsoft.RedHatOpenShift 20:07:48 UTC · failed at e2e · test Customer should be able to successfully upgrade control plane minor version from 4.22 minor to 5.0 minor · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:133]: failed to create HCP cluster "cp-ystream-upgrade-5-0-bn5fww" at install version 4.22 Unexpected error: <fmt.wrapError \| 0xc000d9b800>: failed to create HCP cluster cp-ystream-upgrade-5-0-bn5fww: failed starting cluster creation "cp-ystream-upgrade-5-0-bn5fww" in resourcegroup="rg-cp-ystream-upgrade-5-0-nphnd2d8z9tk": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cp-ystream-upgrade-5-0-nphnd2d8z9tk/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cp-ystream-upgrade-5-0-bn5fww -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:133]: failed to create HCP cluster "cp-ystream-upgrade-5-0-bn5fww" at install version 4.22 Unexpected error: <fmt.wrapError \| 0xc000d9b800>: failed to create HCP cluster cp-ystream-upgrade-5-0-bn5fww: failed starting cluster creation "cp-ystream-upgrade-5-0-bn5fww" in resourcegroup="rg-cp-ystream-upgrade-5-0-nphnd2d8z9tk": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cp-ystream-upgrade-5-0-nphnd2d8z9tk/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cp-ystream-upgrade-5-0-bn5fww -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: LocationNotAvailableForResourceType; detail message The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'.; provider Microsoft.RedHatOpenShift 20:07:48 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 5.0 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:123]: failed to create HCP cluster "cluster-zstream-5-0-bnc5gf" with version 5.0.0-rc.2 on candidate channel Unexpected error: <fmt.wrapError \| 0xc00083ee20>: failed to create HCP cluster cluster-zstream-5-0-bnc5gf: failed starting cluster creation "cluster-zstream-5-0-bnc5gf" in resourcegroup="rg-zstream-upgrade-5-0-spmmbkw6q2j9": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-5-0-spmmbkw6q2j9/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-zstream-5-0-bnc5gf -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:123]: failed to create HCP cluster "cluster-zstream-5-0-bnc5gf" with version 5.0.0-rc.2 on candidate channel Unexpected error: <fmt.wrapError \| 0xc00083ee20>: failed to create HCP cluster cluster-zstream-5-0-bnc5gf: failed starting cluster creation "cluster-zstream-5-0-bnc5gf" in resourcegroup="rg-zstream-upgrade-5-0-spmmbkw6q2j9": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-5-0-spmmbkw6q2j9/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-zstream-5-0-bnc5gf -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: LocationNotAvailableForResourceType; detail message The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'.; provider Microsoft.RedHatOpenShift 20:07:48 UTC · failed at e2e · test Customer should be able to retrieve cluster and nodepool status active versions · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_nodepool_active_versions.go:89]: failed to create HCP cluster "v20261001" via v20261001preview Unexpected error: <fmt.wrapError \| 0xc001220680>: failed to create HCP cluster v20261001: failed starting cluster creation "v20261001" in resourcegroup="v20261001preview-pwf9pxbkvk9w": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/v20261001preview-pwf9pxbkvk9w/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/v20261001 -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_nodepool_active_versions.go:89]: failed to create HCP cluster "v20261001" via v20261001preview Unexpected error: <fmt.wrapError \| 0xc001220680>: failed to create HCP cluster v20261001: failed starting cluster creation "v20261001" in resourcegroup="v20261001preview-pwf9pxbkvk9w": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/v20261001preview-pwf9pxbkvk9w/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/v20261001 -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: LocationNotAvailableForResourceType; detail message The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'.; provider Microsoft.RedHatOpenShift 20:07:48 UTC · failed at e2e · test Customer should upgrade a nodepool to a version without Cincinnati upgrade edge z-stream upgrade without Cincinnati edge in 4.20 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:301]: failed to create HCP cluster np-noedge-25fcqw Unexpected error: <fmt.wrapError \| 0xc000f12e80>: failed to create HCP cluster np-noedge-25fcqw: failed starting cluster creation "np-noedge-25fcqw" in resourcegroup="rg-np-noedge-25fcqw-mkk44mbtzfcl": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-noedge-25fcqw-mkk44mbtzfcl/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-noedge-25fcqw -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:301]: failed to create HCP cluster np-noedge-25fcqw Unexpected error: <fmt.wrapError \| 0xc000f12e80>: failed to create HCP cluster np-noedge-25fcqw: failed starting cluster creation "np-noedge-25fcqw" in resourcegroup="rg-np-noedge-25fcqw-mkk44mbtzfcl": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-noedge-25fcqw-mkk44mbtzfcl/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-noedge-25fcqw -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: LocationNotAvailableForResourceType; detail message The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'.; provider Microsoft.RedHatOpenShift 20:07:48 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.23 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:123]: HCP cluster rg-candidate-4-23-ttflt6t5z8w7/cluster-candidate-4-23-kznbgd should provision Unexpected error: <fmt.wrapError \| 0xc000d89720>: failed to create HCP cluster cluster-candidate-4-23-kznbgd: failed starting cluster creation "cluster-candidate-4-23-kznbgd" in resourcegroup="rg-candidate-4-23-ttflt6t5z8w7": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-4-23-ttflt6t5z8w7/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-candidate-4-23-kznbgd -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:123]: HCP cluster rg-candidate-4-23-ttflt6t5z8w7/cluster-candidate-4-23-kznbgd should provision Unexpected error: <fmt.wrapError \| 0xc000d89720>: failed to create HCP cluster cluster-candidate-4-23-kznbgd: failed starting cluster creation "cluster-candidate-4-23-kznbgd" in resourcegroup="rg-candidate-4-23-ttflt6t5z8w7": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-4-23-ttflt6t5z8w7/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-candidate-4-23-kznbgd -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: LocationNotAvailableForResourceType; detail message The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'.; provider Microsoft.RedHatOpenShift 20:07:48 UTC · failed at e2e · test Customer should be able to create an HCP cluster then delete it by deleting the customer resource group · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delete_cx_rg.go:90]: failed to create HCP cluster "cx-rg-hcp-cluster" Unexpected error: <fmt.wrapError \| 0xc000ab2980>: failed to create HCP cluster cx-rg-hcp-cluster: failed starting cluster creation "cx-rg-hcp-cluster" in resourcegroup="cx-rg-cluster-82nxjntf8mtx": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/cx-rg-cluster-82nxjntf8mtx/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cx-rg-hcp-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delete_cx_rg.go:90]: failed to create HCP cluster "cx-rg-hcp-cluster" Unexpected error: <fmt.wrapError \| 0xc000ab2980>: failed to create HCP cluster cx-rg-hcp-cluster: failed starting cluster creation "cx-rg-hcp-cluster" in resourcegroup="cx-rg-cluster-82nxjntf8mtx": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/cx-rg-cluster-82nxjntf8mtx/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cx-rg-hcp-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: LocationNotAvailableForResourceType; detail message The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'.; provider Microsoft.RedHatOpenShift 20:07:48 UTC · failed at e2e · test Customer should be able to test admin credentials before cluster ready, then full admin credential lifecycle · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:101]: failed to begin creating HCP cluster "admin-cred-lifecycle-n8fmrx" Unexpected error: <fmt.wrapError \| 0xc001578f40>: failed starting cluster creation "admin-cred-lifecycle-n8fmrx" in resourcegroup="admin-credential-lifecycle-test-g8tk97spx6sk": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/admin-credential-lifecycle-test-g8tk97spx6sk/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/admin-cred-lifecycle-n8fmrx -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:101]: failed to begin creating HCP cluster "admin-cred-lifecycle-n8fmrx" Unexpected error: <fmt.wrapError \| 0xc001578f40>: failed starting cluster creation "admin-cred-lifecycle-n8fmrx" in resourcegroup="admin-credential-lifecycle-test-g8tk97spx6sk": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/admin-credential-lifecycle-test-g8tk97spx6sk/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/admin-cred-lifecycle-n8fmrx -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: LocationNotAvailableForResourceType; detail message The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'.; provider Microsoft.RedHatOpenShift 20:07:48 UTC · failed at e2e · test Customer should be able to create an HCP cluster with custom autoscaling and update the autoscaling configuration · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:108]: failed to create HCP cluster "autoscaling-hcp-cluster" with custom autoscaling Unexpected error: <fmt.wrapError \| 0xc000fa6520>: failed to create HCP cluster autoscaling-hcp-cluster: failed starting cluster creation "autoscaling-hcp-cluster" in resourcegroup="autoscaling-cluster-xzp28kftfsgl": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/autoscaling-cluster-xzp28kftfsgl/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/autoscaling-hcp-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:108]: failed to create HCP cluster "autoscaling-hcp-cluster" with custom autoscaling Unexpected error: <fmt.wrapError \| 0xc000fa6520>: failed to create HCP cluster autoscaling-hcp-cluster: failed starting cluster creation "autoscaling-hcp-cluster" in resourcegroup="autoscaling-cluster-xzp28kftfsgl": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/autoscaling-cluster-xzp28kftfsgl/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/autoscaling-hcp-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: LocationNotAvailableForResourceType; detail message The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'.; provider Microsoft.RedHatOpenShift 20:07:48 UTC · failed at e2e · test Customer should upgrade a nodepool skipping one minor version (+2) from 4.20.z to 4.22.zLatest · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:441]: failed to create HCP cluster np-skip-minor-5c9stf at version 4.22.15 Unexpected error: <fmt.wrapError \| 0xc000816b20>: failed to create HCP cluster np-skip-minor-5c9stf: failed starting cluster creation "np-skip-minor-5c9stf" in resourcegroup="rg-np-skip-minor-5c9stf-zgbfk5cxflhd": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-skip-minor-5c9stf-zgbfk5cxflhd/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-skip-minor-5c9stf -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:441]: failed to create HCP cluster np-skip-minor-5c9stf at version 4.22.15 Unexpected error: <fmt.wrapError \| 0xc000816b20>: failed to create HCP cluster np-skip-minor-5c9stf: failed starting cluster creation "np-skip-minor-5c9stf" in resourcegroup="rg-np-skip-minor-5c9stf-zgbfk5cxflhd": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-skip-minor-5c9stf-zgbfk5cxflhd/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-skip-minor-5c9stf -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: LocationNotAvailableForResourceType; detail message The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'.; provider Microsoft.RedHatOpenShift 20:07:48 UTC · failed at e2e · test ARO HCP Service should successfully provision cluster when MI role assignments are added after cluster creation · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:140]: failed to start cluster creation Unexpected error: <exported.ResponseError \| 0xc0018ece40>: PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/delayed-rbac-cwk7z6fq2hf6/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/delayed-rbac-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:140]: failed to start cluster creation Unexpected error: <exported.ResponseError \| 0xc0018ece40>: PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/delayed-rbac-cwk7z6fq2hf6/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/delayed-rbac-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: LocationNotAvailableForResourceType; detail message The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'.; provider Microsoft.RedHatOpenShift 20:07:48 UTC · failed at e2e · test Customer should be able to create a cluster and node pools with aggregated advanced features · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:208]: failed to create HCP cluster "agg-cluster" with aggregated settings Unexpected error: <fmt.wrapError \| 0xc000119340>: failed starting cluster creation "agg-cluster" in resourcegroup="feature-aggregation-m4g9t2jskz2r": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/feature-aggregation-m4g9t2jskz2r/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/agg-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:208]: failed to create HCP cluster "agg-cluster" with aggregated settings Unexpected error: <fmt.wrapError \| 0xc000119340>: failed starting cluster creation "agg-cluster" in resourcegroup="feature-aggregation-m4g9t2jskz2r": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/feature-aggregation-m4g9t2jskz2r/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/agg-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: LocationNotAvailableForResourceType -------------------------------------------------------------------------------- { "error": { "code": "LocationNotAvailableForResourceType", "message": "The provided location 'westus3' is not available for resource type 'Microsoft.RedHatOpenShift/HCPOpenShiftClusters'. List of available regions for the resource type is 'switzerlandnorth,canadacentral,australiaeast,westeurope,centralindia,brazilsouth,eastus2,uksouth'." } } -------------------------------------------------------------------------------- ... occurred
13:55:50	1h 28m 31s	branch-ci-Azure-ARO-HCP-main-e2e-stage-e2e-parallel post-goodmerged PR No additional run metadata captured	uksouth	e2e	failed	n/a	55	e2e (55) ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 13:55:50 UTC · failed at e2e · test Customer should be able to successfully upgrade control plane minor version from 4.22 minor to 5.0 minor · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:133]: failed to create HCP cluster "cp-ystream-upgrade-5-0-q9n7xf" at install version 4.22 Unexpected error: <fmt.wrapError \| 0xc0015a82e0>: failed to create HCP cluster cp-ystream-upgrade-5-0-q9n7xf: failed waiting for cluster="cp-ystream-upgrade-5-0-q9n7xf" in resourcegroup="rg-cp-ystream-upgrade-5-0-mbh76rk9sprx" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/e50bc50d-ed3a-44e5-99bf-5d82436a9665 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:133]: failed to create HCP cluster "cp-ystream-upgrade-5-0-q9n7xf" at install version 4.22 Unexpected error: <fmt.wrapError \| 0xc0015a82e0>: failed to create HCP cluster cp-ystream-upgrade-5-0-q9n7xf: failed waiting for cluster="cp-ystream-upgrade-5-0-q9n7xf" in resourcegroup="rg-cp-ystream-upgrade-5-0-mbh76rk9sprx" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/e50bc50d-ed3a-44e5-99bf-5d82436a9665 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 13:55:50 UTC · failed at e2e · test Customer should create a cluster with private ingress using v20260901preview and verify the ingress is internal · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_ingress.go:87]: failed to create HCP cluster "private-ingress" with private ingress Unexpected error: <fmt.wrapError \| 0xc001106580>: failed to create HCP cluster private-ingress: failed waiting for cluster="private-ingress" in resourcegroup="private-ingress-rjr22k9ck9q9" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/e00f99a2-ee29-44a8-b972-faed7dd9107f -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_ingress.go:87]: failed to create HCP cluster "private-ingress" with private ingress Unexpected error: <fmt.wrapError \| 0xc001106580>: failed to create HCP cluster private-ingress: failed waiting for cluster="private-ingress" in resourcegroup="private-ingress-rjr22k9ck9q9" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/e00f99a2-ee29-44a8-b972-faed7dd9107f -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 13:55:50 UTC · failed at e2e · test Customer should be able to delete a nodepool · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_delete.go:81]: failed to create HCP cluster np-delete-hcp-cluster Unexpected error: <fmt.wrapError \| 0xc000c9afe0>: failed to create HCP cluster np-delete-hcp-cluster: failed waiting for cluster="np-delete-hcp-cluster" in resourcegroup="nodepool-delete-gnrv6kblsb4v" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/2ff2bc98-36d7-4657-b844-1f907e6770ba -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_delete.go:81]: failed to create HCP cluster np-delete-hcp-cluster Unexpected error: <fmt.wrapError \| 0xc000c9afe0>: failed to create HCP cluster np-delete-hcp-cluster: failed waiting for cluster="np-delete-hcp-cluster" in resourcegroup="nodepool-delete-gnrv6kblsb4v" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/2ff2bc98-36d7-4657-b844-1f907e6770ba -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 13:55:50 UTC · failed at e2e · test Customer should be able to create an HCP cluster with Image Registry not present · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/image_registry_cluster_create.go:89]: failed to create HCP cluster with image registry disabled Unexpected error: <fmt.wrapError \| 0xc0011ba1c0>: failed to create HCP cluster disabled-image-registry-hcp-cluster: failed waiting for cluster="disabled-image-registry-hcp-cluster" in resourcegroup="disabled-image-registry-gmkdjq6v4l8d" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/87a6922d-74fc-4481-b171-9e921e47725f -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/image_registry_cluster_create.go:89]: failed to create HCP cluster with image registry disabled Unexpected error: <fmt.wrapError \| 0xc0011ba1c0>: failed to create HCP cluster disabled-image-registry-hcp-cluster: failed waiting for cluster="disabled-image-registry-hcp-cluster" in resourcegroup="disabled-image-registry-gmkdjq6v4l8d" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/87a6922d-74fc-4481-b171-9e921e47725f -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 13:55:50 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.23 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:123]: HCP cluster rg-candidate-4-23-p49qttrgxjfs/cluster-candidate-4-23-h8xv9w should provision Unexpected error: <fmt.wrapError \| 0xc000a0e9e0>: failed to create HCP cluster cluster-candidate-4-23-h8xv9w: failed waiting for cluster="cluster-candidate-4-23-h8xv9w" in resourcegroup="rg-candidate-4-23-p49qttrgxjfs" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/775da561-b2d7-4a7a-bbc2-e2713e469817 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:123]: HCP cluster rg-candidate-4-23-p49qttrgxjfs/cluster-candidate-4-23-h8xv9w should provision Unexpected error: <fmt.wrapError \| 0xc000a0e9e0>: failed to create HCP cluster cluster-candidate-4-23-h8xv9w: failed waiting for cluster="cluster-candidate-4-23-h8xv9w" in resourcegroup="rg-candidate-4-23-p49qttrgxjfs" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/775da561-b2d7-4a7a-bbc2-e2713e469817 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 13:55:50 UTC · failed at e2e · test Customer should be able to issue the default ingress serving certificate from Let's Encrypt using cert-manager with ACME DNS-01 challenges · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_ingress_cert_letsencrypt.go:124]: failed to create HCP cluster Unexpected error: <fmt.wrapError \| 0xc000aa81e0>: failed to create HCP cluster ingress-cert-le: failed waiting for cluster="ingress-cert-le" in resourcegroup="ingress-cert-le-wfnsr7qsscm2" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/26a20c52-fa37-4aba-b728-744aade6e61d -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_ingress_cert_letsencrypt.go:124]: failed to create HCP cluster Unexpected error: <fmt.wrapError \| 0xc000aa81e0>: failed to create HCP cluster ingress-cert-le: failed waiting for cluster="ingress-cert-le" in resourcegroup="ingress-cert-le-wfnsr7qsscm2" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/26a20c52-fa37-4aba-b728-744aade6e61d -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 13:55:50 UTC · failed at e2e · test FIPS Mode Support with v2026-09-01 API should create an HCP cluster with FIPS mode enabled via cryptoRestrictions property · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_fips_mode.go:87]: failed to create HCP cluster "fips-enabled-cluster" with cryptoRestrictions set to FIPS Unexpected error: <fmt.wrapError \| 0xc000f90380>: failed waiting for cluster="fips-enabled-cluster" in resourcegroup="fips-enabled-fz5ccggcv7ng" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/da969d58-8ed2-4cb8-98b5-5e49f25760a7 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_fips_mode.go:87]: failed to create HCP cluster "fips-enabled-cluster" with cryptoRestrictions set to FIPS Unexpected error: <fmt.wrapError \| 0xc000f90380>: failed waiting for cluster="fips-enabled-cluster" in resourcegroup="fips-enabled-fz5ccggcv7ng" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/da969d58-8ed2-4cb8-98b5-5e49f25760a7 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 13:55:50 UTC · failed at e2e · test Customer should be able to create a cluster with default autoscaling and a nodepool with autoscaling enabled up to replica limits · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:102]: failed to create HCP cluster np-autoscale-cluster Unexpected error: <fmt.wrapError \| 0xc000af4360>: failed to create HCP cluster np-autoscale-cluster: failed waiting for cluster="np-autoscale-cluster" in resourcegroup="np-autoscaling-fgdbfv79mc7d" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/5959ed79-fe6d-4e6e-9ad6-d8747b628ab2 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:102]: failed to create HCP cluster np-autoscale-cluster Unexpected error: <fmt.wrapError \| 0xc000af4360>: failed to create HCP cluster np-autoscale-cluster: failed waiting for cluster="np-autoscale-cluster" in resourcegroup="np-autoscaling-fgdbfv79mc7d" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/5959ed79-fe6d-4e6e-9ad6-d8747b628ab2 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 13:55:50 UTC · failed at e2e · test Nodepool Ephemeral OS Disk should create a nodepool with ephemeral OS disk when autoRepair is enabled · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_ephemeral_osdisk.go:84]: failed to create HCP cluster ephemeral-disk Unexpected error: <fmt.wrapError \| 0xc00178a640>: failed to create HCP cluster ephemeral-disk: failed waiting for cluster="ephemeral-disk" in resourcegroup="ephemeral-osdisk-8n2xcvsmnqjs" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b55f35c3-ff27-4b24-b063-f8561b8ce85d -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_ephemeral_osdisk.go:84]: failed to create HCP cluster ephemeral-disk Unexpected error: <fmt.wrapError \| 0xc00178a640>: failed to create HCP cluster ephemeral-disk: failed waiting for cluster="ephemeral-disk" in resourcegroup="ephemeral-osdisk-8n2xcvsmnqjs" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b55f35c3-ff27-4b24-b063-f8561b8ce85d -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 13:55:50 UTC · failed at e2e · test Customer should be able to create an HCP cluster with back-level version 4.20 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:152]: failed to create HCP cluster version 4.20 Unexpected error: <fmt.wrapError \| 0xc000936680>: failed waiting for cluster="cluster-ver-4-20" in resourcegroup="rg-cluster-back-version-mv29z56qcdwh" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/94b0888e-53d5-420f-92bb-9785a421357a -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:152]: failed to create HCP cluster version 4.20 Unexpected error: <fmt.wrapError \| 0xc000936680>: failed waiting for cluster="cluster-ver-4-20" in resourcegroup="rg-cluster-back-version-mv29z56qcdwh" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/94b0888e-53d5-420f-92bb-9785a421357a -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 13:55:50 UTC · failed at e2e · test Customer should be able to create an HCP cluster and manage pull secrets · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_pullsecret.go:113]: failed to create HCP cluster for pull secret test Unexpected error: <fmt.wrapError \| 0xc00097c6e0>: failed to create HCP cluster pullsecret-hcp-cluster: failed waiting for cluster="pullsecret-hcp-cluster" in resourcegroup="pullsecret-test-ntxx89l7lfdg" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4e64c071-46f0-4057-8137-99940245e456 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_pullsecret.go:113]: failed to create HCP cluster for pull secret test Unexpected error: <fmt.wrapError \| 0xc00097c6e0>: failed to create HCP cluster pullsecret-hcp-cluster: failed waiting for cluster="pullsecret-hcp-cluster" in resourcegroup="pullsecret-test-ntxx89l7lfdg" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4e64c071-46f0-4057-8137-99940245e456 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 13:55:50 UTC · failed at e2e · test Customer should be able to list available HCP OpenShift versions and validate response content · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_versions.go:43]: failed to list HCP OpenShift versions Unexpected error: <exported.ResponseError \| 0xc00166f260>: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOpenShiftVersions -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_versions.go:43]: failed to list HCP OpenShift versions Unexpected error: <exported.ResponseError \| 0xc00166f260>: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOpenShiftVersions -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 13:55:50 UTC · failed at e2e · test Customer should create a cluster with private KAS and verify API server is only reachable from VNet while ingress remains public · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:91]: failed to create HCP cluster "private-kas" with private KAS Unexpected error: <fmt.wrapError \| 0xc000b5e720>: failed to create HCP cluster private-kas: failed waiting for cluster="private-kas" in resourcegroup="private-kas-vbqclhz48q7d" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/dac67df7-9a2f-4479-ad7e-7f7a442c996c -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:91]: failed to create HCP cluster "private-kas" with private KAS Unexpected error: <fmt.wrapError \| 0xc000b5e720>: failed to create HCP cluster private-kas: failed waiting for cluster="private-kas" in resourcegroup="private-kas-vbqclhz48q7d" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/dac67df7-9a2f-4479-ad7e-7f7a442c996c -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 13:55:50 UTC · failed at e2e · test Customer should downgrade a nodepool to a lower minor version from 4.22.zLatest to 4.20.zLatest · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:720]: failed to create HCP cluster Unexpected error: <fmt.wrapError \| 0xc00173a180>: failed to create HCP cluster np-dg-minor-mw6fw8: failed waiting for cluster="np-dg-minor-mw6fw8" in resourcegroup="rg-np-dg-minor-mw6fw8-fv7854tv6zx6" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/6e066b83-100f-40e1-8a40-6aa10f3d747e -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:720]: failed to create HCP cluster Unexpected error: <fmt.wrapError \| 0xc00173a180>: failed to create HCP cluster np-dg-minor-mw6fw8: failed waiting for cluster="np-dg-minor-mw6fw8" in resourcegroup="rg-np-dg-minor-mw6fw8-fv7854tv6zx6" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/6e066b83-100f-40e1-8a40-6aa10f3d747e -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 13:55:50 UTC · failed at e2e · test Customer should be able to create an HCP cluster and custom node pool osDisk size · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_nodepool_osdisk.go:80]: failed to create HCP cluster "hcp-cluster-np-128" Unexpected error: <fmt.wrapError \| 0xc000900660>: failed to create HCP cluster hcp-cluster-np-128: failed waiting for cluster="hcp-cluster-np-128" in resourcegroup="clusternp128-whdhftj7qn5d" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/6471fa0f-d154-448c-8e08-84fa720e0e52 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_nodepool_osdisk.go:80]: failed to create HCP cluster "hcp-cluster-np-128" Unexpected error: <fmt.wrapError \| 0xc000900660>: failed to create HCP cluster hcp-cluster-np-128: failed waiting for cluster="hcp-cluster-np-128" in resourcegroup="clusternp128-whdhftj7qn5d" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/6471fa0f-d154-448c-8e08-84fa720e0e52 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 13:55:50 UTC · failed at e2e · test ARO HCP Service should successfully provision cluster when MI role assignments are added after cluster creation · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:238]: failed to get admin REST config for HCP cluster Unexpected error: <fmt.wrapError \| 0xc000b4c8c0>: failed waiting for hcpCluster="delayed-rbac-cluster" in resourcegroup="delayed-rbac-kbtzww7244jn" to finish getting creds: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/53365482-c4a1-4a54-9bc2-e53d7610a2ef -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:238]: failed to get admin REST config for HCP cluster Unexpected error: <fmt.wrapError \| 0xc000b4c8c0>: failed waiting for hcpCluster="delayed-rbac-cluster" in resourcegroup="delayed-rbac-kbtzww7244jn" to finish getting creds: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/53365482-c4a1-4a54-9bc2-e53d7610a2ef -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 13:55:50 UTC · failed at e2e · test Customer should downgrade a nodepool version z-stream downgrade from 4.21.zLatest to 4.21.zPrevious · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:580]: failed to create HCP cluster Unexpected error: <fmt.wrapError \| 0xc000898dc0>: failed to create HCP cluster np-downgrade-gpn9mw: failed waiting for cluster="np-downgrade-gpn9mw" in resourcegroup="rg-np-downgrade-gpn9mw-r62s5hw6qpcx" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/6b48c831-1af9-4e6e-9083-5e0aafbca54b -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:580]: failed to create HCP cluster Unexpected error: <fmt.wrapError \| 0xc000898dc0>: failed to create HCP cluster np-downgrade-gpn9mw: failed waiting for cluster="np-downgrade-gpn9mw" in resourcegroup="rg-np-downgrade-gpn9mw-r62s5hw6qpcx" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/6b48c831-1af9-4e6e-9083-5e0aafbca54b -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 13:55:50 UTC · failed at e2e · test Customer should be able to forward control plane logs to a storage account and Event Hub via shoebox diagnostic settings · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/shoebox_logs.go:272]: failed to create HCP cluster shoebox-hcp-cluster Unexpected error: <fmt.wrapError \| 0xc000b07980>: failed to create HCP cluster shoebox-hcp-cluster: failed waiting for cluster="shoebox-hcp-cluster" in resourcegroup="shoebox-cluster-ct4q5w8z58wh" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/44875744-19fa-48cd-9879-d39c75585e03 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/shoebox_logs.go:272]: failed to create HCP cluster shoebox-hcp-cluster Unexpected error: <fmt.wrapError \| 0xc000b07980>: failed to create HCP cluster shoebox-hcp-cluster: failed waiting for cluster="shoebox-hcp-cluster" in resourcegroup="shoebox-cluster-ct4q5w8z58wh" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/44875744-19fa-48cd-9879-d39c75585e03 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 13:55:50 UTC · failed at e2e · test Customer should be able to create a cluster and node pools with aggregated advanced features · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:208]: failed to create HCP cluster "agg-cluster" with aggregated settings Unexpected error: <fmt.wrapError \| 0xc000d0e420>: failed waiting for cluster="agg-cluster" in resourcegroup="feature-aggregation-9h28b7rm7p7q" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/fe262be4-afb0-46ac-ae85-2dea20ec65da -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:208]: failed to create HCP cluster "agg-cluster" with aggregated settings Unexpected error: <fmt.wrapError \| 0xc000d0e420>: failed waiting for cluster="agg-cluster" in resourcegroup="feature-aggregation-9h28b7rm7p7q" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/fe262be4-afb0-46ac-ae85-2dea20ec65da -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 13:55:50 UTC · failed at e2e · test Create HCPOpenShiftCluster with Private KeyVault should create a cluster with private keyvault using v20251223preview API · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:99]: failed to create HCP cluster "private-kv-cluster" with private keyvault Unexpected error: <fmt.wrapError \| 0xc000e34960>: failed waiting for cluster="private-kv-cluster" in resourcegroup="private-keyvault-9l5kkq8td965" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/7e0f9e14-a270-4413-931e-d3e3b11a1154 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:99]: failed to create HCP cluster "private-kv-cluster" with private keyvault Unexpected error: <fmt.wrapError \| 0xc000e34960>: failed waiting for cluster="private-kv-cluster" in resourcegroup="private-keyvault-9l5kkq8td965" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/7e0f9e14-a270-4413-931e-d3e3b11a1154 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 13:55:50 UTC · failed at e2e · test Customer should create a cluster using v20260901preview API and verify cluster health · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_v20260901preview.go:86]: failed to create HCP cluster "v20260901" via v20260901preview Unexpected error: <fmt.wrapError \| 0xc001800360>: failed to create HCP cluster v20260901: failed waiting for cluster="v20260901" in resourcegroup="v20260901preview-fbch8rjgjcdt" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/30333c79-2349-4fe1-aefd-512a2d96ca07 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_v20260901preview.go:86]: failed to create HCP cluster "v20260901" via v20260901preview Unexpected error: <fmt.wrapError \| 0xc001800360>: failed to create HCP cluster v20260901: failed waiting for cluster="v20260901" in resourcegroup="v20260901preview-fbch8rjgjcdt" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/30333c79-2349-4fe1-aefd-512a2d96ca07 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 13:55:50 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 4.22 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:123]: failed to create HCP cluster "cluster-zstream-4-22-ttqwsf" with version 4.22.14 on candidate channel Unexpected error: <fmt.wrapError \| 0xc001084400>: failed to create HCP cluster cluster-zstream-4-22-ttqwsf: failed waiting for cluster="cluster-zstream-4-22-ttqwsf" in resourcegroup="rg-zstream-upgrade-4-22-hswnfth76bl9" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/9aba5af8-d524-474f-8952-a84e13286973 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:123]: failed to create HCP cluster "cluster-zstream-4-22-ttqwsf" with version 4.22.14 on candidate channel Unexpected error: <fmt.wrapError \| 0xc001084400>: failed to create HCP cluster cluster-zstream-4-22-ttqwsf: failed waiting for cluster="cluster-zstream-4-22-ttqwsf" in resourcegroup="rg-zstream-upgrade-4-22-hswnfth76bl9" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/9aba5af8-d524-474f-8952-a84e13286973 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 13:55:50 UTC · failed at e2e · test Customer should be able to lifecycle and confirm external auth on a cluster · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_list_and_verify.go:87]: failed to create HCP cluster for external auth list test Unexpected error: <fmt.wrapError \| 0xc001036880>: failed to create HCP cluster ea-list46zfdd: failed waiting for cluster="ea-list46zfdd" in resourcegroup="ea-list-79xdwcv5tf4m" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/74932894-78b3-4bea-aeaa-d0849c360355 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_list_and_verify.go:87]: failed to create HCP cluster for external auth list test Unexpected error: <fmt.wrapError \| 0xc001036880>: failed to create HCP cluster ea-list46zfdd: failed waiting for cluster="ea-list46zfdd" in resourcegroup="ea-list-79xdwcv5tf4m" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/74932894-78b3-4bea-aeaa-d0849c360355 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 13:55:50 UTC · failed at e2e · test Customer should be able to create a no-CNI private cluster with a private key vault, a nodepool and install cilium CNI successfully · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:107]: failed to create HCP cluster "cilium-cluster" with no CNI and private etcd Unexpected error: <fmt.wrapError \| 0xc000d24160>: failed waiting for cluster="cilium-cluster" in resourcegroup="complex-cilium-kv-xszsmtdmvz98" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/6dc4fe03-f2a4-42a5-b56a-4bd6880a15b4 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:107]: failed to create HCP cluster "cilium-cluster" with no CNI and private etcd Unexpected error: <fmt.wrapError \| 0xc000d24160>: failed waiting for cluster="cilium-cluster" in resourcegroup="complex-cilium-kv-xszsmtdmvz98" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/6dc4fe03-f2a4-42a5-b56a-4bd6880a15b4 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 13:55:50 UTC · failed at e2e · test Customer should upgrade and update a nodepool from 4.20.z to 4.20.zLatest · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:146]: failed to create HCP cluster np-version-upgrade-cluster-v6qfj7 with version 4.20.39 Unexpected error: <fmt.wrapError \| 0xc0015da7c0>: failed to create HCP cluster np-version-upgrade-cluster-v6qfj7: failed waiting for cluster="np-version-upgrade-cluster-v6qfj7" in resourcegroup="rg-np-version-upgrade-v6qfj7-skg6jrnxc5p6" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/c7f3b390-27b8-429d-b49e-f6681f383963 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:146]: failed to create HCP cluster np-version-upgrade-cluster-v6qfj7 with version 4.20.39 Unexpected error: <fmt.wrapError \| 0xc0015da7c0>: failed to create HCP cluster np-version-upgrade-cluster-v6qfj7: failed waiting for cluster="np-version-upgrade-cluster-v6qfj7" in resourcegroup="rg-np-version-upgrade-v6qfj7-skg6jrnxc5p6" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/c7f3b390-27b8-429d-b49e-f6681f383963 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 13:55:50 UTC · failed at e2e · test Customer should not be able to reuse subnets and NSGs between clusters · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_nsg_subnet_reuse.go:107]: failed to create seed HCP cluster basic-cluster Unexpected error: <fmt.wrapError \| 0xc000828440>: failed to create HCP cluster basic-cluster: failed waiting for cluster="basic-cluster" in resourcegroup="rg-cluster-nsg-subnet-reuse-d87x9gm5jhzj" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/3877314f-c3de-45b0-bd82-35e621bcf79f -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_nsg_subnet_reuse.go:107]: failed to create seed HCP cluster basic-cluster Unexpected error: <fmt.wrapError \| 0xc000828440>: failed to create HCP cluster basic-cluster: failed waiting for cluster="basic-cluster" in resourcegroup="rg-cluster-nsg-subnet-reuse-d87x9gm5jhzj" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/3877314f-c3de-45b0-bd82-35e621bcf79f -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 13:55:50 UTC · failed at e2e · test Update HCPOpenShiftCluster Negative creates a cluster and fails to update its name with a PATCH request · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:84]: failed to create HCP cluster for patch-name test Unexpected error: <fmt.wrapError \| 0xc000ef01e0>: failed to create HCP cluster patch-name-cluster: failed waiting for cluster="patch-name-cluster" in resourcegroup="patch-name-pprdvsqzxr2q" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0ad9bd8b-8087-4a46-86ff-311840c0da2a -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:84]: failed to create HCP cluster for patch-name test Unexpected error: <fmt.wrapError \| 0xc000ef01e0>: failed to create HCP cluster patch-name-cluster: failed waiting for cluster="patch-name-cluster" in resourcegroup="patch-name-pprdvsqzxr2q" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0ad9bd8b-8087-4a46-86ff-311840c0da2a -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 13:55:50 UTC · failed at e2e · test Customer should be able to create an HCP cluster then delete it by deleting the customer resource group · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delete_cx_rg.go:90]: failed to create HCP cluster "cx-rg-hcp-cluster" Unexpected error: <fmt.wrapError \| 0xc000a36a60>: failed to create HCP cluster cx-rg-hcp-cluster: failed waiting for cluster="cx-rg-hcp-cluster" in resourcegroup="cx-rg-cluster-2ztbw8htnxgv" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/38bfbf27-f7bb-4d37-9b0b-8c8ea498da96 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delete_cx_rg.go:90]: failed to create HCP cluster "cx-rg-hcp-cluster" Unexpected error: <fmt.wrapError \| 0xc000a36a60>: failed to create HCP cluster cx-rg-hcp-cluster: failed waiting for cluster="cx-rg-hcp-cluster" in resourcegroup="cx-rg-cluster-2ztbw8htnxgv" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/38bfbf27-f7bb-4d37-9b0b-8c8ea498da96 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 13:55:50 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 4.21 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:123]: failed to create HCP cluster "cluster-zstream-4-21-jbbr7p" with version 4.21.33 on candidate channel Unexpected error: <fmt.wrapError \| 0xc0018025a0>: failed to create HCP cluster cluster-zstream-4-21-jbbr7p: failed waiting for cluster="cluster-zstream-4-21-jbbr7p" in resourcegroup="rg-zstream-upgrade-4-21-rxn6xbk849kh" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/5866f782-63d0-4727-b1f9-fbb6238a4cf4 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:123]: failed to create HCP cluster "cluster-zstream-4-21-jbbr7p" with version 4.21.33 on candidate channel Unexpected error: <fmt.wrapError \| 0xc0018025a0>: failed to create HCP cluster cluster-zstream-4-21-jbbr7p: failed waiting for cluster="cluster-zstream-4-21-jbbr7p" in resourcegroup="rg-zstream-upgrade-4-21-rxn6xbk849kh" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/5866f782-63d0-4727-b1f9-fbb6238a4cf4 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 13:55:50 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 5.0 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:123]: HCP cluster rg-candidate-5-0-cz9xt5tdn72x/cluster-candidate-5-0-5qdbnm should provision Unexpected error: <fmt.wrapError \| 0xc00060b4e0>: failed to create HCP cluster cluster-candidate-5-0-5qdbnm: failed waiting for cluster="cluster-candidate-5-0-5qdbnm" in resourcegroup="rg-candidate-5-0-cz9xt5tdn72x" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/44f6089e-d9b1-4a58-b233-8bf4446e75a6 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:123]: HCP cluster rg-candidate-5-0-cz9xt5tdn72x/cluster-candidate-5-0-5qdbnm should provision Unexpected error: <fmt.wrapError \| 0xc00060b4e0>: failed to create HCP cluster cluster-candidate-5-0-5qdbnm: failed waiting for cluster="cluster-candidate-5-0-5qdbnm" in resourcegroup="rg-candidate-5-0-cz9xt5tdn72x" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/44f6089e-d9b1-4a58-b233-8bf4446e75a6 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 13:55:50 UTC · failed at e2e · test Update HCPOpenShiftCluster Positive creates a cluster and updates tags with a PATCH request · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:161]: failed to create HCP cluster for patch-tags test Unexpected error: <fmt.wrapError \| 0xc000dd05e0>: failed to create HCP cluster patch-tags-cluster: failed waiting for cluster="patch-tags-cluster" in resourcegroup="patch-tags-l8gvsztb94w8" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/31016ec8-3d37-4831-8008-f5f433d4a7e6 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:161]: failed to create HCP cluster for patch-tags test Unexpected error: <fmt.wrapError \| 0xc000dd05e0>: failed to create HCP cluster patch-tags-cluster: failed waiting for cluster="patch-tags-cluster" in resourcegroup="patch-tags-l8gvsztb94w8" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/31016ec8-3d37-4831-8008-f5f433d4a7e6 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 13:55:50 UTC · failed at e2e · test Customer should upgrade a nodepool skipping one minor version (+2) from 4.20.z to 4.22.zLatest · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:441]: failed to create HCP cluster np-skip-minor-84tr9g at version 4.22.15 Unexpected error: <fmt.wrapError \| 0xc00140a300>: failed to create HCP cluster np-skip-minor-84tr9g: failed waiting for cluster="np-skip-minor-84tr9g" in resourcegroup="rg-np-skip-minor-84tr9g-bf984wxm78rb" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/78280dad-f48c-47ab-96e5-a964252849a6 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:441]: failed to create HCP cluster np-skip-minor-84tr9g at version 4.22.15 Unexpected error: <fmt.wrapError \| 0xc00140a300>: failed to create HCP cluster np-skip-minor-84tr9g: failed waiting for cluster="np-skip-minor-84tr9g" in resourcegroup="rg-np-skip-minor-84tr9g-bf984wxm78rb" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/78280dad-f48c-47ab-96e5-a964252849a6 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 13:55:50 UTC · failed at e2e · test Customer should be able to create several HCP clusters in their customer resource group, but not in the same managed resource group · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:131]: failed to wait for first cluster "basic-hcp-cluster" to complete creation (timeout '30.000000' minutes) Unexpected error: <exported.ResponseError \| 0xc0017b88d0>: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/8f140fe3-a6eb-4a04-8ee6-f53c88e39f17 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:131]: failed to wait for first cluster "basic-hcp-cluster" to complete creation (timeout '30.000000' minutes) Unexpected error: <exported.ResponseError \| 0xc0017b88d0>: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/8f140fe3-a6eb-4a04-8ee6-f53c88e39f17 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 13:55:50 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 5.0 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:123]: failed to create HCP cluster "cluster-zstream-5-0-8dk887" with version 5.0.0-rc.2 on candidate channel Unexpected error: <fmt.wrapError \| 0xc000adc8c0>: failed to create HCP cluster cluster-zstream-5-0-8dk887: failed waiting for cluster="cluster-zstream-5-0-8dk887" in resourcegroup="rg-zstream-upgrade-5-0-l9598lhfkdvh" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/09132225-4087-467b-a92a-dc104be7a5ec -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:123]: failed to create HCP cluster "cluster-zstream-5-0-8dk887" with version 5.0.0-rc.2 on candidate channel Unexpected error: <fmt.wrapError \| 0xc000adc8c0>: failed to create HCP cluster cluster-zstream-5-0-8dk887: failed waiting for cluster="cluster-zstream-5-0-8dk887" in resourcegroup="rg-zstream-upgrade-5-0-l9598lhfkdvh" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/09132225-4087-467b-a92a-dc104be7a5ec -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 13:55:50 UTC · failed at e2e · test Customer should create an HCP cluster and validate TLS certificates · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:96]: failed to create HCP cluster for TLS endpoint test Unexpected error: <fmt.wrapError \| 0xc000172620>: failed to create HCP cluster tls-endpoint-hcp-cluster: failed waiting for cluster="tls-endpoint-hcp-cluster" in resourcegroup="tls-endpoint-cluster-lhkhbbx7mz6b" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/470867e6-8d72-4c8c-bcba-f7fc908984b2 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:96]: failed to create HCP cluster for TLS endpoint test Unexpected error: <fmt.wrapError \| 0xc000172620>: failed to create HCP cluster tls-endpoint-hcp-cluster: failed waiting for cluster="tls-endpoint-hcp-cluster" in resourcegroup="tls-endpoint-cluster-lhkhbbx7mz6b" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/470867e6-8d72-4c8c-bcba-f7fc908984b2 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 13:55:50 UTC · failed at e2e · test Customer should upgrade and update a nodepool from 4.20.z to 4.21.zLatest · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:146]: failed to create HCP cluster np-version-upgrade-cluster-9dncbv with version 4.21.34 Unexpected error: <fmt.wrapError \| 0xc00063e6e0>: failed to create HCP cluster np-version-upgrade-cluster-9dncbv: failed waiting for cluster="np-version-upgrade-cluster-9dncbv" in resourcegroup="rg-np-version-upgrade-9dncbv-xp64h6q8pgbx" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b67dc797-8bcb-4368-a47e-f51ae12621d8 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:146]: failed to create HCP cluster np-version-upgrade-cluster-9dncbv with version 4.21.34 Unexpected error: <fmt.wrapError \| 0xc00063e6e0>: failed to create HCP cluster np-version-upgrade-cluster-9dncbv: failed waiting for cluster="np-version-upgrade-cluster-9dncbv" in resourcegroup="rg-np-version-upgrade-9dncbv-xp64h6q8pgbx" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b67dc797-8bcb-4368-a47e-f51ae12621d8 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 13:55:50 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.20 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:123]: HCP cluster rg-candidate-4-20-d6llpw2w8455/cluster-candidate-4-20-vvt6h2 should provision Unexpected error: <fmt.wrapError \| 0xc000f9a580>: failed to create HCP cluster cluster-candidate-4-20-vvt6h2: failed waiting for cluster="cluster-candidate-4-20-vvt6h2" in resourcegroup="rg-candidate-4-20-d6llpw2w8455" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/7da8b990-fede-4a5f-94f3-9b16f5a9b2c7 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:123]: HCP cluster rg-candidate-4-20-d6llpw2w8455/cluster-candidate-4-20-vvt6h2 should provision Unexpected error: <fmt.wrapError \| 0xc000f9a580>: failed to create HCP cluster cluster-candidate-4-20-vvt6h2: failed waiting for cluster="cluster-candidate-4-20-vvt6h2" in resourcegroup="rg-candidate-4-20-d6llpw2w8455" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/7da8b990-fede-4a5f-94f3-9b16f5a9b2c7 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 13:55:50 UTC · failed at e2e · test Customer should be able to successfully upgrade control plane minor version from 4.22 minor to 4.23 minor · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:133]: failed to create HCP cluster "cp-ystream-upgrade-4-23-pf9n2c" at install version 4.22 Unexpected error: <fmt.wrapError \| 0xc0016f22e0>: failed to create HCP cluster cp-ystream-upgrade-4-23-pf9n2c: failed waiting for cluster="cp-ystream-upgrade-4-23-pf9n2c" in resourcegroup="rg-cp-ystream-upgrade-4-23-9jksqp8f2hr5" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/e1d5ba86-b04f-4228-8b57-27eb837e9c7f -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:133]: failed to create HCP cluster "cp-ystream-upgrade-4-23-pf9n2c" at install version 4.22 Unexpected error: <fmt.wrapError \| 0xc0016f22e0>: failed to create HCP cluster cp-ystream-upgrade-4-23-pf9n2c: failed waiting for cluster="cp-ystream-upgrade-4-23-pf9n2c" in resourcegroup="rg-cp-ystream-upgrade-4-23-9jksqp8f2hr5" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/e1d5ba86-b04f-4228-8b57-27eb837e9c7f -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 13:55:50 UTC · failed at e2e · test Customer should be able to test admin credentials before cluster ready, then full admin credential lifecycle · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:190]: failed to request admin credential 1 Unexpected error: <fmt.wrapError \| 0xc000fb79e0>: failed waiting for hcpCluster="admin-cred-lifecycle-rz6kqx" in resourcegroup="admin-credential-lifecycle-test-zz2pjh67gwwb" to finish getting creds: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/2ba375f6-9fe0-49c8-b17c-7f0b96934061 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:190]: failed to request admin credential 1 Unexpected error: <fmt.wrapError \| 0xc000fb79e0>: failed waiting for hcpCluster="admin-cred-lifecycle-rz6kqx" in resourcegroup="admin-credential-lifecycle-test-zz2pjh67gwwb" to finish getting creds: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/2ba375f6-9fe0-49c8-b17c-7f0b96934061 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 13:55:50 UTC · failed at e2e · test Customer should respect cluster-wide node limits with nodepool autoscaling · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:291]: failed to create HCP cluster node-limit-cluster with MaxNodesTotal limit Unexpected error: <fmt.wrapError \| 0xc0006068a0>: failed to create HCP cluster node-limit-cluster: failed waiting for cluster="node-limit-cluster" in resourcegroup="node-limit-test-d684r5cmn2cw" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/56b84e34-7609-4b5d-86d2-51c899622a53 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:291]: failed to create HCP cluster node-limit-cluster with MaxNodesTotal limit Unexpected error: <fmt.wrapError \| 0xc0006068a0>: failed to create HCP cluster node-limit-cluster: failed waiting for cluster="node-limit-cluster" in resourcegroup="node-limit-test-d684r5cmn2cw" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/56b84e34-7609-4b5d-86d2-51c899622a53 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 13:55:50 UTC · failed at e2e · test Customer should be able to successfully upgrade control plane minor version from 4.20 minor to 4.21 minor · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:133]: failed to create HCP cluster "cp-ystream-upgrade-4-21-kf6fv7" at install version 4.20 Unexpected error: <fmt.wrapError \| 0xc0009a86a0>: failed to create HCP cluster cp-ystream-upgrade-4-21-kf6fv7: failed waiting for cluster="cp-ystream-upgrade-4-21-kf6fv7" in resourcegroup="rg-cp-ystream-upgrade-4-21-nlzv5dsxfrrc" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4e284640-004d-4939-97cc-47ce133ee77d -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:133]: failed to create HCP cluster "cp-ystream-upgrade-4-21-kf6fv7" at install version 4.20 Unexpected error: <fmt.wrapError \| 0xc0009a86a0>: failed to create HCP cluster cp-ystream-upgrade-4-21-kf6fv7: failed waiting for cluster="cp-ystream-upgrade-4-21-kf6fv7" in resourcegroup="rg-cp-ystream-upgrade-4-21-nlzv5dsxfrrc" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4e284640-004d-4939-97cc-47ce133ee77d -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 13:55:50 UTC · failed at e2e · test Customer should be able to create node pools with ARM64-based VMs · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/arm64_nodepool.go:88]: failed to create HCP cluster "arm64-vm-hcp-cluster" Unexpected error: <fmt.wrapError \| 0xc001110320>: failed to create HCP cluster arm64-vm-hcp-cluster: failed waiting for cluster="arm64-vm-hcp-cluster" in resourcegroup="arm64-vm-cluster-tz5pdqdgjmrw" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/5ba22792-42a3-4249-8a6d-1ccaf5e1029c -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/arm64_nodepool.go:88]: failed to create HCP cluster "arm64-vm-hcp-cluster" Unexpected error: <fmt.wrapError \| 0xc001110320>: failed to create HCP cluster arm64-vm-hcp-cluster: failed waiting for cluster="arm64-vm-hcp-cluster" in resourcegroup="arm64-vm-cluster-tz5pdqdgjmrw" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/5ba22792-42a3-4249-8a6d-1ccaf5e1029c -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 13:55:50 UTC · failed at e2e · test Customer should not be able to perform various invalid operations on cluster resources · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/simple_negative_cases.go:110]: failed to create HCP cluster negative-tests-cluster Unexpected error: <fmt.wrapError \| 0xc001170440>: failed to create HCP cluster negative-tests-cluster: failed waiting for cluster="negative-tests-cluster" in resourcegroup="rg-negative-tests-snl76l5kjj4q" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/2debef79-d8a4-4dbb-b867-f8cba67ed38d -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/simple_negative_cases.go:110]: failed to create HCP cluster negative-tests-cluster Unexpected error: <fmt.wrapError \| 0xc001170440>: failed to create HCP cluster negative-tests-cluster: failed waiting for cluster="negative-tests-cluster" in resourcegroup="rg-negative-tests-snl76l5kjj4q" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/2debef79-d8a4-4dbb-b867-f8cba67ed38d -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 13:55:50 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 4.20 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:123]: failed to create HCP cluster "cluster-zstream-4-20-2sbqkx" with version 4.20.38 on candidate channel Unexpected error: <fmt.wrapError \| 0xc000eeea80>: failed to create HCP cluster cluster-zstream-4-20-2sbqkx: failed waiting for cluster="cluster-zstream-4-20-2sbqkx" in resourcegroup="rg-zstream-upgrade-4-20-tk6s9fw45dg4" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/762eb000-3830-4b9d-92b9-24d469611d95 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:123]: failed to create HCP cluster "cluster-zstream-4-20-2sbqkx" with version 4.20.38 on candidate channel Unexpected error: <fmt.wrapError \| 0xc000eeea80>: failed to create HCP cluster cluster-zstream-4-20-2sbqkx: failed waiting for cluster="cluster-zstream-4-20-2sbqkx" in resourcegroup="rg-zstream-upgrade-4-20-tk6s9fw45dg4" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/762eb000-3830-4b9d-92b9-24d469611d95 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 13:55:50 UTC · failed at e2e · test Customer should be able to cordon, drain, and uncordon a node in an HCP cluster · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_cordon.go:90]: failed to create HCP cluster cordon-hcp-cluster Unexpected error: <fmt.wrapError \| 0xc000788340>: failed to create HCP cluster cordon-hcp-cluster: failed waiting for cluster="cordon-hcp-cluster" in resourcegroup="rg-cordon-test-khvsfbtbxq2c" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/be2c01e2-4220-4646-bc9d-468d7099961c -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_cordon.go:90]: failed to create HCP cluster cordon-hcp-cluster Unexpected error: <fmt.wrapError \| 0xc000788340>: failed to create HCP cluster cordon-hcp-cluster: failed waiting for cluster="cordon-hcp-cluster" in resourcegroup="rg-cordon-test-khvsfbtbxq2c" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/be2c01e2-4220-4646-bc9d-468d7099961c -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 13:55:50 UTC · failed at e2e · test Customer should be able to create a HCP cluster and use cilium CNI plugin · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_cni_cilium.go:88]: failed to create HCP cluster "cilium-cl" without CNI Unexpected error: <fmt.wrapError \| 0xc000a44400>: failed to create HCP cluster cilium-cl: failed waiting for cluster="cilium-cl" in resourcegroup="cni-cilium-pffhm2cq6f4q" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/72ff852d-07ea-4cf7-8b3e-75e91ccd6844 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_cni_cilium.go:88]: failed to create HCP cluster "cilium-cl" without CNI Unexpected error: <fmt.wrapError \| 0xc000a44400>: failed to create HCP cluster cilium-cl: failed waiting for cluster="cilium-cl" in resourcegroup="cni-cilium-pffhm2cq6f4q" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/72ff852d-07ea-4cf7-8b3e-75e91ccd6844 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 13:55:50 UTC · failed at e2e · test Customer should be able to update node pool labels and taints · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_labels_taints.go:92]: failed to create HCP cluster cluster-np-labels-taints Unexpected error: <fmt.wrapError \| 0xc000994380>: failed to create HCP cluster cluster-np-labels-taints: failed waiting for cluster="cluster-np-labels-taints" in resourcegroup="rg-np-labels-taints-2v5hnvkbp6nz" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/ac014577-36dc-4208-bce8-f0e66ebed0a8 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_labels_taints.go:92]: failed to create HCP cluster cluster-np-labels-taints Unexpected error: <fmt.wrapError \| 0xc000994380>: failed to create HCP cluster cluster-np-labels-taints: failed waiting for cluster="cluster-np-labels-taints" in resourcegroup="rg-np-labels-taints-2v5hnvkbp6nz" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/ac014577-36dc-4208-bce8-f0e66ebed0a8 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 13:55:50 UTC · failed at e2e · test Customer should upgrade a nodepool to a version without Cincinnati upgrade edge z-stream upgrade without Cincinnati edge in 4.20 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:295]: Unexpected error: <errors.joinError \| 0xc0012b4300>: failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster="np-noedge-vf56pz" in resourcegroup="rg-np-noedge-vf56pz-nwsgmksht5fp" to finish deleting: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/998f1803-4553-4717-9f65-bbfa6f28ace6 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:295]: Unexpected error: <errors.joinError \| 0xc0012b4300>: failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster="np-noedge-vf56pz" in resourcegroup="rg-np-noedge-vf56pz-nwsgmksht5fp" to finish deleting: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/998f1803-4553-4717-9f65-bbfa6f28ace6 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 13:55:50 UTC · failed at e2e · test Customer should be able to delete an HCP cluster whose managed identities were deleted first · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delete_missing_identities.go:113]: failed to create HCP cluster "missing-mi-hcp-cluster" Unexpected error: <fmt.wrapError \| 0xc001290360>: failed to create HCP cluster missing-mi-hcp-cluster: failed waiting for cluster="missing-mi-hcp-cluster" in resourcegroup="delete-mi-rg-ghbftw2cv4rh" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/66de4363-9d94-46ea-bfff-873cb3efa5b1 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delete_missing_identities.go:113]: failed to create HCP cluster "missing-mi-hcp-cluster" Unexpected error: <fmt.wrapError \| 0xc001290360>: failed to create HCP cluster missing-mi-hcp-cluster: failed waiting for cluster="missing-mi-hcp-cluster" in resourcegroup="delete-mi-rg-ghbftw2cv4rh" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/66de4363-9d94-46ea-bfff-873cb3efa5b1 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 13:55:50 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.22 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:123]: HCP cluster rg-candidate-4-22-pctpkcgcwr47/cluster-candidate-4-22-fqmxwl should provision Unexpected error: <fmt.wrapError \| 0xc000c2b560>: failed to create HCP cluster cluster-candidate-4-22-fqmxwl: failed waiting for cluster="cluster-candidate-4-22-fqmxwl" in resourcegroup="rg-candidate-4-22-pctpkcgcwr47" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/3b87b683-d3ee-4c0c-a392-3e0b820cd2d6 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:123]: HCP cluster rg-candidate-4-22-pctpkcgcwr47/cluster-candidate-4-22-fqmxwl should provision Unexpected error: <fmt.wrapError \| 0xc000c2b560>: failed to create HCP cluster cluster-candidate-4-22-fqmxwl: failed waiting for cluster="cluster-candidate-4-22-fqmxwl" in resourcegroup="rg-candidate-4-22-pctpkcgcwr47" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/3b87b683-d3ee-4c0c-a392-3e0b820cd2d6 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 13:55:50 UTC · failed at e2e · test Customer should be able to successfully upgrade control plane minor version from 4.21 minor to 4.22 minor · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:133]: failed to create HCP cluster "cp-ystream-upgrade-4-22-74dgdp" at install version 4.21 Unexpected error: <fmt.wrapError \| 0xc000a36500>: failed to create HCP cluster cp-ystream-upgrade-4-22-74dgdp: failed waiting for cluster="cp-ystream-upgrade-4-22-74dgdp" in resourcegroup="rg-cp-ystream-upgrade-4-22-m9l42vdrlhj8" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/ec116fea-ca7e-4a3f-9d3f-5f3c8f61a81e -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:133]: failed to create HCP cluster "cp-ystream-upgrade-4-22-74dgdp" at install version 4.21 Unexpected error: <fmt.wrapError \| 0xc000a36500>: failed to create HCP cluster cp-ystream-upgrade-4-22-74dgdp: failed waiting for cluster="cp-ystream-upgrade-4-22-74dgdp" in resourcegroup="rg-cp-ystream-upgrade-4-22-m9l42vdrlhj8" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/ec116fea-ca7e-4a3f-9d3f-5f3c8f61a81e -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 13:55:50 UTC · failed at e2e · test Authorized CIDRs Connectivity should allow API access only from authorized VM IP · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:111]: failed to create HCP cluster "cidr-connectivity-test" with authorized CIDRs Unexpected error: <fmt.wrapError \| 0xc000f06380>: failed to create HCP cluster cidr-connectivity-test: failed waiting for cluster="cidr-connectivity-test" in resourcegroup="e2e-cidr-connectivity-sgj22g9zp2jg" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/ed833655-04f4-4a8d-83cd-dfd1d5a5550d -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:111]: failed to create HCP cluster "cidr-connectivity-test" with authorized CIDRs Unexpected error: <fmt.wrapError \| 0xc000f06380>: failed to create HCP cluster cidr-connectivity-test: failed waiting for cluster="cidr-connectivity-test" in resourcegroup="e2e-cidr-connectivity-sgj22g9zp2jg" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/ed833655-04f4-4a8d-83cd-dfd1d5a5550d -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 13:55:50 UTC · failed at e2e · test Customer should be able to create an HCP cluster with custom autoscaling and update the autoscaling configuration · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:108]: failed to create HCP cluster "autoscaling-hcp-cluster" with custom autoscaling Unexpected error: <fmt.wrapError \| 0xc0011104a0>: failed to create HCP cluster autoscaling-hcp-cluster: failed waiting for cluster="autoscaling-hcp-cluster" in resourcegroup="autoscaling-cluster-t9ln8p8l2tq5" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/7b953e56-e747-4da4-a09c-0706792a8104 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:108]: failed to create HCP cluster "autoscaling-hcp-cluster" with custom autoscaling Unexpected error: <fmt.wrapError \| 0xc0011104a0>: failed to create HCP cluster autoscaling-hcp-cluster: failed waiting for cluster="autoscaling-hcp-cluster" in resourcegroup="autoscaling-cluster-t9ln8p8l2tq5" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/7b953e56-e747-4da4-a09c-0706792a8104 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 13:55:50 UTC · failed at e2e · test Customer should be able to create a cluster with an external auth config and get the external auth config · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:94]: failed to create HCP cluster for external auth test Unexpected error: <fmt.wrapError \| 0xc000d744c0>: failed to create HCP cluster ea-cluster: failed waiting for cluster="ea-cluster" in resourcegroup="external-auth-cluster-6lxzqp285qjw" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4c99174c-b9a9-41b9-b961-8dea7b33a43a -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:94]: failed to create HCP cluster for external auth test Unexpected error: <fmt.wrapError \| 0xc000d744c0>: failed to create HCP cluster ea-cluster: failed waiting for cluster="ea-cluster" in resourcegroup="external-auth-cluster-6lxzqp285qjw" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4c99174c-b9a9-41b9-b961-8dea7b33a43a -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 13:55:50 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.21 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:123]: HCP cluster rg-candidate-4-21-cjhdsrxcz2fx/cluster-candidate-4-21-ktgnhz should provision Unexpected error: <fmt.wrapError \| 0xc000d5e480>: failed to create HCP cluster cluster-candidate-4-21-ktgnhz: failed waiting for cluster="cluster-candidate-4-21-ktgnhz" in resourcegroup="rg-candidate-4-21-cjhdsrxcz2fx" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/db8d62fd-625d-44ba-891b-e4b1141e3aaf -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:123]: HCP cluster rg-candidate-4-21-cjhdsrxcz2fx/cluster-candidate-4-21-ktgnhz should provision Unexpected error: <fmt.wrapError \| 0xc000d5e480>: failed to create HCP cluster cluster-candidate-4-21-ktgnhz: failed waiting for cluster="cluster-candidate-4-21-ktgnhz" in resourcegroup="rg-candidate-4-21-cjhdsrxcz2fx" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/db8d62fd-625d-44ba-891b-e4b1141e3aaf -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred
09:46:15	21m 56s	branch-ci-Azure-ARO-HCP-main-e2e-stage-e2e-parallel post-goodmerged PR No additional run metadata captured	uksouth	e2e	failed	n/a	58	e2e (58) ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 09:46:15 UTC · failed at e2e · test Customer should be able to create an HCP cluster with Image Registry not present · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/image_registry_cluster_create.go:89]: failed to create HCP cluster with image registry disabled Unexpected error: <fmt.wrapError \| 0xc000eb4320>: failed to create HCP cluster disabled-image-registry-hcp-cluster: failed waiting for cluster="disabled-image-registry-hcp-cluster" in resourcegroup="disabled-image-registry-cwpk2drntd97" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/faf859b1-74ca-422c-b81f-40ff813f96a2 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/image_registry_cluster_create.go:89]: failed to create HCP cluster with image registry disabled Unexpected error: <fmt.wrapError \| 0xc000eb4320>: failed to create HCP cluster disabled-image-registry-hcp-cluster: failed waiting for cluster="disabled-image-registry-hcp-cluster" in resourcegroup="disabled-image-registry-cwpk2drntd97" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/faf859b1-74ca-422c-b81f-40ff813f96a2 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 09:46:15 UTC · failed at e2e · test Customer should be able to successfully upgrade control plane minor version from 4.20 minor to 4.21 minor · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:133]: failed to create HCP cluster "cp-ystream-upgrade-4-21-fvtqgz" at install version 4.20 Unexpected error: <fmt.wrapError \| 0xc0018fe320>: failed to create HCP cluster cp-ystream-upgrade-4-21-fvtqgz: failed waiting for cluster="cp-ystream-upgrade-4-21-fvtqgz" in resourcegroup="rg-cp-ystream-upgrade-4-21-6r7mpcjl4xmc" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/e165024c-c364-4891-809d-d1b8ec4c0137 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:133]: failed to create HCP cluster "cp-ystream-upgrade-4-21-fvtqgz" at install version 4.20 Unexpected error: <fmt.wrapError \| 0xc0018fe320>: failed to create HCP cluster cp-ystream-upgrade-4-21-fvtqgz: failed waiting for cluster="cp-ystream-upgrade-4-21-fvtqgz" in resourcegroup="rg-cp-ystream-upgrade-4-21-6r7mpcjl4xmc" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/e165024c-c364-4891-809d-d1b8ec4c0137 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 09:46:15 UTC · failed at e2e · test Update HCPOpenShiftCluster Positive creates a cluster and updates tags with a PATCH request · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:161]: failed to create HCP cluster for patch-tags test Unexpected error: <fmt.wrapError \| 0xc000a863e0>: failed to create HCP cluster patch-tags-cluster: failed waiting for cluster="patch-tags-cluster" in resourcegroup="patch-tags-xp5s75thlg74" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/5a1c6cd7-0b9c-44ad-b5d5-b0ae7c231b30 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:161]: failed to create HCP cluster for patch-tags test Unexpected error: <fmt.wrapError \| 0xc000a863e0>: failed to create HCP cluster patch-tags-cluster: failed waiting for cluster="patch-tags-cluster" in resourcegroup="patch-tags-xp5s75thlg74" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/5a1c6cd7-0b9c-44ad-b5d5-b0ae7c231b30 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 09:46:15 UTC · failed at e2e · test Customer should upgrade and update a nodepool from 4.20.z to 4.20.zLatest · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:146]: failed to create HCP cluster np-version-upgrade-cluster-ld27nr with version 4.20.39 Unexpected error: <fmt.wrapError \| 0xc000d0e560>: failed to create HCP cluster np-version-upgrade-cluster-ld27nr: failed waiting for cluster="np-version-upgrade-cluster-ld27nr" in resourcegroup="rg-np-version-upgrade-ld27nr-8f7lqzmgbkgh" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/91dfcb8f-f729-443a-a281-0b106e15369f -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:146]: failed to create HCP cluster np-version-upgrade-cluster-ld27nr with version 4.20.39 Unexpected error: <fmt.wrapError \| 0xc000d0e560>: failed to create HCP cluster np-version-upgrade-cluster-ld27nr: failed waiting for cluster="np-version-upgrade-cluster-ld27nr" in resourcegroup="rg-np-version-upgrade-ld27nr-8f7lqzmgbkgh" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/91dfcb8f-f729-443a-a281-0b106e15369f -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 09:46:15 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 5.0 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:123]: HCP cluster rg-candidate-5-0-d2xngq5ft4rm/cluster-candidate-5-0-p8cf4r should provision Unexpected error: <fmt.wrapError \| 0xc001698320>: failed to create HCP cluster cluster-candidate-5-0-p8cf4r: failed waiting for cluster="cluster-candidate-5-0-p8cf4r" in resourcegroup="rg-candidate-5-0-d2xngq5ft4rm" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/c9af701d-bf05-4976-a460-12142920482e -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:123]: HCP cluster rg-candidate-5-0-d2xngq5ft4rm/cluster-candidate-5-0-p8cf4r should provision Unexpected error: <fmt.wrapError \| 0xc001698320>: failed to create HCP cluster cluster-candidate-5-0-p8cf4r: failed waiting for cluster="cluster-candidate-5-0-p8cf4r" in resourcegroup="rg-candidate-5-0-d2xngq5ft4rm" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/c9af701d-bf05-4976-a460-12142920482e -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 09:46:15 UTC · failed at e2e · test Customer should be able to use workload identity via the cluster OIDC issuer URL · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/oidc_issuer_workload_identity.go:227]: failed to create HCP cluster oidc-wi-cluster Unexpected error: <fmt.wrapError \| 0xc000af78a0>: failed to create HCP cluster oidc-wi-cluster: failed waiting for cluster="oidc-wi-cluster" in resourcegroup="oidc-wi-sph95f8lnpwg" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/1313ecbf-d0cc-4d2d-9b07-84a74a29744b -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/oidc_issuer_workload_identity.go:227]: failed to create HCP cluster oidc-wi-cluster Unexpected error: <fmt.wrapError \| 0xc000af78a0>: failed to create HCP cluster oidc-wi-cluster: failed waiting for cluster="oidc-wi-cluster" in resourcegroup="oidc-wi-sph95f8lnpwg" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/1313ecbf-d0cc-4d2d-9b07-84a74a29744b -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 09:46:15 UTC · failed at e2e · test Customer should not be able to reuse subnets and NSGs between clusters · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_nsg_subnet_reuse.go:107]: failed to create seed HCP cluster basic-cluster Unexpected error: <fmt.wrapError \| 0xc000d08420>: failed to create HCP cluster basic-cluster: failed waiting for cluster="basic-cluster" in resourcegroup="rg-cluster-nsg-subnet-reuse-7n9t7cs46q4q" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/f9cbfc48-8e76-42d6-a1f1-a8774fadc2e3 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_nsg_subnet_reuse.go:107]: failed to create seed HCP cluster basic-cluster Unexpected error: <fmt.wrapError \| 0xc000d08420>: failed to create HCP cluster basic-cluster: failed waiting for cluster="basic-cluster" in resourcegroup="rg-cluster-nsg-subnet-reuse-7n9t7cs46q4q" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/f9cbfc48-8e76-42d6-a1f1-a8774fadc2e3 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 09:46:15 UTC · failed at e2e · test Customer should be able to create node pools with ARM64-based VMs · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/arm64_nodepool.go:88]: failed to create HCP cluster "arm64-vm-hcp-cluster" Unexpected error: <fmt.wrapError \| 0xc001932580>: failed to create HCP cluster arm64-vm-hcp-cluster: failed waiting for cluster="arm64-vm-hcp-cluster" in resourcegroup="arm64-vm-cluster-mrlc55xjgcmk" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/9bb82f6c-adbb-43a6-962f-7137873d0f24 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/arm64_nodepool.go:88]: failed to create HCP cluster "arm64-vm-hcp-cluster" Unexpected error: <fmt.wrapError \| 0xc001932580>: failed to create HCP cluster arm64-vm-hcp-cluster: failed waiting for cluster="arm64-vm-hcp-cluster" in resourcegroup="arm64-vm-cluster-mrlc55xjgcmk" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/9bb82f6c-adbb-43a6-962f-7137873d0f24 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 09:46:15 UTC · failed at e2e · test Customer should be able to create several HCP clusters in their customer resource group, but not in the same managed resource group · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:131]: failed to wait for first cluster "basic-hcp-cluster" to complete creation (timeout '30.000000' minutes) Unexpected error: <exported.ResponseError \| 0xc000f21b90>: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/2e994b63-b5d1-4d53-9ad6-35d7d81b98ee -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:131]: failed to wait for first cluster "basic-hcp-cluster" to complete creation (timeout '30.000000' minutes) Unexpected error: <exported.ResponseError \| 0xc000f21b90>: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/2e994b63-b5d1-4d53-9ad6-35d7d81b98ee -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 09:46:15 UTC · failed at e2e · test Customer should downgrade a nodepool to a lower minor version from 4.22.zLatest to 4.20.zLatest · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:720]: failed to create HCP cluster Unexpected error: <fmt.wrapError \| 0xc001102140>: failed to create HCP cluster np-dg-minor-9bjdkb: failed waiting for cluster="np-dg-minor-9bjdkb" in resourcegroup="rg-np-dg-minor-9bjdkb-r76v7w6gdltx" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/24bc8bbf-18f2-4cec-897d-e2520fc04755 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:720]: failed to create HCP cluster Unexpected error: <fmt.wrapError \| 0xc001102140>: failed to create HCP cluster np-dg-minor-9bjdkb: failed waiting for cluster="np-dg-minor-9bjdkb" in resourcegroup="rg-np-dg-minor-9bjdkb-r76v7w6gdltx" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/24bc8bbf-18f2-4cec-897d-e2520fc04755 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 09:46:15 UTC · failed at e2e · test Customer should be able to forward control plane logs to a storage account and Event Hub via shoebox diagnostic settings · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/shoebox_logs.go:272]: failed to create HCP cluster shoebox-hcp-cluster Unexpected error: <fmt.wrapError \| 0xc0008a4480>: failed to create HCP cluster shoebox-hcp-cluster: failed waiting for cluster="shoebox-hcp-cluster" in resourcegroup="shoebox-cluster-4mwltxjn8kws" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/676e4bc8-7f06-4673-897b-29f113927a9f -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/shoebox_logs.go:272]: failed to create HCP cluster shoebox-hcp-cluster Unexpected error: <fmt.wrapError \| 0xc0008a4480>: failed to create HCP cluster shoebox-hcp-cluster: failed waiting for cluster="shoebox-hcp-cluster" in resourcegroup="shoebox-cluster-4mwltxjn8kws" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/676e4bc8-7f06-4673-897b-29f113927a9f -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 09:46:15 UTC · failed at e2e · test Customer should create an HCP cluster and validate TLS certificates · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:96]: failed to create HCP cluster for TLS endpoint test Unexpected error: <fmt.wrapError \| 0xc000f042c0>: failed to create HCP cluster tls-endpoint-hcp-cluster: failed waiting for cluster="tls-endpoint-hcp-cluster" in resourcegroup="tls-endpoint-cluster-8pm5zswdndlq" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b3a4ed24-0e21-47fa-a184-ad76c454abd9 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:96]: failed to create HCP cluster for TLS endpoint test Unexpected error: <fmt.wrapError \| 0xc000f042c0>: failed to create HCP cluster tls-endpoint-hcp-cluster: failed waiting for cluster="tls-endpoint-hcp-cluster" in resourcegroup="tls-endpoint-cluster-8pm5zswdndlq" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b3a4ed24-0e21-47fa-a184-ad76c454abd9 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 09:46:15 UTC · failed at e2e · test Customer should be able to create a no-CNI private cluster with a private key vault, a nodepool and install cilium CNI successfully · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:107]: failed to create HCP cluster "cilium-cluster" with no CNI and private etcd Unexpected error: <fmt.wrapError \| 0xc0012061a0>: failed waiting for cluster="cilium-cluster" in resourcegroup="complex-cilium-kv-qcrqtc27wlb2" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/a314c15b-4ea8-4cb8-a031-af4ed11745d9 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:107]: failed to create HCP cluster "cilium-cluster" with no CNI and private etcd Unexpected error: <fmt.wrapError \| 0xc0012061a0>: failed waiting for cluster="cilium-cluster" in resourcegroup="complex-cilium-kv-qcrqtc27wlb2" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/a314c15b-4ea8-4cb8-a031-af4ed11745d9 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 09:46:15 UTC · failed at e2e · test Update HCPOpenShiftCluster Negative creates a cluster and fails to update its name with a PATCH request · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:84]: failed to create HCP cluster for patch-name test Unexpected error: <fmt.wrapError \| 0xc00056e5e0>: failed to create HCP cluster patch-name-cluster: failed waiting for cluster="patch-name-cluster" in resourcegroup="patch-name-pcgwd4zkwwvz" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/5dbc5fd5-8da6-4211-82d1-748fd3bbec20 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:84]: failed to create HCP cluster for patch-name test Unexpected error: <fmt.wrapError \| 0xc00056e5e0>: failed to create HCP cluster patch-name-cluster: failed waiting for cluster="patch-name-cluster" in resourcegroup="patch-name-pcgwd4zkwwvz" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/5dbc5fd5-8da6-4211-82d1-748fd3bbec20 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 09:46:15 UTC · failed at e2e · test Customer should create a cluster with private ingress using v20260901preview and verify the ingress is internal · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_ingress.go:87]: failed to create HCP cluster "private-ingress" with private ingress Unexpected error: <fmt.wrapError \| 0xc0010183e0>: failed to create HCP cluster private-ingress: failed waiting for cluster="private-ingress" in resourcegroup="private-ingress-xl9vljxcqtsm" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/d795397f-d472-48b0-8abb-cee5783a4cab -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_ingress.go:87]: failed to create HCP cluster "private-ingress" with private ingress Unexpected error: <fmt.wrapError \| 0xc0010183e0>: failed to create HCP cluster private-ingress: failed waiting for cluster="private-ingress" in resourcegroup="private-ingress-xl9vljxcqtsm" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/d795397f-d472-48b0-8abb-cee5783a4cab -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 09:46:15 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 4.21 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:123]: failed to create HCP cluster "cluster-zstream-4-21-wx28cn" with version 4.21.33 on candidate channel Unexpected error: <fmt.wrapError \| 0xc0007e25a0>: failed to create HCP cluster cluster-zstream-4-21-wx28cn: failed waiting for cluster="cluster-zstream-4-21-wx28cn" in resourcegroup="rg-zstream-upgrade-4-21-989jzmj5vx85" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/d6b54dcf-b16f-4851-8048-129f5e92a00d -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:123]: failed to create HCP cluster "cluster-zstream-4-21-wx28cn" with version 4.21.33 on candidate channel Unexpected error: <fmt.wrapError \| 0xc0007e25a0>: failed to create HCP cluster cluster-zstream-4-21-wx28cn: failed waiting for cluster="cluster-zstream-4-21-wx28cn" in resourcegroup="rg-zstream-upgrade-4-21-989jzmj5vx85" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/d6b54dcf-b16f-4851-8048-129f5e92a00d -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 09:46:15 UTC · failed at e2e · test Customer should be able to delete an HCP cluster whose managed identities were deleted first · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delete_missing_identities.go:113]: failed to create HCP cluster "missing-mi-hcp-cluster" Unexpected error: <fmt.wrapError \| 0xc001086460>: failed to create HCP cluster missing-mi-hcp-cluster: failed waiting for cluster="missing-mi-hcp-cluster" in resourcegroup="delete-mi-rg-hhhh4fbv65rt" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/3aecc727-f2bc-4209-8da8-27f94954007a -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delete_missing_identities.go:113]: failed to create HCP cluster "missing-mi-hcp-cluster" Unexpected error: <fmt.wrapError \| 0xc001086460>: failed to create HCP cluster missing-mi-hcp-cluster: failed waiting for cluster="missing-mi-hcp-cluster" in resourcegroup="delete-mi-rg-hhhh4fbv65rt" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/3aecc727-f2bc-4209-8da8-27f94954007a -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 09:46:15 UTC · failed at e2e · test Customer should be able to create an HCP cluster with back-level version 4.20 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:152]: failed to create HCP cluster version 4.20 Unexpected error: <fmt.wrapError \| 0xc0010ac200>: failed waiting for cluster="cluster-ver-4-20" in resourcegroup="rg-cluster-back-version-8n6csl5lbht7" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/428f6b41-8121-48a1-992f-9dbbf2d7af81 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:152]: failed to create HCP cluster version 4.20 Unexpected error: <fmt.wrapError \| 0xc0010ac200>: failed waiting for cluster="cluster-ver-4-20" in resourcegroup="rg-cluster-back-version-8n6csl5lbht7" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/428f6b41-8121-48a1-992f-9dbbf2d7af81 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 09:46:15 UTC · failed at e2e · test Authorized CIDRs Connectivity should allow API access only from authorized VM IP · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:111]: failed to create HCP cluster "cidr-connectivity-test" with authorized CIDRs Unexpected error: <fmt.wrapError \| 0xc00111cf60>: failed to create HCP cluster cidr-connectivity-test: failed waiting for cluster="cidr-connectivity-test" in resourcegroup="e2e-cidr-connectivity-58z5l9rhf4f6" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/a673e929-1d31-44e9-92e0-dad0c7c87a72 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:111]: failed to create HCP cluster "cidr-connectivity-test" with authorized CIDRs Unexpected error: <fmt.wrapError \| 0xc00111cf60>: failed to create HCP cluster cidr-connectivity-test: failed waiting for cluster="cidr-connectivity-test" in resourcegroup="e2e-cidr-connectivity-58z5l9rhf4f6" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/a673e929-1d31-44e9-92e0-dad0c7c87a72 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 09:46:15 UTC · failed at e2e · test Customer should be able to successfully upgrade control plane minor version from 4.22 minor to 4.23 minor · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:133]: failed to create HCP cluster "cp-ystream-upgrade-4-23-kdmnj4" at install version 4.22 Unexpected error: <fmt.wrapError \| 0xc0006e8e80>: failed to create HCP cluster cp-ystream-upgrade-4-23-kdmnj4: failed waiting for cluster="cp-ystream-upgrade-4-23-kdmnj4" in resourcegroup="rg-cp-ystream-upgrade-4-23-97rzkflgq2nk" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/adbdf657-33a9-407f-8747-1643c0105f4a -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:133]: failed to create HCP cluster "cp-ystream-upgrade-4-23-kdmnj4" at install version 4.22 Unexpected error: <fmt.wrapError \| 0xc0006e8e80>: failed to create HCP cluster cp-ystream-upgrade-4-23-kdmnj4: failed waiting for cluster="cp-ystream-upgrade-4-23-kdmnj4" in resourcegroup="rg-cp-ystream-upgrade-4-23-97rzkflgq2nk" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/adbdf657-33a9-407f-8747-1643c0105f4a -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 09:46:15 UTC · failed at e2e · test Create HCPOpenShiftCluster with Private KeyVault should create a cluster with private keyvault using v20251223preview API · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:99]: failed to create HCP cluster "private-kv-cluster" with private keyvault Unexpected error: <fmt.wrapError \| 0xc00100a2c0>: failed waiting for cluster="private-kv-cluster" in resourcegroup="private-keyvault-2z4prgng29d4" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/e58606bc-443f-4bc1-b14a-e820a57ffeae -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:99]: failed to create HCP cluster "private-kv-cluster" with private keyvault Unexpected error: <fmt.wrapError \| 0xc00100a2c0>: failed waiting for cluster="private-kv-cluster" in resourcegroup="private-keyvault-2z4prgng29d4" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/e58606bc-443f-4bc1-b14a-e820a57ffeae -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 09:46:15 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 4.20 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:123]: failed to create HCP cluster "cluster-zstream-4-20-rxg6m4" with version 4.20.38 on candidate channel Unexpected error: <fmt.wrapError \| 0xc000f881a0>: failed to create HCP cluster cluster-zstream-4-20-rxg6m4: failed waiting for cluster="cluster-zstream-4-20-rxg6m4" in resourcegroup="rg-zstream-upgrade-4-20-bbkphfsgk8l5" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/bacd2f24-2a1b-478d-a054-4475c06ee047 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:123]: failed to create HCP cluster "cluster-zstream-4-20-rxg6m4" with version 4.20.38 on candidate channel Unexpected error: <fmt.wrapError \| 0xc000f881a0>: failed to create HCP cluster cluster-zstream-4-20-rxg6m4: failed waiting for cluster="cluster-zstream-4-20-rxg6m4" in resourcegroup="rg-zstream-upgrade-4-20-bbkphfsgk8l5" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/bacd2f24-2a1b-478d-a054-4475c06ee047 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 09:46:15 UTC · failed at e2e · test Customer should create a cluster with private KAS and verify API server is only reachable from VNet while ingress remains public · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:91]: failed to create HCP cluster "private-kas" with private KAS Unexpected error: <fmt.wrapError \| 0xc0029904a0>: failed to create HCP cluster private-kas: failed waiting for cluster="private-kas" in resourcegroup="private-kas-9f25phpd7qnd" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/926e4f43-e783-4a3d-8146-4a049b151d3d -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:91]: failed to create HCP cluster "private-kas" with private KAS Unexpected error: <fmt.wrapError \| 0xc0029904a0>: failed to create HCP cluster private-kas: failed waiting for cluster="private-kas" in resourcegroup="private-kas-9f25phpd7qnd" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/926e4f43-e783-4a3d-8146-4a049b151d3d -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 09:46:15 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.22 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:123]: HCP cluster rg-candidate-4-22-jn62zfptdlfh/cluster-candidate-4-22-bhd4rp should provision Unexpected error: <fmt.wrapError \| 0xc000e5c2e0>: failed to create HCP cluster cluster-candidate-4-22-bhd4rp: failed waiting for cluster="cluster-candidate-4-22-bhd4rp" in resourcegroup="rg-candidate-4-22-jn62zfptdlfh" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/f434c010-1066-4c2d-b10d-e80bae8b14bb -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:123]: HCP cluster rg-candidate-4-22-jn62zfptdlfh/cluster-candidate-4-22-bhd4rp should provision Unexpected error: <fmt.wrapError \| 0xc000e5c2e0>: failed to create HCP cluster cluster-candidate-4-22-bhd4rp: failed waiting for cluster="cluster-candidate-4-22-bhd4rp" in resourcegroup="rg-candidate-4-22-jn62zfptdlfh" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/f434c010-1066-4c2d-b10d-e80bae8b14bb -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 09:46:15 UTC · failed at e2e · test Customer should not be able to perform various invalid operations on cluster resources · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/simple_negative_cases.go:110]: failed to create HCP cluster negative-tests-cluster Unexpected error: <fmt.wrapError \| 0xc000a034a0>: failed to create HCP cluster negative-tests-cluster: failed waiting for cluster="negative-tests-cluster" in resourcegroup="rg-negative-tests-kj7lztprpdgp" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/2a7a23aa-7e52-4cb5-85bc-a2781b58b1db -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/simple_negative_cases.go:110]: failed to create HCP cluster negative-tests-cluster Unexpected error: <fmt.wrapError \| 0xc000a034a0>: failed to create HCP cluster negative-tests-cluster: failed waiting for cluster="negative-tests-cluster" in resourcegroup="rg-negative-tests-kj7lztprpdgp" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/2a7a23aa-7e52-4cb5-85bc-a2781b58b1db -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 09:46:15 UTC · failed at e2e · test ARO HCP Service should successfully provision cluster when MI role assignments are added after cluster creation · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:238]: failed to get admin REST config for HCP cluster Unexpected error: <fmt.wrapError \| 0xc0006c2ea0>: failed waiting for hcpCluster="delayed-rbac-cluster" in resourcegroup="delayed-rbac-v7vgdzrnb8bv" to finish getting creds: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/bcb4f189-fe27-454c-928a-272dd865e652 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:238]: failed to get admin REST config for HCP cluster Unexpected error: <fmt.wrapError \| 0xc0006c2ea0>: failed waiting for hcpCluster="delayed-rbac-cluster" in resourcegroup="delayed-rbac-v7vgdzrnb8bv" to finish getting creds: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/bcb4f189-fe27-454c-928a-272dd865e652 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 09:46:15 UTC · failed at e2e · test Customer should be able to test admin credentials before cluster ready, then full admin credential lifecycle · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:190]: failed to request admin credential 1 Unexpected error: <fmt.wrapError \| 0xc0006212a0>: failed waiting for hcpCluster="admin-cred-lifecycle-4pv4j7" in resourcegroup="admin-credential-lifecycle-test-4sn6n5rmm9wr" to finish getting creds: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0db35508-1cd6-4c7b-8bdd-a380971142ae -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:190]: failed to request admin credential 1 Unexpected error: <fmt.wrapError \| 0xc0006212a0>: failed waiting for hcpCluster="admin-cred-lifecycle-4pv4j7" in resourcegroup="admin-credential-lifecycle-test-4sn6n5rmm9wr" to finish getting creds: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0db35508-1cd6-4c7b-8bdd-a380971142ae -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 09:46:15 UTC · failed at e2e · test Customer should be able to update node pool labels and taints · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_labels_taints.go:92]: failed to create HCP cluster cluster-np-labels-taints Unexpected error: <fmt.wrapError \| 0xc000d44160>: failed to create HCP cluster cluster-np-labels-taints: failed waiting for cluster="cluster-np-labels-taints" in resourcegroup="rg-np-labels-taints-hn22tfmqv6xl" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/be96828c-3724-4e43-acab-c552ce551ce2 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_labels_taints.go:92]: failed to create HCP cluster cluster-np-labels-taints Unexpected error: <fmt.wrapError \| 0xc000d44160>: failed to create HCP cluster cluster-np-labels-taints: failed waiting for cluster="cluster-np-labels-taints" in resourcegroup="rg-np-labels-taints-hn22tfmqv6xl" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/be96828c-3724-4e43-acab-c552ce551ce2 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 09:46:15 UTC · failed at e2e · test Customer should create a cluster using v20260901preview API and verify cluster health · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_v20260901preview.go:86]: failed to create HCP cluster "v20260901" via v20260901preview Unexpected error: <fmt.wrapError \| 0xc000fe65c0>: failed to create HCP cluster v20260901: failed waiting for cluster="v20260901" in resourcegroup="v20260901preview-hkv2w5rmrwls" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4596f551-7934-4ecf-85de-65361c422156 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_v20260901preview.go:86]: failed to create HCP cluster "v20260901" via v20260901preview Unexpected error: <fmt.wrapError \| 0xc000fe65c0>: failed to create HCP cluster v20260901: failed waiting for cluster="v20260901" in resourcegroup="v20260901preview-hkv2w5rmrwls" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4596f551-7934-4ecf-85de-65361c422156 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 09:46:15 UTC · failed at e2e · test Customer should be able to lifecycle and confirm external auth on a cluster · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_list_and_verify.go:87]: failed to create HCP cluster for external auth list test Unexpected error: <fmt.wrapError \| 0xc00056e520>: failed to create HCP cluster ea-listr7gzhr: failed waiting for cluster="ea-listr7gzhr" in resourcegroup="ea-list-q4f4j8wlsbgz" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/7b621f6b-d48c-483e-947d-297263d4c5b1 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_list_and_verify.go:87]: failed to create HCP cluster for external auth list test Unexpected error: <fmt.wrapError \| 0xc00056e520>: failed to create HCP cluster ea-listr7gzhr: failed waiting for cluster="ea-listr7gzhr" in resourcegroup="ea-list-q4f4j8wlsbgz" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/7b621f6b-d48c-483e-947d-297263d4c5b1 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 09:46:15 UTC · failed at e2e · test Customer should be able to successfully upgrade control plane minor version from 4.21 minor to 4.22 minor · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:133]: failed to create HCP cluster "cp-ystream-upgrade-4-22-hmhj8c" at install version 4.21 Unexpected error: <fmt.wrapError \| 0xc0018ee380>: failed to create HCP cluster cp-ystream-upgrade-4-22-hmhj8c: failed waiting for cluster="cp-ystream-upgrade-4-22-hmhj8c" in resourcegroup="rg-cp-ystream-upgrade-4-22-4rhpk8x2q865" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/6372ad7b-58eb-487c-8a40-fe3745dd5ed5 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:133]: failed to create HCP cluster "cp-ystream-upgrade-4-22-hmhj8c" at install version 4.21 Unexpected error: <fmt.wrapError \| 0xc0018ee380>: failed to create HCP cluster cp-ystream-upgrade-4-22-hmhj8c: failed waiting for cluster="cp-ystream-upgrade-4-22-hmhj8c" in resourcegroup="rg-cp-ystream-upgrade-4-22-4rhpk8x2q865" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/6372ad7b-58eb-487c-8a40-fe3745dd5ed5 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 09:46:15 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.23 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:123]: HCP cluster rg-candidate-4-23-pzxm4gv78clx/cluster-candidate-4-23-glrb6j should provision Unexpected error: <fmt.wrapError \| 0xc00121c300>: failed to create HCP cluster cluster-candidate-4-23-glrb6j: failed waiting for cluster="cluster-candidate-4-23-glrb6j" in resourcegroup="rg-candidate-4-23-pzxm4gv78clx" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/1ef5a635-8738-4780-8a64-b986c2c5543e -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:123]: HCP cluster rg-candidate-4-23-pzxm4gv78clx/cluster-candidate-4-23-glrb6j should provision Unexpected error: <fmt.wrapError \| 0xc00121c300>: failed to create HCP cluster cluster-candidate-4-23-glrb6j: failed waiting for cluster="cluster-candidate-4-23-glrb6j" in resourcegroup="rg-candidate-4-23-pzxm4gv78clx" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/1ef5a635-8738-4780-8a64-b986c2c5543e -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 09:46:15 UTC · failed at e2e · test Customer should be able to create a cluster with default autoscaling and a nodepool with autoscaling enabled up to replica limits · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:102]: failed to create HCP cluster np-autoscale-cluster Unexpected error: <fmt.wrapError \| 0xc00188a400>: failed to create HCP cluster np-autoscale-cluster: failed waiting for cluster="np-autoscale-cluster" in resourcegroup="np-autoscaling-2mv5fcnfnwhx" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/e15b827e-4275-470e-95ae-1bf6fedafeff -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:102]: failed to create HCP cluster np-autoscale-cluster Unexpected error: <fmt.wrapError \| 0xc00188a400>: failed to create HCP cluster np-autoscale-cluster: failed waiting for cluster="np-autoscale-cluster" in resourcegroup="np-autoscaling-2mv5fcnfnwhx" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/e15b827e-4275-470e-95ae-1bf6fedafeff -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 09:46:15 UTC · failed at e2e · test Customer should be able to create an HCP cluster and custom node pool osDisk size · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_nodepool_osdisk.go:80]: failed to create HCP cluster "hcp-cluster-np-128" Unexpected error: <fmt.wrapError \| 0xc000c74580>: failed to create HCP cluster hcp-cluster-np-128: failed waiting for cluster="hcp-cluster-np-128" in resourcegroup="clusternp128-9r6lld9b6rn6" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/f689573b-8f56-4906-b119-4990f4dda41c -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_nodepool_osdisk.go:80]: failed to create HCP cluster "hcp-cluster-np-128" Unexpected error: <fmt.wrapError \| 0xc000c74580>: failed to create HCP cluster hcp-cluster-np-128: failed waiting for cluster="hcp-cluster-np-128" in resourcegroup="clusternp128-9r6lld9b6rn6" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/f689573b-8f56-4906-b119-4990f4dda41c -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 09:46:15 UTC · failed at e2e · test Customer should upgrade a nodepool skipping one minor version (+2) from 4.20.z to 4.22.zLatest · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:441]: failed to create HCP cluster np-skip-minor-ngkdfq at version 4.22.15 Unexpected error: <fmt.wrapError \| 0xc0010fa140>: failed to create HCP cluster np-skip-minor-ngkdfq: failed waiting for cluster="np-skip-minor-ngkdfq" in resourcegroup="rg-np-skip-minor-ngkdfq-gtgs97cnfw88" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/6eaf0c32-6b6d-4990-9635-378ab7eb25d0 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:441]: failed to create HCP cluster np-skip-minor-ngkdfq at version 4.22.15 Unexpected error: <fmt.wrapError \| 0xc0010fa140>: failed to create HCP cluster np-skip-minor-ngkdfq: failed waiting for cluster="np-skip-minor-ngkdfq" in resourcegroup="rg-np-skip-minor-ngkdfq-gtgs97cnfw88" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/6eaf0c32-6b6d-4990-9635-378ab7eb25d0 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 09:46:15 UTC · failed at e2e · test Customer should respect cluster-wide node limits with nodepool autoscaling · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:291]: failed to create HCP cluster node-limit-cluster with MaxNodesTotal limit Unexpected error: <fmt.wrapError \| 0xc000fe2360>: failed to create HCP cluster node-limit-cluster: failed waiting for cluster="node-limit-cluster" in resourcegroup="node-limit-test-6h86gvb8f299" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/5e6a42c9-7d5d-41d2-8caf-6e8d40b86373 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:291]: failed to create HCP cluster node-limit-cluster with MaxNodesTotal limit Unexpected error: <fmt.wrapError \| 0xc000fe2360>: failed to create HCP cluster node-limit-cluster: failed waiting for cluster="node-limit-cluster" in resourcegroup="node-limit-test-6h86gvb8f299" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/5e6a42c9-7d5d-41d2-8caf-6e8d40b86373 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 09:46:15 UTC · failed at e2e · test Customer should be able to cordon, drain, and uncordon a node in an HCP cluster · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_cordon.go:90]: failed to create HCP cluster cordon-hcp-cluster Unexpected error: <fmt.wrapError \| 0xc00061e360>: failed to create HCP cluster cordon-hcp-cluster: failed waiting for cluster="cordon-hcp-cluster" in resourcegroup="rg-cordon-test-26ns9mv9l52c" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b9141558-2e58-48ca-85ee-2663e0c2091c -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_cordon.go:90]: failed to create HCP cluster cordon-hcp-cluster Unexpected error: <fmt.wrapError \| 0xc00061e360>: failed to create HCP cluster cordon-hcp-cluster: failed waiting for cluster="cordon-hcp-cluster" in resourcegroup="rg-cordon-test-26ns9mv9l52c" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b9141558-2e58-48ca-85ee-2663e0c2091c -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 09:46:15 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.20 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:123]: HCP cluster rg-candidate-4-20-fsffnj4lh882/cluster-candidate-4-20-zx8qdk should provision Unexpected error: <fmt.wrapError \| 0xc001338180>: failed to create HCP cluster cluster-candidate-4-20-zx8qdk: failed waiting for cluster="cluster-candidate-4-20-zx8qdk" in resourcegroup="rg-candidate-4-20-fsffnj4lh882" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/71315e35-48d6-4f78-b20e-dd7d8e2f3de8 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:123]: HCP cluster rg-candidate-4-20-fsffnj4lh882/cluster-candidate-4-20-zx8qdk should provision Unexpected error: <fmt.wrapError \| 0xc001338180>: failed to create HCP cluster cluster-candidate-4-20-zx8qdk: failed waiting for cluster="cluster-candidate-4-20-zx8qdk" in resourcegroup="rg-candidate-4-20-fsffnj4lh882" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/71315e35-48d6-4f78-b20e-dd7d8e2f3de8 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 09:46:15 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 5.0 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:123]: failed to create HCP cluster "cluster-zstream-5-0-s9btnw" with version 5.0.0-rc.2 on candidate channel Unexpected error: <fmt.wrapError \| 0xc000f16300>: failed to create HCP cluster cluster-zstream-5-0-s9btnw: failed waiting for cluster="cluster-zstream-5-0-s9btnw" in resourcegroup="rg-zstream-upgrade-5-0-rpfhb4xbm8lf" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/510afec5-4713-4c20-89bb-fb8bc789e01b -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:123]: failed to create HCP cluster "cluster-zstream-5-0-s9btnw" with version 5.0.0-rc.2 on candidate channel Unexpected error: <fmt.wrapError \| 0xc000f16300>: failed to create HCP cluster cluster-zstream-5-0-s9btnw: failed waiting for cluster="cluster-zstream-5-0-s9btnw" in resourcegroup="rg-zstream-upgrade-5-0-rpfhb4xbm8lf" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/510afec5-4713-4c20-89bb-fb8bc789e01b -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 09:46:15 UTC · failed at e2e · test Customer should be able to create a cluster and node pools with aggregated advanced features · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:208]: failed to create HCP cluster "agg-cluster" with aggregated settings Unexpected error: <fmt.wrapError \| 0xc000d84400>: failed waiting for cluster="agg-cluster" in resourcegroup="feature-aggregation-vdpv9pf7pqnl" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/68c5db19-fb03-4720-9089-dbd9b6721ef8 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:208]: failed to create HCP cluster "agg-cluster" with aggregated settings Unexpected error: <fmt.wrapError \| 0xc000d84400>: failed waiting for cluster="agg-cluster" in resourcegroup="feature-aggregation-vdpv9pf7pqnl" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/68c5db19-fb03-4720-9089-dbd9b6721ef8 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 09:46:15 UTC · failed at e2e · test Customer should upgrade and update a nodepool from 4.20.z to 4.21.zLatest · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:146]: failed to create HCP cluster np-version-upgrade-cluster-nnsgh4 with version 4.21.34 Unexpected error: <fmt.wrapError \| 0xc0006e7860>: failed to create HCP cluster np-version-upgrade-cluster-nnsgh4: failed waiting for cluster="np-version-upgrade-cluster-nnsgh4" in resourcegroup="rg-np-version-upgrade-nnsgh4-vkgvlwcxc8v6" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/2c32359b-9c7a-4507-be1f-f8838c95a6d2 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:146]: failed to create HCP cluster np-version-upgrade-cluster-nnsgh4 with version 4.21.34 Unexpected error: <fmt.wrapError \| 0xc0006e7860>: failed to create HCP cluster np-version-upgrade-cluster-nnsgh4: failed waiting for cluster="np-version-upgrade-cluster-nnsgh4" in resourcegroup="rg-np-version-upgrade-nnsgh4-vkgvlwcxc8v6" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/2c32359b-9c7a-4507-be1f-f8838c95a6d2 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 09:46:15 UTC · failed at e2e · test FIPS Mode Support with v2026-09-01 API should create an HCP cluster with FIPS mode enabled via cryptoRestrictions property · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_fips_mode.go:87]: failed to create HCP cluster "fips-enabled-cluster" with cryptoRestrictions set to FIPS Unexpected error: <fmt.wrapError \| 0xc000f60140>: failed waiting for cluster="fips-enabled-cluster" in resourcegroup="fips-enabled-nkvvdh9b6rc2" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/bcf1777c-753d-4ce5-b89a-e090b60007bf -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_fips_mode.go:87]: failed to create HCP cluster "fips-enabled-cluster" with cryptoRestrictions set to FIPS Unexpected error: <fmt.wrapError \| 0xc000f60140>: failed waiting for cluster="fips-enabled-cluster" in resourcegroup="fips-enabled-nkvvdh9b6rc2" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/bcf1777c-753d-4ce5-b89a-e090b60007bf -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 09:46:15 UTC · failed at e2e · test Customer should be able to list available HCP OpenShift versions and validate response content · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_versions.go:43]: failed to list HCP OpenShift versions Unexpected error: <exported.ResponseError \| 0xc00160d1d0>: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOpenShiftVersions -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_versions.go:43]: failed to list HCP OpenShift versions Unexpected error: <exported.ResponseError \| 0xc00160d1d0>: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOpenShiftVersions -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 09:46:15 UTC · failed at e2e · test Customer should be able to create a cluster with an external auth config and get the external auth config · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:94]: failed to create HCP cluster for external auth test Unexpected error: <fmt.wrapError \| 0xc000c28320>: failed to create HCP cluster ea-cluster: failed waiting for cluster="ea-cluster" in resourcegroup="external-auth-cluster-q8ckfq2kzzrm" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0b0c27bc-1671-49c6-b045-aa37b0328d63 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:94]: failed to create HCP cluster for external auth test Unexpected error: <fmt.wrapError \| 0xc000c28320>: failed to create HCP cluster ea-cluster: failed waiting for cluster="ea-cluster" in resourcegroup="external-auth-cluster-q8ckfq2kzzrm" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0b0c27bc-1671-49c6-b045-aa37b0328d63 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 09:46:15 UTC · failed at e2e · test Customer should be able to issue the default ingress serving certificate from Let's Encrypt using cert-manager with ACME DNS-01 challenges · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_ingress_cert_letsencrypt.go:124]: failed to create HCP cluster Unexpected error: <fmt.wrapError \| 0xc000894780>: failed to create HCP cluster ingress-cert-le: failed waiting for cluster="ingress-cert-le" in resourcegroup="ingress-cert-le-2dkfkf57f74b" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b05c995f-8ed2-412c-802f-d23c105fccb1 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_ingress_cert_letsencrypt.go:124]: failed to create HCP cluster Unexpected error: <fmt.wrapError \| 0xc000894780>: failed to create HCP cluster ingress-cert-le: failed waiting for cluster="ingress-cert-le" in resourcegroup="ingress-cert-le-2dkfkf57f74b" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b05c995f-8ed2-412c-802f-d23c105fccb1 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 09:46:15 UTC · failed at e2e · test Customer should be able to create a HCP cluster and use cilium CNI plugin · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_cni_cilium.go:88]: failed to create HCP cluster "cilium-cl" without CNI Unexpected error: <fmt.wrapError \| 0xc000af82e0>: failed to create HCP cluster cilium-cl: failed waiting for cluster="cilium-cl" in resourcegroup="cni-cilium-9pdh5npqgsvn" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/977a97de-e4df-498a-abd0-2e9b6ea6790f -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_cni_cilium.go:88]: failed to create HCP cluster "cilium-cl" without CNI Unexpected error: <fmt.wrapError \| 0xc000af82e0>: failed to create HCP cluster cilium-cl: failed waiting for cluster="cilium-cl" in resourcegroup="cni-cilium-9pdh5npqgsvn" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/977a97de-e4df-498a-abd0-2e9b6ea6790f -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 09:46:15 UTC · failed at e2e · test Customer should upgrade a nodepool to a version without Cincinnati upgrade edge z-stream upgrade without Cincinnati edge in 4.20 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:301]: failed to create HCP cluster np-noedge-59dq65 Unexpected error: <fmt.wrapError \| 0xc000ed8340>: failed to create HCP cluster np-noedge-59dq65: failed waiting for cluster="np-noedge-59dq65" in resourcegroup="rg-np-noedge-59dq65-lrcfd4c5bv9q" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/d1b1ec3e-4489-4f04-9f0c-48725a46b17b -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:301]: failed to create HCP cluster np-noedge-59dq65 Unexpected error: <fmt.wrapError \| 0xc000ed8340>: failed to create HCP cluster np-noedge-59dq65: failed waiting for cluster="np-noedge-59dq65" in resourcegroup="rg-np-noedge-59dq65-lrcfd4c5bv9q" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/d1b1ec3e-4489-4f04-9f0c-48725a46b17b -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 09:46:15 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.21 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:123]: HCP cluster rg-candidate-4-21-qqq6rbqpzr7p/cluster-candidate-4-21-crlmr8 should provision Unexpected error: <fmt.wrapError \| 0xc000b6b160>: failed to create HCP cluster cluster-candidate-4-21-crlmr8: failed waiting for cluster="cluster-candidate-4-21-crlmr8" in resourcegroup="rg-candidate-4-21-qqq6rbqpzr7p" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/471f9b00-7794-4e69-b8a4-2656470605e4 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:123]: HCP cluster rg-candidate-4-21-qqq6rbqpzr7p/cluster-candidate-4-21-crlmr8 should provision Unexpected error: <fmt.wrapError \| 0xc000b6b160>: failed to create HCP cluster cluster-candidate-4-21-crlmr8: failed waiting for cluster="cluster-candidate-4-21-crlmr8" in resourcegroup="rg-candidate-4-21-qqq6rbqpzr7p" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/471f9b00-7794-4e69-b8a4-2656470605e4 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 09:46:15 UTC · failed at e2e · test Customer should be able to create an HCP cluster then delete it by deleting the customer resource group · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delete_cx_rg.go:90]: failed to create HCP cluster "cx-rg-hcp-cluster" Unexpected error: <fmt.wrapError \| 0xc000a064a0>: failed to create HCP cluster cx-rg-hcp-cluster: failed waiting for cluster="cx-rg-hcp-cluster" in resourcegroup="cx-rg-cluster-k925b7qbr9lx" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/43d663d3-fa0d-46cb-bb16-08ea9b2ebd51 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delete_cx_rg.go:90]: failed to create HCP cluster "cx-rg-hcp-cluster" Unexpected error: <fmt.wrapError \| 0xc000a064a0>: failed to create HCP cluster cx-rg-hcp-cluster: failed waiting for cluster="cx-rg-hcp-cluster" in resourcegroup="cx-rg-cluster-k925b7qbr9lx" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/43d663d3-fa0d-46cb-bb16-08ea9b2ebd51 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 09:46:15 UTC · failed at e2e · test Customer should downgrade a nodepool version z-stream downgrade from 4.21.zLatest to 4.21.zPrevious · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:580]: failed to create HCP cluster Unexpected error: <fmt.wrapError \| 0xc001467840>: failed to create HCP cluster np-downgrade-65v5nq: failed waiting for cluster="np-downgrade-65v5nq" in resourcegroup="rg-np-downgrade-65v5nq-kdkvxpkx4x6m" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/7f93af87-9a30-4349-b34b-57c392bd5f61 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:580]: failed to create HCP cluster Unexpected error: <fmt.wrapError \| 0xc001467840>: failed to create HCP cluster np-downgrade-65v5nq: failed waiting for cluster="np-downgrade-65v5nq" in resourcegroup="rg-np-downgrade-65v5nq-kdkvxpkx4x6m" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/7f93af87-9a30-4349-b34b-57c392bd5f61 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 09:46:15 UTC · failed at e2e · test Customer should be able to create an HCP cluster and manage pull secrets · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_pullsecret.go:113]: failed to create HCP cluster for pull secret test Unexpected error: <fmt.wrapError \| 0xc000b0a4e0>: failed to create HCP cluster pullsecret-hcp-cluster: failed waiting for cluster="pullsecret-hcp-cluster" in resourcegroup="pullsecret-test-fsj64wtrrx9r" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/855b8314-4383-4b5e-b217-f13fcb1fba98 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_pullsecret.go:113]: failed to create HCP cluster for pull secret test Unexpected error: <fmt.wrapError \| 0xc000b0a4e0>: failed to create HCP cluster pullsecret-hcp-cluster: failed waiting for cluster="pullsecret-hcp-cluster" in resourcegroup="pullsecret-test-fsj64wtrrx9r" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/855b8314-4383-4b5e-b217-f13fcb1fba98 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 09:46:15 UTC · failed at e2e · test Nodepool Ephemeral OS Disk should create a nodepool with ephemeral OS disk when autoRepair is enabled · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_ephemeral_osdisk.go:84]: failed to create HCP cluster ephemeral-disk Unexpected error: <fmt.wrapError \| 0xc00076a2a0>: failed to create HCP cluster ephemeral-disk: failed waiting for cluster="ephemeral-disk" in resourcegroup="ephemeral-osdisk-tbgmd5ksgg4m" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b893d11a-1012-4cfc-b8d4-0c75c8612b93 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_ephemeral_osdisk.go:84]: failed to create HCP cluster ephemeral-disk Unexpected error: <fmt.wrapError \| 0xc00076a2a0>: failed to create HCP cluster ephemeral-disk: failed waiting for cluster="ephemeral-disk" in resourcegroup="ephemeral-osdisk-tbgmd5ksgg4m" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b893d11a-1012-4cfc-b8d4-0c75c8612b93 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 09:46:15 UTC · failed at e2e · test Customer should be able to create an HCP cluster and manage ImageDigestMirrors · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/idms_lifecycle.go:113]: failed to create HCP cluster with ImageDigestMirrors Unexpected error: <fmt.wrapError \| 0xc001017520>: failed to create HCP cluster idms-e2e-hcp-cluster: failed waiting for cluster="idms-e2e-hcp-cluster" in resourcegroup="idms-gvp7kwdrzpqg" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/cea47209-8be4-4454-b59a-b2d2d3a02020 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/idms_lifecycle.go:113]: failed to create HCP cluster with ImageDigestMirrors Unexpected error: <fmt.wrapError \| 0xc001017520>: failed to create HCP cluster idms-e2e-hcp-cluster: failed waiting for cluster="idms-e2e-hcp-cluster" in resourcegroup="idms-gvp7kwdrzpqg" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/cea47209-8be4-4454-b59a-b2d2d3a02020 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 09:46:15 UTC · failed at e2e · test Customer should be able to delete a nodepool · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_delete.go:81]: failed to create HCP cluster np-delete-hcp-cluster Unexpected error: <fmt.wrapError \| 0xc000b1e040>: failed to create HCP cluster np-delete-hcp-cluster: failed waiting for cluster="np-delete-hcp-cluster" in resourcegroup="nodepool-delete-z5kkgmdx6d7k" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/ffb337e3-ea22-456f-98e2-73505c08aba9 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_delete.go:81]: failed to create HCP cluster np-delete-hcp-cluster Unexpected error: <fmt.wrapError \| 0xc000b1e040>: failed to create HCP cluster np-delete-hcp-cluster: failed waiting for cluster="np-delete-hcp-cluster" in resourcegroup="nodepool-delete-z5kkgmdx6d7k" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/ffb337e3-ea22-456f-98e2-73505c08aba9 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 09:46:15 UTC · failed at e2e · test Customer should upgrade and update a nodepool from 4.21.z to 4.21.zLatest · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:146]: failed to create HCP cluster np-version-upgrade-cluster-2xq2jm with version 4.21.34 Unexpected error: <fmt.wrapError \| 0xc00168a2e0>: failed to create HCP cluster np-version-upgrade-cluster-2xq2jm: failed waiting for cluster="np-version-upgrade-cluster-2xq2jm" in resourcegroup="rg-np-version-upgrade-2xq2jm-f4fntnkwsc7b" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/c6a594c7-ff04-411e-8f23-7c633cfb8f90 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:146]: failed to create HCP cluster np-version-upgrade-cluster-2xq2jm with version 4.21.34 Unexpected error: <fmt.wrapError \| 0xc00168a2e0>: failed to create HCP cluster np-version-upgrade-cluster-2xq2jm: failed waiting for cluster="np-version-upgrade-cluster-2xq2jm" in resourcegroup="rg-np-version-upgrade-2xq2jm-f4fntnkwsc7b" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/c6a594c7-ff04-411e-8f23-7c633cfb8f90 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 09:46:15 UTC · failed at e2e · test Customer should be able to successfully upgrade control plane minor version from 4.22 minor to 5.0 minor · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:133]: failed to create HCP cluster "cp-ystream-upgrade-5-0-z7586h" at install version 4.22 Unexpected error: <fmt.wrapError \| 0xc000f4c380>: failed to create HCP cluster cp-ystream-upgrade-5-0-z7586h: failed waiting for cluster="cp-ystream-upgrade-5-0-z7586h" in resourcegroup="rg-cp-ystream-upgrade-5-0-njzfgn9tq5s7" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/8c645b4d-7ccc-488a-99d1-bbd87ec339bd -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:133]: failed to create HCP cluster "cp-ystream-upgrade-5-0-z7586h" at install version 4.22 Unexpected error: <fmt.wrapError \| 0xc000f4c380>: failed to create HCP cluster cp-ystream-upgrade-5-0-z7586h: failed waiting for cluster="cp-ystream-upgrade-5-0-z7586h" in resourcegroup="rg-cp-ystream-upgrade-5-0-njzfgn9tq5s7" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/8c645b4d-7ccc-488a-99d1-bbd87ec339bd -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 09:46:15 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 4.22 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:123]: failed to create HCP cluster "cluster-zstream-4-22-2f8qzb" with version 4.22.14 on candidate channel Unexpected error: <fmt.wrapError \| 0xc000c8e180>: failed to create HCP cluster cluster-zstream-4-22-2f8qzb: failed waiting for cluster="cluster-zstream-4-22-2f8qzb" in resourcegroup="rg-zstream-upgrade-4-22-gd9jz8tlz7sq" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/fe969e12-fb4e-4c38-a0fc-bf741d1193ac -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:123]: failed to create HCP cluster "cluster-zstream-4-22-2f8qzb" with version 4.22.14 on candidate channel Unexpected error: <fmt.wrapError \| 0xc000c8e180>: failed to create HCP cluster cluster-zstream-4-22-2f8qzb: failed waiting for cluster="cluster-zstream-4-22-2f8qzb" in resourcegroup="rg-zstream-upgrade-4-22-gd9jz8tlz7sq" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/fe969e12-fb4e-4c38-a0fc-bf741d1193ac -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 09:46:15 UTC · failed at e2e · test Customer should be able to create an HCP cluster with custom autoscaling and update the autoscaling configuration · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:108]: failed to create HCP cluster "autoscaling-hcp-cluster" with custom autoscaling Unexpected error: <fmt.wrapError \| 0xc00114a7a0>: failed to create HCP cluster autoscaling-hcp-cluster: failed waiting for cluster="autoscaling-hcp-cluster" in resourcegroup="autoscaling-cluster-2b22rvngvs8d" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/ba73cb16-b51f-486a-8cf3-ad7d6cb3101b -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:108]: failed to create HCP cluster "autoscaling-hcp-cluster" with custom autoscaling Unexpected error: <fmt.wrapError \| 0xc00114a7a0>: failed to create HCP cluster autoscaling-hcp-cluster: failed waiting for cluster="autoscaling-hcp-cluster" in resourcegroup="autoscaling-cluster-2b22rvngvs8d" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/ba73cb16-b51f-486a-8cf3-ad7d6cb3101b -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred
08:37:33	37m 58s	branch-ci-Azure-ARO-HCP-main-e2e-stage-e2e-parallel post-goodmerged PR No additional run metadata captured	uksouth	e2e	failed	n/a	58	e2e (58) ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 08:37:33 UTC · failed at e2e · test Customer should be able to create a HCP cluster and use cilium CNI plugin · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_cni_cilium.go:88]: failed to create HCP cluster "cilium-cl" without CNI Unexpected error: <fmt.wrapError \| 0xc0010ce1c0>: failed to create HCP cluster cilium-cl: failed waiting for cluster="cilium-cl" in resourcegroup="cni-cilium-z2x9kzpfjpcs" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0c43c069-d635-43e0-b928-4061190e3f39 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_cni_cilium.go:88]: failed to create HCP cluster "cilium-cl" without CNI Unexpected error: <fmt.wrapError \| 0xc0010ce1c0>: failed to create HCP cluster cilium-cl: failed waiting for cluster="cilium-cl" in resourcegroup="cni-cilium-z2x9kzpfjpcs" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0c43c069-d635-43e0-b928-4061190e3f39 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 08:37:33 UTC · failed at e2e · test Customer should be able to create an HCP cluster then delete it by deleting the customer resource group · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delete_cx_rg.go:90]: failed to create HCP cluster "cx-rg-hcp-cluster" Unexpected error: <fmt.wrapError \| 0xc000930ac0>: failed to create HCP cluster cx-rg-hcp-cluster: failed waiting for cluster="cx-rg-hcp-cluster" in resourcegroup="cx-rg-cluster-vgqpfgqrdt6c" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/2773dd74-f105-4eec-8543-dcfca8209086 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delete_cx_rg.go:90]: failed to create HCP cluster "cx-rg-hcp-cluster" Unexpected error: <fmt.wrapError \| 0xc000930ac0>: failed to create HCP cluster cx-rg-hcp-cluster: failed waiting for cluster="cx-rg-hcp-cluster" in resourcegroup="cx-rg-cluster-vgqpfgqrdt6c" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/2773dd74-f105-4eec-8543-dcfca8209086 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 08:37:33 UTC · failed at e2e · test Customer should be able to list available HCP OpenShift versions and validate response content · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_versions.go:43]: failed to list HCP OpenShift versions Unexpected error: <exported.ResponseError \| 0xc0009d70b0>: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOpenShiftVersions -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_versions.go:43]: failed to list HCP OpenShift versions Unexpected error: <exported.ResponseError \| 0xc0009d70b0>: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOpenShiftVersions -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 08:37:33 UTC · failed at e2e · test Customer should downgrade a nodepool to a lower minor version from 4.22.zLatest to 4.20.zLatest · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:720]: failed to create HCP cluster Unexpected error: <fmt.wrapError \| 0xc0014f2120>: failed to create HCP cluster np-dg-minor-ll5r5h: failed waiting for cluster="np-dg-minor-ll5r5h" in resourcegroup="rg-np-dg-minor-ll5r5h-kxt2jpft7wlk" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/c9e8ad1c-f40a-41ea-a448-d45cea9434f2 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:720]: failed to create HCP cluster Unexpected error: <fmt.wrapError \| 0xc0014f2120>: failed to create HCP cluster np-dg-minor-ll5r5h: failed waiting for cluster="np-dg-minor-ll5r5h" in resourcegroup="rg-np-dg-minor-ll5r5h-kxt2jpft7wlk" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/c9e8ad1c-f40a-41ea-a448-d45cea9434f2 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 08:37:33 UTC · failed at e2e · test Update HCPOpenShiftCluster Negative creates a cluster and fails to update its name with a PATCH request · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:84]: failed to create HCP cluster for patch-name test Unexpected error: <fmt.wrapError \| 0xc0009211a0>: failed to create HCP cluster patch-name-cluster: failed waiting for cluster="patch-name-cluster" in resourcegroup="patch-name-9g2pvfqcgvqk" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/08b80599-f178-48a6-8e63-9bffb24ac253 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:84]: failed to create HCP cluster for patch-name test Unexpected error: <fmt.wrapError \| 0xc0009211a0>: failed to create HCP cluster patch-name-cluster: failed waiting for cluster="patch-name-cluster" in resourcegroup="patch-name-9g2pvfqcgvqk" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/08b80599-f178-48a6-8e63-9bffb24ac253 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 08:37:33 UTC · failed at e2e · test FIPS Mode Support with v2026-09-01 API should create an HCP cluster with FIPS mode enabled via cryptoRestrictions property · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_fips_mode.go:87]: failed to create HCP cluster "fips-enabled-cluster" with cryptoRestrictions set to FIPS Unexpected error: <fmt.wrapError \| 0xc0006c6260>: failed waiting for cluster="fips-enabled-cluster" in resourcegroup="fips-enabled-lxfxstq9hdc6" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/6be43266-8deb-477a-985e-217212b3c9c3 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_fips_mode.go:87]: failed to create HCP cluster "fips-enabled-cluster" with cryptoRestrictions set to FIPS Unexpected error: <fmt.wrapError \| 0xc0006c6260>: failed waiting for cluster="fips-enabled-cluster" in resourcegroup="fips-enabled-lxfxstq9hdc6" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/6be43266-8deb-477a-985e-217212b3c9c3 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 08:37:33 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 4.20 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:123]: failed to create HCP cluster "cluster-zstream-4-20-7zwmn2" with version 4.20.38 on candidate channel Unexpected error: <fmt.wrapError \| 0xc001406160>: failed to create HCP cluster cluster-zstream-4-20-7zwmn2: failed waiting for cluster="cluster-zstream-4-20-7zwmn2" in resourcegroup="rg-zstream-upgrade-4-20-6tvfgzhnb9f7" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/dac5e483-8093-43ab-a249-0cfe987da88b -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:123]: failed to create HCP cluster "cluster-zstream-4-20-7zwmn2" with version 4.20.38 on candidate channel Unexpected error: <fmt.wrapError \| 0xc001406160>: failed to create HCP cluster cluster-zstream-4-20-7zwmn2: failed waiting for cluster="cluster-zstream-4-20-7zwmn2" in resourcegroup="rg-zstream-upgrade-4-20-6tvfgzhnb9f7" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/dac5e483-8093-43ab-a249-0cfe987da88b -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 08:37:33 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.23 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:123]: HCP cluster rg-candidate-4-23-gmqrwq5g864s/cluster-candidate-4-23-hvdmh7 should provision Unexpected error: <fmt.wrapError \| 0xc000b92ae0>: failed to create HCP cluster cluster-candidate-4-23-hvdmh7: failed waiting for cluster="cluster-candidate-4-23-hvdmh7" in resourcegroup="rg-candidate-4-23-gmqrwq5g864s" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/68096478-e4e8-46af-b3df-410513444ada -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:123]: HCP cluster rg-candidate-4-23-gmqrwq5g864s/cluster-candidate-4-23-hvdmh7 should provision Unexpected error: <fmt.wrapError \| 0xc000b92ae0>: failed to create HCP cluster cluster-candidate-4-23-hvdmh7: failed waiting for cluster="cluster-candidate-4-23-hvdmh7" in resourcegroup="rg-candidate-4-23-gmqrwq5g864s" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/68096478-e4e8-46af-b3df-410513444ada -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 08:37:33 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 4.21 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:123]: failed to create HCP cluster "cluster-zstream-4-21-wx86j9" with version 4.21.33 on candidate channel Unexpected error: <fmt.wrapError \| 0xc000f88240>: failed to create HCP cluster cluster-zstream-4-21-wx86j9: failed waiting for cluster="cluster-zstream-4-21-wx86j9" in resourcegroup="rg-zstream-upgrade-4-21-84nwx9kjvq9s" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/24167e87-3944-4069-af0b-84a754828b65 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:123]: failed to create HCP cluster "cluster-zstream-4-21-wx86j9" with version 4.21.33 on candidate channel Unexpected error: <fmt.wrapError \| 0xc000f88240>: failed to create HCP cluster cluster-zstream-4-21-wx86j9: failed waiting for cluster="cluster-zstream-4-21-wx86j9" in resourcegroup="rg-zstream-upgrade-4-21-84nwx9kjvq9s" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/24167e87-3944-4069-af0b-84a754828b65 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 08:37:33 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 4.22 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:123]: failed to create HCP cluster "cluster-zstream-4-22-htjlq8" with version 4.22.14 on candidate channel Unexpected error: <fmt.wrapError \| 0xc001654420>: failed to create HCP cluster cluster-zstream-4-22-htjlq8: failed waiting for cluster="cluster-zstream-4-22-htjlq8" in resourcegroup="rg-zstream-upgrade-4-22-9qbd8bqls6jc" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0e4a0cf4-07d6-461f-a890-6f064bd6c922 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:123]: failed to create HCP cluster "cluster-zstream-4-22-htjlq8" with version 4.22.14 on candidate channel Unexpected error: <fmt.wrapError \| 0xc001654420>: failed to create HCP cluster cluster-zstream-4-22-htjlq8: failed waiting for cluster="cluster-zstream-4-22-htjlq8" in resourcegroup="rg-zstream-upgrade-4-22-9qbd8bqls6jc" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0e4a0cf4-07d6-461f-a890-6f064bd6c922 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 08:37:33 UTC · failed at e2e · test Customer should be able to update node pool labels and taints · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_labels_taints.go:92]: failed to create HCP cluster cluster-np-labels-taints Unexpected error: <fmt.wrapError \| 0xc000a32800>: failed to create HCP cluster cluster-np-labels-taints: failed waiting for cluster="cluster-np-labels-taints" in resourcegroup="rg-np-labels-taints-dfxwwmw5pfzj" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/1fb11f91-985b-4696-85d4-b6884c2c3515 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_labels_taints.go:92]: failed to create HCP cluster cluster-np-labels-taints Unexpected error: <fmt.wrapError \| 0xc000a32800>: failed to create HCP cluster cluster-np-labels-taints: failed waiting for cluster="cluster-np-labels-taints" in resourcegroup="rg-np-labels-taints-dfxwwmw5pfzj" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/1fb11f91-985b-4696-85d4-b6884c2c3515 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 08:37:33 UTC · failed at e2e · test Customer should be able to forward control plane logs to a storage account and Event Hub via shoebox diagnostic settings · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/shoebox_logs.go:272]: failed to create HCP cluster shoebox-hcp-cluster Unexpected error: <fmt.wrapError \| 0xc0006c2cc0>: failed to create HCP cluster shoebox-hcp-cluster: failed waiting for cluster="shoebox-hcp-cluster" in resourcegroup="shoebox-cluster-bthlczlxh8pn" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/74364df9-e093-4ce7-8de4-4462f8abffb3 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/shoebox_logs.go:272]: failed to create HCP cluster shoebox-hcp-cluster Unexpected error: <fmt.wrapError \| 0xc0006c2cc0>: failed to create HCP cluster shoebox-hcp-cluster: failed waiting for cluster="shoebox-hcp-cluster" in resourcegroup="shoebox-cluster-bthlczlxh8pn" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/74364df9-e093-4ce7-8de4-4462f8abffb3 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 08:37:33 UTC · failed at e2e · test Customer should downgrade a nodepool version z-stream downgrade from 4.21.zLatest to 4.21.zPrevious · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:580]: failed to create HCP cluster Unexpected error: <fmt.wrapError \| 0xc000b0e2e0>: failed to create HCP cluster np-downgrade-mf92ds: failed waiting for cluster="np-downgrade-mf92ds" in resourcegroup="rg-np-downgrade-mf92ds-74h5g5s57m94" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/60adbefa-dfde-42a1-ab60-90a912d72889 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:580]: failed to create HCP cluster Unexpected error: <fmt.wrapError \| 0xc000b0e2e0>: failed to create HCP cluster np-downgrade-mf92ds: failed waiting for cluster="np-downgrade-mf92ds" in resourcegroup="rg-np-downgrade-mf92ds-74h5g5s57m94" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/60adbefa-dfde-42a1-ab60-90a912d72889 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 08:37:33 UTC · failed at e2e · test Customer should be able to create a cluster with default autoscaling and a nodepool with autoscaling enabled up to replica limits · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:102]: failed to create HCP cluster np-autoscale-cluster Unexpected error: <fmt.wrapError \| 0xc0026e43e0>: failed to create HCP cluster np-autoscale-cluster: failed waiting for cluster="np-autoscale-cluster" in resourcegroup="np-autoscaling-4fs4j62vswlx" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/c7c82017-f2c4-4311-ac0c-894ea597df58 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:102]: failed to create HCP cluster np-autoscale-cluster Unexpected error: <fmt.wrapError \| 0xc0026e43e0>: failed to create HCP cluster np-autoscale-cluster: failed waiting for cluster="np-autoscale-cluster" in resourcegroup="np-autoscaling-4fs4j62vswlx" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/c7c82017-f2c4-4311-ac0c-894ea597df58 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 08:37:33 UTC · failed at e2e · test Customer should be able to successfully upgrade control plane minor version from 4.22 minor to 5.0 minor · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:133]: failed to create HCP cluster "cp-ystream-upgrade-5-0-bjzsqc" at install version 4.22 Unexpected error: <fmt.wrapError \| 0xc00165c1a0>: failed to create HCP cluster cp-ystream-upgrade-5-0-bjzsqc: failed waiting for cluster="cp-ystream-upgrade-5-0-bjzsqc" in resourcegroup="rg-cp-ystream-upgrade-5-0-scvm2wn84kfb" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/86c9a4c9-79c6-426c-b678-3f5b50e1372f -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:133]: failed to create HCP cluster "cp-ystream-upgrade-5-0-bjzsqc" at install version 4.22 Unexpected error: <fmt.wrapError \| 0xc00165c1a0>: failed to create HCP cluster cp-ystream-upgrade-5-0-bjzsqc: failed waiting for cluster="cp-ystream-upgrade-5-0-bjzsqc" in resourcegroup="rg-cp-ystream-upgrade-5-0-scvm2wn84kfb" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/86c9a4c9-79c6-426c-b678-3f5b50e1372f -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 08:37:33 UTC · failed at e2e · test ARO HCP Service should successfully provision cluster when MI role assignments are added after cluster creation · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:238]: failed to get admin REST config for HCP cluster Unexpected error: <fmt.wrapError \| 0xc000c00720>: failed waiting for hcpCluster="delayed-rbac-cluster" in resourcegroup="delayed-rbac-2vgmjks25m4m" to finish getting creds: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/8b769151-6e53-4fdc-879e-257a7ef5809d -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:238]: failed to get admin REST config for HCP cluster Unexpected error: <fmt.wrapError \| 0xc000c00720>: failed waiting for hcpCluster="delayed-rbac-cluster" in resourcegroup="delayed-rbac-2vgmjks25m4m" to finish getting creds: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/8b769151-6e53-4fdc-879e-257a7ef5809d -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 08:37:33 UTC · failed at e2e · test Customer should be able to test admin credentials before cluster ready, then full admin credential lifecycle · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:190]: failed to request admin credential 1 Unexpected error: <fmt.wrapError \| 0xc00073dde0>: failed waiting for hcpCluster="admin-cred-lifecycle-8lf47p" in resourcegroup="admin-credential-lifecycle-test-2rs7chhwpc4n" to finish getting creds: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/43c02385-5fa8-4618-85da-cf7b4f4d1da6 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:190]: failed to request admin credential 1 Unexpected error: <fmt.wrapError \| 0xc00073dde0>: failed waiting for hcpCluster="admin-cred-lifecycle-8lf47p" in resourcegroup="admin-credential-lifecycle-test-2rs7chhwpc4n" to finish getting creds: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/43c02385-5fa8-4618-85da-cf7b4f4d1da6 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 08:37:33 UTC · failed at e2e · test Customer should be able to create node pools with ARM64-based VMs · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/arm64_nodepool.go:88]: failed to create HCP cluster "arm64-vm-hcp-cluster" Unexpected error: <fmt.wrapError \| 0xc0010ae120>: failed to create HCP cluster arm64-vm-hcp-cluster: failed waiting for cluster="arm64-vm-hcp-cluster" in resourcegroup="arm64-vm-cluster-rm2gp8bzthz4" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/3fca39ef-cfc8-43b3-9b0f-46e3f6b5f13d -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/arm64_nodepool.go:88]: failed to create HCP cluster "arm64-vm-hcp-cluster" Unexpected error: <fmt.wrapError \| 0xc0010ae120>: failed to create HCP cluster arm64-vm-hcp-cluster: failed waiting for cluster="arm64-vm-hcp-cluster" in resourcegroup="arm64-vm-cluster-rm2gp8bzthz4" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/3fca39ef-cfc8-43b3-9b0f-46e3f6b5f13d -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 08:37:33 UTC · failed at e2e · test Customer should upgrade and update a nodepool from 4.21.z to 4.21.zLatest · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:146]: failed to create HCP cluster np-version-upgrade-cluster-bmxs6w with version 4.21.34 Unexpected error: <fmt.wrapError \| 0xc00052f260>: failed to create HCP cluster np-version-upgrade-cluster-bmxs6w: failed waiting for cluster="np-version-upgrade-cluster-bmxs6w" in resourcegroup="rg-np-version-upgrade-bmxs6w-2hrwrsfnjsgz" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/76f0a7b5-18a2-4186-9264-5605e6b3ba75 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:146]: failed to create HCP cluster np-version-upgrade-cluster-bmxs6w with version 4.21.34 Unexpected error: <fmt.wrapError \| 0xc00052f260>: failed to create HCP cluster np-version-upgrade-cluster-bmxs6w: failed waiting for cluster="np-version-upgrade-cluster-bmxs6w" in resourcegroup="rg-np-version-upgrade-bmxs6w-2hrwrsfnjsgz" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/76f0a7b5-18a2-4186-9264-5605e6b3ba75 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 08:37:33 UTC · failed at e2e · test Customer should be able to issue the default ingress serving certificate from Let's Encrypt using cert-manager with ACME DNS-01 challenges · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_ingress_cert_letsencrypt.go:124]: failed to create HCP cluster Unexpected error: <fmt.wrapError \| 0xc0010f61c0>: failed to create HCP cluster ingress-cert-le: failed waiting for cluster="ingress-cert-le" in resourcegroup="ingress-cert-le-2f5s4s5292qz" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/43bed0b7-637d-4f9e-a76c-dd1c0759f849 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_ingress_cert_letsencrypt.go:124]: failed to create HCP cluster Unexpected error: <fmt.wrapError \| 0xc0010f61c0>: failed to create HCP cluster ingress-cert-le: failed waiting for cluster="ingress-cert-le" in resourcegroup="ingress-cert-le-2f5s4s5292qz" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/43bed0b7-637d-4f9e-a76c-dd1c0759f849 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 08:37:33 UTC · failed at e2e · test Customer should be able to create several HCP clusters in their customer resource group, but not in the same managed resource group · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:131]: failed to wait for first cluster "basic-hcp-cluster" to complete creation (timeout '30.000000' minutes) Unexpected error: <exported.ResponseError \| 0xc0017f2570>: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/fdb8d5ee-e534-4c8a-bab8-e124a2281894 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:131]: failed to wait for first cluster "basic-hcp-cluster" to complete creation (timeout '30.000000' minutes) Unexpected error: <exported.ResponseError \| 0xc0017f2570>: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/fdb8d5ee-e534-4c8a-bab8-e124a2281894 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 08:37:33 UTC · failed at e2e · test Customer should be able to create an HCP cluster and custom node pool osDisk size · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_nodepool_osdisk.go:80]: failed to create HCP cluster "hcp-cluster-np-128" Unexpected error: <fmt.wrapError \| 0xc0010ea200>: failed to create HCP cluster hcp-cluster-np-128: failed waiting for cluster="hcp-cluster-np-128" in resourcegroup="clusternp128-2mz766hrmsh4" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/a7d6ebbd-21d5-4168-b9ef-6769c57a7383 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_nodepool_osdisk.go:80]: failed to create HCP cluster "hcp-cluster-np-128" Unexpected error: <fmt.wrapError \| 0xc0010ea200>: failed to create HCP cluster hcp-cluster-np-128: failed waiting for cluster="hcp-cluster-np-128" in resourcegroup="clusternp128-2mz766hrmsh4" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/a7d6ebbd-21d5-4168-b9ef-6769c57a7383 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 08:37:33 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 5.0 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:123]: failed to create HCP cluster "cluster-zstream-5-0-lpjzpv" with version 5.0.0-rc.2 on candidate channel Unexpected error: <fmt.wrapError \| 0xc001122160>: failed to create HCP cluster cluster-zstream-5-0-lpjzpv: failed waiting for cluster="cluster-zstream-5-0-lpjzpv" in resourcegroup="rg-zstream-upgrade-5-0-gcnzx6nn895z" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/3356f23a-d69a-4d18-a2fa-975aa1dba2ae -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:123]: failed to create HCP cluster "cluster-zstream-5-0-lpjzpv" with version 5.0.0-rc.2 on candidate channel Unexpected error: <fmt.wrapError \| 0xc001122160>: failed to create HCP cluster cluster-zstream-5-0-lpjzpv: failed waiting for cluster="cluster-zstream-5-0-lpjzpv" in resourcegroup="rg-zstream-upgrade-5-0-gcnzx6nn895z" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/3356f23a-d69a-4d18-a2fa-975aa1dba2ae -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 08:37:33 UTC · failed at e2e · test Nodepool Ephemeral OS Disk should create a nodepool with ephemeral OS disk when autoRepair is enabled · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_ephemeral_osdisk.go:84]: failed to create HCP cluster ephemeral-disk Unexpected error: <fmt.wrapError \| 0xc000e98480>: failed to create HCP cluster ephemeral-disk: failed waiting for cluster="ephemeral-disk" in resourcegroup="ephemeral-osdisk-pf8nhhfwdldv" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/ac18bad4-a3a9-4bae-b590-c11b5fae6e54 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_ephemeral_osdisk.go:84]: failed to create HCP cluster ephemeral-disk Unexpected error: <fmt.wrapError \| 0xc000e98480>: failed to create HCP cluster ephemeral-disk: failed waiting for cluster="ephemeral-disk" in resourcegroup="ephemeral-osdisk-pf8nhhfwdldv" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/ac18bad4-a3a9-4bae-b590-c11b5fae6e54 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 08:37:33 UTC · failed at e2e · test Customer should be able to use workload identity via the cluster OIDC issuer URL · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/oidc_issuer_workload_identity.go:227]: failed to create HCP cluster oidc-wi-cluster Unexpected error: <fmt.wrapError \| 0xc00159e140>: failed to create HCP cluster oidc-wi-cluster: failed waiting for cluster="oidc-wi-cluster" in resourcegroup="oidc-wi-5mpvwz2274ds" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/dba4e3ec-d50c-4db8-a47e-128c7f0b8594 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/oidc_issuer_workload_identity.go:227]: failed to create HCP cluster oidc-wi-cluster Unexpected error: <fmt.wrapError \| 0xc00159e140>: failed to create HCP cluster oidc-wi-cluster: failed waiting for cluster="oidc-wi-cluster" in resourcegroup="oidc-wi-5mpvwz2274ds" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/dba4e3ec-d50c-4db8-a47e-128c7f0b8594 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 08:37:33 UTC · failed at e2e · test Customer should be able to cordon, drain, and uncordon a node in an HCP cluster · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_cordon.go:90]: failed to create HCP cluster cordon-hcp-cluster Unexpected error: <fmt.wrapError \| 0xc0008c3960>: failed to create HCP cluster cordon-hcp-cluster: failed waiting for cluster="cordon-hcp-cluster" in resourcegroup="rg-cordon-test-9bd2bz6h9p9p" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/df51608a-347c-4931-9425-91b0f221a8ed -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_cordon.go:90]: failed to create HCP cluster cordon-hcp-cluster Unexpected error: <fmt.wrapError \| 0xc0008c3960>: failed to create HCP cluster cordon-hcp-cluster: failed waiting for cluster="cordon-hcp-cluster" in resourcegroup="rg-cordon-test-9bd2bz6h9p9p" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/df51608a-347c-4931-9425-91b0f221a8ed -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 08:37:33 UTC · failed at e2e · test Authorized CIDRs Connectivity should allow API access only from authorized VM IP · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:111]: failed to create HCP cluster "cidr-connectivity-test" with authorized CIDRs Unexpected error: <fmt.wrapError \| 0xc002b1c3a0>: failed to create HCP cluster cidr-connectivity-test: failed waiting for cluster="cidr-connectivity-test" in resourcegroup="e2e-cidr-connectivity-jnp7zdxxg8cw" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/1e8153a6-1e58-434a-8ce9-d05c5a075d11 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:111]: failed to create HCP cluster "cidr-connectivity-test" with authorized CIDRs Unexpected error: <fmt.wrapError \| 0xc002b1c3a0>: failed to create HCP cluster cidr-connectivity-test: failed waiting for cluster="cidr-connectivity-test" in resourcegroup="e2e-cidr-connectivity-jnp7zdxxg8cw" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/1e8153a6-1e58-434a-8ce9-d05c5a075d11 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 08:37:33 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.21 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:123]: HCP cluster rg-candidate-4-21-dvvd9lz9qfrf/cluster-candidate-4-21-2ncnls should provision Unexpected error: <fmt.wrapError \| 0xc000a4e180>: failed to create HCP cluster cluster-candidate-4-21-2ncnls: failed waiting for cluster="cluster-candidate-4-21-2ncnls" in resourcegroup="rg-candidate-4-21-dvvd9lz9qfrf" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/5fd73cfc-f003-451a-83bf-31b74bcd067d -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:123]: HCP cluster rg-candidate-4-21-dvvd9lz9qfrf/cluster-candidate-4-21-2ncnls should provision Unexpected error: <fmt.wrapError \| 0xc000a4e180>: failed to create HCP cluster cluster-candidate-4-21-2ncnls: failed waiting for cluster="cluster-candidate-4-21-2ncnls" in resourcegroup="rg-candidate-4-21-dvvd9lz9qfrf" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/5fd73cfc-f003-451a-83bf-31b74bcd067d -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 08:37:33 UTC · failed at e2e · test Customer should be able to create a cluster and node pools with aggregated advanced features · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:208]: failed to create HCP cluster "agg-cluster" with aggregated settings Unexpected error: <fmt.wrapError \| 0xc000802d80>: failed waiting for cluster="agg-cluster" in resourcegroup="feature-aggregation-z2h4x96fwvfv" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/46f2b135-92d2-48b7-b2e1-ed95e6c6a6d2 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:208]: failed to create HCP cluster "agg-cluster" with aggregated settings Unexpected error: <fmt.wrapError \| 0xc000802d80>: failed waiting for cluster="agg-cluster" in resourcegroup="feature-aggregation-z2h4x96fwvfv" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/46f2b135-92d2-48b7-b2e1-ed95e6c6a6d2 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 08:37:33 UTC · failed at e2e · test Customer should be able to successfully upgrade control plane minor version from 4.20 minor to 4.21 minor · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:133]: failed to create HCP cluster "cp-ystream-upgrade-4-21-jszlc6" at install version 4.20 Unexpected error: <fmt.wrapError \| 0xc00180a340>: failed to create HCP cluster cp-ystream-upgrade-4-21-jszlc6: failed waiting for cluster="cp-ystream-upgrade-4-21-jszlc6" in resourcegroup="rg-cp-ystream-upgrade-4-21-9c5wxsfjpwnk" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/48384837-d606-4bf7-adb9-e5ebfc01f203 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:133]: failed to create HCP cluster "cp-ystream-upgrade-4-21-jszlc6" at install version 4.20 Unexpected error: <fmt.wrapError \| 0xc00180a340>: failed to create HCP cluster cp-ystream-upgrade-4-21-jszlc6: failed waiting for cluster="cp-ystream-upgrade-4-21-jszlc6" in resourcegroup="rg-cp-ystream-upgrade-4-21-9c5wxsfjpwnk" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/48384837-d606-4bf7-adb9-e5ebfc01f203 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 08:37:33 UTC · failed at e2e · test Customer should respect cluster-wide node limits with nodepool autoscaling · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:291]: failed to create HCP cluster node-limit-cluster with MaxNodesTotal limit Unexpected error: <fmt.wrapError \| 0xc000c4c940>: failed to create HCP cluster node-limit-cluster: failed waiting for cluster="node-limit-cluster" in resourcegroup="node-limit-test-94jgsqlwdmnq" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/ddf1ab16-458d-4415-a7f9-065212e41e1e -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:291]: failed to create HCP cluster node-limit-cluster with MaxNodesTotal limit Unexpected error: <fmt.wrapError \| 0xc000c4c940>: failed to create HCP cluster node-limit-cluster: failed waiting for cluster="node-limit-cluster" in resourcegroup="node-limit-test-94jgsqlwdmnq" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/ddf1ab16-458d-4415-a7f9-065212e41e1e -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 08:37:33 UTC · failed at e2e · test Customer should be able to create an HCP cluster with Image Registry not present · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/image_registry_cluster_create.go:89]: failed to create HCP cluster with image registry disabled Unexpected error: <fmt.wrapError \| 0xc0011ea140>: failed to create HCP cluster disabled-image-registry-hcp-cluster: failed waiting for cluster="disabled-image-registry-hcp-cluster" in resourcegroup="disabled-image-registry-fbvzdt8brrln" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/04444596-313d-4dc6-b1bb-e85239ee352a -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/image_registry_cluster_create.go:89]: failed to create HCP cluster with image registry disabled Unexpected error: <fmt.wrapError \| 0xc0011ea140>: failed to create HCP cluster disabled-image-registry-hcp-cluster: failed waiting for cluster="disabled-image-registry-hcp-cluster" in resourcegroup="disabled-image-registry-fbvzdt8brrln" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/04444596-313d-4dc6-b1bb-e85239ee352a -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 08:37:33 UTC · failed at e2e · test Customer should upgrade and update a nodepool from 4.20.z to 4.20.zLatest · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:146]: failed to create HCP cluster np-version-upgrade-cluster-fj4pb7 with version 4.20.39 Unexpected error: <fmt.wrapError \| 0xc001042700>: failed to create HCP cluster np-version-upgrade-cluster-fj4pb7: failed waiting for cluster="np-version-upgrade-cluster-fj4pb7" in resourcegroup="rg-np-version-upgrade-fj4pb7-qmmpv2hmtwpx" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/316360e5-e07a-44b2-aa4c-7e7deed05703 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:146]: failed to create HCP cluster np-version-upgrade-cluster-fj4pb7 with version 4.20.39 Unexpected error: <fmt.wrapError \| 0xc001042700>: failed to create HCP cluster np-version-upgrade-cluster-fj4pb7: failed waiting for cluster="np-version-upgrade-cluster-fj4pb7" in resourcegroup="rg-np-version-upgrade-fj4pb7-qmmpv2hmtwpx" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/316360e5-e07a-44b2-aa4c-7e7deed05703 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 08:37:33 UTC · failed at e2e · test Customer should create an HCP cluster and validate TLS certificates · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:96]: failed to create HCP cluster for TLS endpoint test Unexpected error: <fmt.wrapError \| 0xc0011a4380>: failed to create HCP cluster tls-endpoint-hcp-cluster: failed waiting for cluster="tls-endpoint-hcp-cluster" in resourcegroup="tls-endpoint-cluster-mgqjjwc44whb" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/ee9b81a3-da8d-4906-9d58-b1a3eca2efb8 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:96]: failed to create HCP cluster for TLS endpoint test Unexpected error: <fmt.wrapError \| 0xc0011a4380>: failed to create HCP cluster tls-endpoint-hcp-cluster: failed waiting for cluster="tls-endpoint-hcp-cluster" in resourcegroup="tls-endpoint-cluster-mgqjjwc44whb" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/ee9b81a3-da8d-4906-9d58-b1a3eca2efb8 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 08:37:33 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.22 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:123]: HCP cluster rg-candidate-4-22-hxj8jzjsj4wx/cluster-candidate-4-22-f2c7ls should provision Unexpected error: <fmt.wrapError \| 0xc000c955a0>: failed to create HCP cluster cluster-candidate-4-22-f2c7ls: failed waiting for cluster="cluster-candidate-4-22-f2c7ls" in resourcegroup="rg-candidate-4-22-hxj8jzjsj4wx" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/f8e9c5fd-c381-42f0-b97f-a20bab0ee86e -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:123]: HCP cluster rg-candidate-4-22-hxj8jzjsj4wx/cluster-candidate-4-22-f2c7ls should provision Unexpected error: <fmt.wrapError \| 0xc000c955a0>: failed to create HCP cluster cluster-candidate-4-22-f2c7ls: failed waiting for cluster="cluster-candidate-4-22-f2c7ls" in resourcegroup="rg-candidate-4-22-hxj8jzjsj4wx" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/f8e9c5fd-c381-42f0-b97f-a20bab0ee86e -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 08:37:33 UTC · failed at e2e · test Customer should upgrade a nodepool to a version without Cincinnati upgrade edge z-stream upgrade without Cincinnati edge in 4.20 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:301]: failed to create HCP cluster np-noedge-7mrwf2 Unexpected error: <fmt.wrapError \| 0xc000d02460>: failed to create HCP cluster np-noedge-7mrwf2: failed waiting for cluster="np-noedge-7mrwf2" in resourcegroup="rg-np-noedge-7mrwf2-w9l84zwng9gf" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0e081b02-f88b-42de-ba62-6fadaf13df49 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:301]: failed to create HCP cluster np-noedge-7mrwf2 Unexpected error: <fmt.wrapError \| 0xc000d02460>: failed to create HCP cluster np-noedge-7mrwf2: failed waiting for cluster="np-noedge-7mrwf2" in resourcegroup="rg-np-noedge-7mrwf2-w9l84zwng9gf" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0e081b02-f88b-42de-ba62-6fadaf13df49 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 08:37:33 UTC · failed at e2e · test Customer should be able to delete an HCP cluster whose managed identities were deleted first · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delete_missing_identities.go:113]: failed to create HCP cluster "missing-mi-hcp-cluster" Unexpected error: <fmt.wrapError \| 0xc00067aae0>: failed to create HCP cluster missing-mi-hcp-cluster: failed waiting for cluster="missing-mi-hcp-cluster" in resourcegroup="delete-mi-rg-wzq8hbkhk8st" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/20bc14f5-a9ed-4d5d-ae0e-e61acc5df8a3 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delete_missing_identities.go:113]: failed to create HCP cluster "missing-mi-hcp-cluster" Unexpected error: <fmt.wrapError \| 0xc00067aae0>: failed to create HCP cluster missing-mi-hcp-cluster: failed waiting for cluster="missing-mi-hcp-cluster" in resourcegroup="delete-mi-rg-wzq8hbkhk8st" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/20bc14f5-a9ed-4d5d-ae0e-e61acc5df8a3 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 08:37:33 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.20 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:123]: HCP cluster rg-candidate-4-20-9hqllf7rhbnn/cluster-candidate-4-20-69x5dp should provision Unexpected error: <fmt.wrapError \| 0xc0009a0700>: failed to create HCP cluster cluster-candidate-4-20-69x5dp: failed waiting for cluster="cluster-candidate-4-20-69x5dp" in resourcegroup="rg-candidate-4-20-9hqllf7rhbnn" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/5614ff0a-a8db-40c4-8d86-43931a0311b8 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:123]: HCP cluster rg-candidate-4-20-9hqllf7rhbnn/cluster-candidate-4-20-69x5dp should provision Unexpected error: <fmt.wrapError \| 0xc0009a0700>: failed to create HCP cluster cluster-candidate-4-20-69x5dp: failed waiting for cluster="cluster-candidate-4-20-69x5dp" in resourcegroup="rg-candidate-4-20-9hqllf7rhbnn" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/5614ff0a-a8db-40c4-8d86-43931a0311b8 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 08:37:33 UTC · failed at e2e · test Customer should be able to create a no-CNI private cluster with a private key vault, a nodepool and install cilium CNI successfully · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:107]: failed to create HCP cluster "cilium-cluster" with no CNI and private etcd Unexpected error: <fmt.wrapError \| 0xc000c60960>: failed waiting for cluster="cilium-cluster" in resourcegroup="complex-cilium-kv-gxgjqjvxxxj6" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/8ff8a326-d85b-4588-8d64-4668f5c77da2 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:107]: failed to create HCP cluster "cilium-cluster" with no CNI and private etcd Unexpected error: <fmt.wrapError \| 0xc000c60960>: failed waiting for cluster="cilium-cluster" in resourcegroup="complex-cilium-kv-gxgjqjvxxxj6" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/8ff8a326-d85b-4588-8d64-4668f5c77da2 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 08:37:33 UTC · failed at e2e · test Customer should be able to create an HCP cluster and manage pull secrets · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_pullsecret.go:113]: failed to create HCP cluster for pull secret test Unexpected error: <fmt.wrapError \| 0xc0007fa460>: failed to create HCP cluster pullsecret-hcp-cluster: failed waiting for cluster="pullsecret-hcp-cluster" in resourcegroup="pullsecret-test-dhzt66pct8mh" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/192cb9af-f4dd-4dfb-b72d-621de0f96c43 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_pullsecret.go:113]: failed to create HCP cluster for pull secret test Unexpected error: <fmt.wrapError \| 0xc0007fa460>: failed to create HCP cluster pullsecret-hcp-cluster: failed waiting for cluster="pullsecret-hcp-cluster" in resourcegroup="pullsecret-test-dhzt66pct8mh" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/192cb9af-f4dd-4dfb-b72d-621de0f96c43 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 08:37:33 UTC · failed at e2e · test Customer should be able to successfully upgrade control plane minor version from 4.22 minor to 4.23 minor · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:133]: failed to create HCP cluster "cp-ystream-upgrade-4-23-mh5q44" at install version 4.22 Unexpected error: <fmt.wrapError \| 0xc0009fa4e0>: failed to create HCP cluster cp-ystream-upgrade-4-23-mh5q44: failed waiting for cluster="cp-ystream-upgrade-4-23-mh5q44" in resourcegroup="rg-cp-ystream-upgrade-4-23-dzbbg7dkxq5q" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/c60713bb-3780-4f3f-b848-5be7053f3663 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:133]: failed to create HCP cluster "cp-ystream-upgrade-4-23-mh5q44" at install version 4.22 Unexpected error: <fmt.wrapError \| 0xc0009fa4e0>: failed to create HCP cluster cp-ystream-upgrade-4-23-mh5q44: failed waiting for cluster="cp-ystream-upgrade-4-23-mh5q44" in resourcegroup="rg-cp-ystream-upgrade-4-23-dzbbg7dkxq5q" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/c60713bb-3780-4f3f-b848-5be7053f3663 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 08:37:33 UTC · failed at e2e · test Update HCPOpenShiftCluster Positive creates a cluster and updates tags with a PATCH request · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:161]: failed to create HCP cluster for patch-tags test Unexpected error: <fmt.wrapError \| 0xc000c12aa0>: failed to create HCP cluster patch-tags-cluster: failed waiting for cluster="patch-tags-cluster" in resourcegroup="patch-tags-vhrfcsbl2kmh" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/7eaf31f9-fff5-41e2-ab4e-b85d8caa782e -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:161]: failed to create HCP cluster for patch-tags test Unexpected error: <fmt.wrapError \| 0xc000c12aa0>: failed to create HCP cluster patch-tags-cluster: failed waiting for cluster="patch-tags-cluster" in resourcegroup="patch-tags-vhrfcsbl2kmh" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/7eaf31f9-fff5-41e2-ab4e-b85d8caa782e -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 08:37:33 UTC · failed at e2e · test Customer should be able to create an HCP cluster and manage ImageDigestMirrors · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/idms_lifecycle.go:113]: failed to create HCP cluster with ImageDigestMirrors Unexpected error: <fmt.wrapError \| 0xc000ddc4a0>: failed to create HCP cluster idms-e2e-hcp-cluster: failed waiting for cluster="idms-e2e-hcp-cluster" in resourcegroup="idms-tb9jrz2hwj8c" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b3601a9e-bdb8-49f6-a169-d2bab0ef2a88 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/idms_lifecycle.go:113]: failed to create HCP cluster with ImageDigestMirrors Unexpected error: <fmt.wrapError \| 0xc000ddc4a0>: failed to create HCP cluster idms-e2e-hcp-cluster: failed waiting for cluster="idms-e2e-hcp-cluster" in resourcegroup="idms-tb9jrz2hwj8c" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b3601a9e-bdb8-49f6-a169-d2bab0ef2a88 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 08:37:33 UTC · failed at e2e · test Customer should not be able to perform various invalid operations on cluster resources · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/simple_negative_cases.go:110]: failed to create HCP cluster negative-tests-cluster Unexpected error: <fmt.wrapError \| 0xc000a88200>: failed to create HCP cluster negative-tests-cluster: failed waiting for cluster="negative-tests-cluster" in resourcegroup="rg-negative-tests-p8j4tmlkfz9s" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/323e35c2-8229-47c9-9ac2-6f239975f95c -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/simple_negative_cases.go:110]: failed to create HCP cluster negative-tests-cluster Unexpected error: <fmt.wrapError \| 0xc000a88200>: failed to create HCP cluster negative-tests-cluster: failed waiting for cluster="negative-tests-cluster" in resourcegroup="rg-negative-tests-p8j4tmlkfz9s" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/323e35c2-8229-47c9-9ac2-6f239975f95c -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 08:37:33 UTC · failed at e2e · test Customer should be able to create an HCP cluster with back-level version 4.20 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:152]: failed to create HCP cluster version 4.20 Unexpected error: <fmt.wrapError \| 0xc000a96320>: failed waiting for cluster="cluster-ver-4-20" in resourcegroup="rg-cluster-back-version-rpjktqwsql26" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/74e939b7-1005-49c4-8393-bc500b6f83a5 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:152]: failed to create HCP cluster version 4.20 Unexpected error: <fmt.wrapError \| 0xc000a96320>: failed waiting for cluster="cluster-ver-4-20" in resourcegroup="rg-cluster-back-version-rpjktqwsql26" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/74e939b7-1005-49c4-8393-bc500b6f83a5 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 08:37:33 UTC · failed at e2e · test Customer should be able to create an HCP cluster with custom autoscaling and update the autoscaling configuration · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:108]: failed to create HCP cluster "autoscaling-hcp-cluster" with custom autoscaling Unexpected error: <fmt.wrapError \| 0xc000a805e0>: failed to create HCP cluster autoscaling-hcp-cluster: failed waiting for cluster="autoscaling-hcp-cluster" in resourcegroup="autoscaling-cluster-bw84xtccbhp5" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/220ee6e2-189a-404f-9a4c-ac05385f929c -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:108]: failed to create HCP cluster "autoscaling-hcp-cluster" with custom autoscaling Unexpected error: <fmt.wrapError \| 0xc000a805e0>: failed to create HCP cluster autoscaling-hcp-cluster: failed waiting for cluster="autoscaling-hcp-cluster" in resourcegroup="autoscaling-cluster-bw84xtccbhp5" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/220ee6e2-189a-404f-9a4c-ac05385f929c -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 08:37:33 UTC · failed at e2e · test Customer should be able to successfully upgrade control plane minor version from 4.21 minor to 4.22 minor · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:133]: failed to create HCP cluster "cp-ystream-upgrade-4-22-jzwpzg" at install version 4.21 Unexpected error: <fmt.wrapError \| 0xc000cb41c0>: failed to create HCP cluster cp-ystream-upgrade-4-22-jzwpzg: failed waiting for cluster="cp-ystream-upgrade-4-22-jzwpzg" in resourcegroup="rg-cp-ystream-upgrade-4-22-x2c887tw67g8" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/ee53a009-5244-40be-88eb-ee6e096be5ce -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:133]: failed to create HCP cluster "cp-ystream-upgrade-4-22-jzwpzg" at install version 4.21 Unexpected error: <fmt.wrapError \| 0xc000cb41c0>: failed to create HCP cluster cp-ystream-upgrade-4-22-jzwpzg: failed waiting for cluster="cp-ystream-upgrade-4-22-jzwpzg" in resourcegroup="rg-cp-ystream-upgrade-4-22-x2c887tw67g8" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/ee53a009-5244-40be-88eb-ee6e096be5ce -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 08:37:33 UTC · failed at e2e · test Customer should upgrade a nodepool skipping one minor version (+2) from 4.20.z to 4.22.zLatest · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:441]: failed to create HCP cluster np-skip-minor-rhxnvd at version 4.22.15 Unexpected error: <fmt.wrapError \| 0xc000d133e0>: failed to create HCP cluster np-skip-minor-rhxnvd: failed waiting for cluster="np-skip-minor-rhxnvd" in resourcegroup="rg-np-skip-minor-rhxnvd-p58vwfz6fjfr" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/7b9fc5fd-52eb-415c-9b34-eb04c67db17f -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:441]: failed to create HCP cluster np-skip-minor-rhxnvd at version 4.22.15 Unexpected error: <fmt.wrapError \| 0xc000d133e0>: failed to create HCP cluster np-skip-minor-rhxnvd: failed waiting for cluster="np-skip-minor-rhxnvd" in resourcegroup="rg-np-skip-minor-rhxnvd-p58vwfz6fjfr" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/7b9fc5fd-52eb-415c-9b34-eb04c67db17f -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 08:37:33 UTC · failed at e2e · test Customer should be able to lifecycle and confirm external auth on a cluster · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_list_and_verify.go:87]: failed to create HCP cluster for external auth list test Unexpected error: <fmt.wrapError \| 0xc000fd0140>: failed to create HCP cluster ea-listcwghrc: failed waiting for cluster="ea-listcwghrc" in resourcegroup="ea-list-87dmsp5cvfdc" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4f422e3a-fbeb-4f55-8dc5-72f74cefb46e -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_list_and_verify.go:87]: failed to create HCP cluster for external auth list test Unexpected error: <fmt.wrapError \| 0xc000fd0140>: failed to create HCP cluster ea-listcwghrc: failed waiting for cluster="ea-listcwghrc" in resourcegroup="ea-list-87dmsp5cvfdc" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/4f422e3a-fbeb-4f55-8dc5-72f74cefb46e -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 08:37:33 UTC · failed at e2e · test Customer should upgrade and update a nodepool from 4.20.z to 4.21.zLatest · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:146]: failed to create HCP cluster np-version-upgrade-cluster-t4fq96 with version 4.21.34 Unexpected error: <fmt.wrapError \| 0xc001664140>: failed to create HCP cluster np-version-upgrade-cluster-t4fq96: failed waiting for cluster="np-version-upgrade-cluster-t4fq96" in resourcegroup="rg-np-version-upgrade-t4fq96-sxs2pw2sbfpm" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b629eadb-edb1-436e-b96a-1333d66e0129 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:146]: failed to create HCP cluster np-version-upgrade-cluster-t4fq96 with version 4.21.34 Unexpected error: <fmt.wrapError \| 0xc001664140>: failed to create HCP cluster np-version-upgrade-cluster-t4fq96: failed waiting for cluster="np-version-upgrade-cluster-t4fq96" in resourcegroup="rg-np-version-upgrade-t4fq96-sxs2pw2sbfpm" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b629eadb-edb1-436e-b96a-1333d66e0129 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 08:37:33 UTC · failed at e2e · test Create HCPOpenShiftCluster with Private KeyVault should create a cluster with private keyvault using v20251223preview API · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:99]: failed to create HCP cluster "private-kv-cluster" with private keyvault Unexpected error: <fmt.wrapError \| 0xc000fce340>: failed waiting for cluster="private-kv-cluster" in resourcegroup="private-keyvault-8g8nztgsvfrg" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/8181c71c-8aa7-4bb1-9d0b-cda7ba17b489 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:99]: failed to create HCP cluster "private-kv-cluster" with private keyvault Unexpected error: <fmt.wrapError \| 0xc000fce340>: failed waiting for cluster="private-kv-cluster" in resourcegroup="private-keyvault-8g8nztgsvfrg" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/8181c71c-8aa7-4bb1-9d0b-cda7ba17b489 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 08:37:33 UTC · failed at e2e · test Customer should be able to create a cluster with an external auth config and get the external auth config · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:94]: failed to create HCP cluster for external auth test Unexpected error: <fmt.wrapError \| 0xc00101c180>: failed to create HCP cluster ea-cluster: failed waiting for cluster="ea-cluster" in resourcegroup="external-auth-cluster-zj4pb6lz9fcr" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/9f0857bd-047e-4488-8e92-d32991104e91 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:94]: failed to create HCP cluster for external auth test Unexpected error: <fmt.wrapError \| 0xc00101c180>: failed to create HCP cluster ea-cluster: failed waiting for cluster="ea-cluster" in resourcegroup="external-auth-cluster-zj4pb6lz9fcr" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/9f0857bd-047e-4488-8e92-d32991104e91 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 08:37:33 UTC · failed at e2e · test Customer should create a cluster with private KAS and verify API server is only reachable from VNet while ingress remains public · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:91]: failed to create HCP cluster "private-kas" with private KAS Unexpected error: <fmt.wrapError \| 0xc000ff8720>: failed to create HCP cluster private-kas: failed waiting for cluster="private-kas" in resourcegroup="private-kas-s5fm48wzgxnx" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/bf1fb47c-c0e0-48ae-9e60-b5f0bb511539 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:91]: failed to create HCP cluster "private-kas" with private KAS Unexpected error: <fmt.wrapError \| 0xc000ff8720>: failed to create HCP cluster private-kas: failed waiting for cluster="private-kas" in resourcegroup="private-kas-s5fm48wzgxnx" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/bf1fb47c-c0e0-48ae-9e60-b5f0bb511539 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 08:37:33 UTC · failed at e2e · test Customer should not be able to reuse subnets and NSGs between clusters · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_nsg_subnet_reuse.go:107]: failed to create seed HCP cluster basic-cluster Unexpected error: <fmt.wrapError \| 0xc00017eb60>: failed to create HCP cluster basic-cluster: failed waiting for cluster="basic-cluster" in resourcegroup="rg-cluster-nsg-subnet-reuse-z5bwmmgwqzxc" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/dc662588-7be0-4f38-a33c-686764d69d47 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_nsg_subnet_reuse.go:107]: failed to create seed HCP cluster basic-cluster Unexpected error: <fmt.wrapError \| 0xc00017eb60>: failed to create HCP cluster basic-cluster: failed waiting for cluster="basic-cluster" in resourcegroup="rg-cluster-nsg-subnet-reuse-z5bwmmgwqzxc" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/dc662588-7be0-4f38-a33c-686764d69d47 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 08:37:33 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 5.0 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:123]: HCP cluster rg-candidate-5-0-qwjhw2znn64b/cluster-candidate-5-0-dwllgv should provision Unexpected error: <fmt.wrapError \| 0xc000f1e1a0>: failed to create HCP cluster cluster-candidate-5-0-dwllgv: failed waiting for cluster="cluster-candidate-5-0-dwllgv" in resourcegroup="rg-candidate-5-0-qwjhw2znn64b" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/8390f874-3988-4d1a-9f9a-a231f2da3eab -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:123]: HCP cluster rg-candidate-5-0-qwjhw2znn64b/cluster-candidate-5-0-dwllgv should provision Unexpected error: <fmt.wrapError \| 0xc000f1e1a0>: failed to create HCP cluster cluster-candidate-5-0-dwllgv: failed waiting for cluster="cluster-candidate-5-0-dwllgv" in resourcegroup="rg-candidate-5-0-qwjhw2znn64b" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/8390f874-3988-4d1a-9f9a-a231f2da3eab -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 08:37:33 UTC · failed at e2e · test Customer should be able to delete a nodepool · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_delete.go:81]: failed to create HCP cluster np-delete-hcp-cluster Unexpected error: <fmt.wrapError \| 0xc000a34a20>: failed to create HCP cluster np-delete-hcp-cluster: failed waiting for cluster="np-delete-hcp-cluster" in resourcegroup="nodepool-delete-n9xkt4t8wxvs" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/c9309b1d-f416-4aea-87a8-fc29e712ceb5 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_delete.go:81]: failed to create HCP cluster np-delete-hcp-cluster Unexpected error: <fmt.wrapError \| 0xc000a34a20>: failed to create HCP cluster np-delete-hcp-cluster: failed waiting for cluster="np-delete-hcp-cluster" in resourcegroup="nodepool-delete-n9xkt4t8wxvs" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/c9309b1d-f416-4aea-87a8-fc29e712ceb5 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 08:37:33 UTC · failed at e2e · test Customer should create a cluster using v20260901preview API and verify cluster health · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_v20260901preview.go:86]: failed to create HCP cluster "v20260901" via v20260901preview Unexpected error: <fmt.wrapError \| 0xc001626600>: failed to create HCP cluster v20260901: failed waiting for cluster="v20260901" in resourcegroup="v20260901preview-sjczp869j2n7" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/772f5453-23ef-49ef-9bdb-63e2dbe81c91 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_v20260901preview.go:86]: failed to create HCP cluster "v20260901" via v20260901preview Unexpected error: <fmt.wrapError \| 0xc001626600>: failed to create HCP cluster v20260901: failed waiting for cluster="v20260901" in resourcegroup="v20260901preview-sjczp869j2n7" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/772f5453-23ef-49ef-9bdb-63e2dbe81c91 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 08:37:33 UTC · failed at e2e · test Customer should create a cluster with private ingress using v20260901preview and verify the ingress is internal · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_ingress.go:87]: failed to create HCP cluster "private-ingress" with private ingress Unexpected error: <fmt.wrapError \| 0xc00103a700>: failed to create HCP cluster private-ingress: failed waiting for cluster="private-ingress" in resourcegroup="private-ingress-jlwhkxsvk8nq" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/867cd450-604e-41a5-b442-4f55b470d528 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_ingress.go:87]: failed to create HCP cluster "private-ingress" with private ingress Unexpected error: <fmt.wrapError \| 0xc00103a700>: failed to create HCP cluster private-ingress: failed waiting for cluster="private-ingress" in resourcegroup="private-ingress-jlwhkxsvk8nq" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/867cd450-604e-41a5-b442-4f55b470d528 -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred
05:07:03	20m 39s	branch-ci-Azure-ARO-HCP-main-e2e-stage-e2e-parallel post-goodmerged PR No additional run metadata captured	uksouth	e2e	failed	n/a	59	e2e (59) ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 05:07:03 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 4.22 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:106]: failed to create customer resources for z-stream cluster "cluster-zstream-4-22-k7zrmf" Unexpected error: <fmt.wrapError \| 0xc001091f40>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cluster-zstream-4-22-k7zrmf-vzcmmv" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-zstream-4-22-k7zrmf-vzcmmv/operationStatuses/08584113795721473635 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-22-l6ttmcjqchrr/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-guxmg5tcq4f4g\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-03/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:106]: failed to create customer resources for z-stream cluster "cluster-zstream-4-22-k7zrmf" Unexpected error: <fmt.wrapError \| 0xc001091f40>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cluster-zstream-4-22-k7zrmf-vzcmmv" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-zstream-4-22-k7zrmf-vzcmmv/operationStatuses/08584113795721473635 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-22-l6ttmcjqchrr/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-guxmg5tcq4f4g\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-03/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 05:07:03 UTC · failed at e2e · test Customer should be able to create an HCP cluster then delete it by deleting the customer resource group · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delete_cx_rg.go:80]: failed to create customer resources for cluster "cx-rg-hcp-cluster" Unexpected error: <fmt.wrapError \| 0xc000ae2620>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cx-rg-hcp-cluster-tczj7p" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cx-rg-hcp-cluster-tczj7p/operationStatuses/08584113797417778439 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-11/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/cx-rg-cluster-pmnmz2hvttl5/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-fmxs36wu5sow2\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delete_cx_rg.go:80]: failed to create customer resources for cluster "cx-rg-hcp-cluster" Unexpected error: <fmt.wrapError \| 0xc000ae2620>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cx-rg-hcp-cluster-tczj7p" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cx-rg-hcp-cluster-tczj7p/operationStatuses/08584113797417778439 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-11/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/cx-rg-cluster-pmnmz2hvttl5/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-fmxs36wu5sow2\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 05:07:03 UTC · failed at e2e · test Customer should be able to create a cluster with default autoscaling and a nodepool with autoscaling enabled up to replica limits · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:93]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc00292a760>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-np-autoscale-cluster-6x8jr2" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-np-autoscale-cluster-6x8jr2/operationStatuses/08584113795062405694 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-20/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/np-autoscaling-rp7t4n8bpkjs/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-ssrwqrnhyyabs\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:93]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc00292a760>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-np-autoscale-cluster-6x8jr2" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-np-autoscale-cluster-6x8jr2/operationStatuses/08584113795062405694 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-20/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/np-autoscaling-rp7t4n8bpkjs/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-ssrwqrnhyyabs\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 05:07:03 UTC · failed at e2e · test Customer should be able to forward control plane logs to a storage account and Event Hub via shoebox diagnostic settings · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/shoebox_logs.go:268]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000813b80>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-shoebox-hcp-cluster-v6ssms" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-shoebox-hcp-cluster-v6ssms/operationStatuses/08584113794004887021 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/shoebox-cluster-pjzftjg6drs8/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-22eudp3sazqeq\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-00/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/shoebox_logs.go:268]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000813b80>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-shoebox-hcp-cluster-v6ssms" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-shoebox-hcp-cluster-v6ssms/operationStatuses/08584113794004887021 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/shoebox-cluster-pjzftjg6drs8/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-22eudp3sazqeq\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-00/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 05:07:03 UTC · failed at e2e · test Customer should be able to use workload identity via the cluster OIDC issuer URL · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/oidc_issuer_workload_identity.go:218]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000c173c0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-oidc-wi-cluster-gnz8xj" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-oidc-wi-cluster-gnz8xj/operationStatuses/08584113793892834537 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/oidc-wi-dgmkvhdn8svv/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-s6kwp7xg43dbi\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-02/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/oidc_issuer_workload_identity.go:218]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000c173c0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-oidc-wi-cluster-gnz8xj" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-oidc-wi-cluster-gnz8xj/operationStatuses/08584113793892834537 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/oidc-wi-dgmkvhdn8svv/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-s6kwp7xg43dbi\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-02/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 05:07:03 UTC · failed at e2e · test Customer should be able to create several HCP clusters in their customer resource group, but not in the same managed resource group · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:81]: failed to create customer resources for first cluster "basic-hcp-cluster" Unexpected error: <fmt.wrapError \| 0xc0010c52a0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-basic-hcp-cluster-926kkz" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-basic-hcp-cluster-926kkz/operationStatuses/08584113797461583341 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/customer-rg-xz8szf6bckm4/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-eqbgltuyzhpg4\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-04/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:81]: failed to create customer resources for first cluster "basic-hcp-cluster" Unexpected error: <fmt.wrapError \| 0xc0010c52a0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-basic-hcp-cluster-926kkz" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-basic-hcp-cluster-926kkz/operationStatuses/08584113797461583341 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/customer-rg-xz8szf6bckm4/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-eqbgltuyzhpg4\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-04/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 05:07:03 UTC · failed at e2e · test Customer should create a cluster using v20260901preview API and verify cluster health · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_v20260901preview.go:70]: failed to create customer resources for v20260901preview cluster Unexpected error: <fmt.wrapError \| 0xc000f4eaa0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-v20260901-nmdb6d" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-v20260901-nmdb6d/operationStatuses/08584113797613972324 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/v20260901preview-s8cf494trjq4/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-xdteugpxef23k\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-08/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_v20260901preview.go:70]: failed to create customer resources for v20260901preview cluster Unexpected error: <fmt.wrapError \| 0xc000f4eaa0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-v20260901-nmdb6d" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-v20260901-nmdb6d/operationStatuses/08584113797613972324 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/v20260901preview-s8cf494trjq4/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-xdteugpxef23k\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-08/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 05:07:03 UTC · failed at e2e · test Customer should be able to create an HCP cluster and custom node pool osDisk size · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_nodepool_osdisk.go:71]: failed to create customer resources for cluster "hcp-cluster-np-128" Unexpected error: <fmt.wrapError \| 0xc0004091e0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-hcp-cluster-np-128-trb4z9" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-hcp-cluster-np-128-trb4z9/operationStatuses/08584113797564088180 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/clusternp128-q2f888mn2cwt/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-3ljyjjx2n7gxo\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-20/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_nodepool_osdisk.go:71]: failed to create customer resources for cluster "hcp-cluster-np-128" Unexpected error: <fmt.wrapError \| 0xc0004091e0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-hcp-cluster-np-128-trb4z9" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-hcp-cluster-np-128-trb4z9/operationStatuses/08584113797564088180 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/clusternp128-q2f888mn2cwt/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-3ljyjjx2n7gxo\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-20/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 05:07:03 UTC · failed at e2e · test Customer should be able to delete a nodepool · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_delete.go:71]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000815280>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-np-delete-hcp-cluster-tz2dtt" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-np-delete-hcp-cluster-tz2dtt/operationStatuses/08584113794261759859 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-22/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/nodepool-delete-gxqmcjbj27xb/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-rcpnvqrcxdqmo\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_delete.go:71]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000815280>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-np-delete-hcp-cluster-tz2dtt" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-np-delete-hcp-cluster-tz2dtt/operationStatuses/08584113794261759859 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-22/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/nodepool-delete-gxqmcjbj27xb/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-rcpnvqrcxdqmo\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 05:07:03 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 5.0 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:112]: failed to create customer resources for cluster "cluster-candidate-5-0-sqzzfn" Unexpected error: <fmt.wrapError \| 0xc00110a600>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cluster-candidate-5-0-sqzzfn-lmmjjl" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-candidate-5-0-sqzzfn-lmmjjl/operationStatuses/08584113795936376304 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-5-0-jgtkss8655cz/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-vx4yub7b7dgwa\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-06/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:112]: failed to create customer resources for cluster "cluster-candidate-5-0-sqzzfn" Unexpected error: <fmt.wrapError \| 0xc00110a600>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cluster-candidate-5-0-sqzzfn-lmmjjl" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-candidate-5-0-sqzzfn-lmmjjl/operationStatuses/08584113795936376304 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-5-0-jgtkss8655cz/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-vx4yub7b7dgwa\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-06/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 05:07:03 UTC · failed at e2e · test Customer should be able to successfully upgrade control plane minor version from 4.20 minor to 4.21 minor · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:123]: failed to create customer resources for y-stream upgrade cluster "cp-ystream-upgrade-4-21-xpz8fh" Unexpected error: <fmt.wrapError \| 0xc0010081e0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cp-ystream-upgrade-4-21-xpz8fh-lm75lj" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cp-ystream-upgrade-4-21-xpz8fh-lm75lj/operationStatuses/08584113795344225784 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cp-ystream-upgrade-4-21-7d9tj4kd8q9d/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-yvqbgfau4idhs\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-08/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:123]: failed to create customer resources for y-stream upgrade cluster "cp-ystream-upgrade-4-21-xpz8fh" Unexpected error: <fmt.wrapError \| 0xc0010081e0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cp-ystream-upgrade-4-21-xpz8fh-lm75lj" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cp-ystream-upgrade-4-21-xpz8fh-lm75lj/operationStatuses/08584113795344225784 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cp-ystream-upgrade-4-21-7d9tj4kd8q9d/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-yvqbgfau4idhs\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-08/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 05:07:03 UTC · failed at e2e · test Customer should be able to create a cluster with an external auth config and get the external auth config · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:85]: failed to create customer resources for external auth cluster Unexpected error: <fmt.wrapError \| 0xc000b9fd20>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-ea-cluster-2dfr5b" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-ea-cluster-2dfr5b/operationStatuses/08584113794969206545 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-13/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/external-auth-cluster-fgdjlrr4zv8j/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-ltdphft7fgrwg\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:85]: failed to create customer resources for external auth cluster Unexpected error: <fmt.wrapError \| 0xc000b9fd20>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-ea-cluster-2dfr5b" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-ea-cluster-2dfr5b/operationStatuses/08584113794969206545 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-13/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/external-auth-cluster-fgdjlrr4zv8j/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-ltdphft7fgrwg\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 05:07:03 UTC · failed at e2e · test Customer should respect cluster-wide node limits with nodepool autoscaling · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:283]: failed to create cluster customer resources for node-limit test Unexpected error: <fmt.wrapError \| 0xc000e7dae0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-node-limit-cluster-4lkc6c" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-node-limit-cluster-4lkc6c/operationStatuses/08584113794966440147 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/node-limit-test-48zl586s8mmx/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-j6nw3ohx7bzo4\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-24/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:283]: failed to create cluster customer resources for node-limit test Unexpected error: <fmt.wrapError \| 0xc000e7dae0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-node-limit-cluster-4lkc6c" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-node-limit-cluster-4lkc6c/operationStatuses/08584113794966440147 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/node-limit-test-48zl586s8mmx/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-j6nw3ohx7bzo4\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-24/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 05:07:03 UTC · failed at e2e · test Customer should be able to update node pool labels and taints · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_labels_taints.go:82]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000c1c8c0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cluster-np-labels-taints-xc8skc" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-np-labels-taints-xc8skc/operationStatuses/08584113794310947201 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-15/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-labels-taints-vnr92zqdsct4/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-r6cuakilyx67k\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_labels_taints.go:82]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000c1c8c0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cluster-np-labels-taints-xc8skc" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-np-labels-taints-xc8skc/operationStatuses/08584113794310947201 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-15/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-labels-taints-vnr92zqdsct4/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-r6cuakilyx67k\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 05:07:03 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.21 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:112]: failed to create customer resources for cluster "cluster-candidate-4-21-snntjr" Unexpected error: <fmt.wrapError \| 0xc000905e20>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cluster-candidate-4-21-snntjr-dl994l" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-candidate-4-21-snntjr-dl994l/operationStatuses/08584113795982950394 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-05/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-4-21-rdlcqg2zmgtz/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-wdrbbs4qtkscu\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:112]: failed to create customer resources for cluster "cluster-candidate-4-21-snntjr" Unexpected error: <fmt.wrapError \| 0xc000905e20>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cluster-candidate-4-21-snntjr-dl994l" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-candidate-4-21-snntjr-dl994l/operationStatuses/08584113795982950394 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-05/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-4-21-rdlcqg2zmgtz/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-wdrbbs4qtkscu\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 05:07:03 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 4.20 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:106]: failed to create customer resources for z-stream cluster "cluster-zstream-4-20-4cn2wf" Unexpected error: <fmt.wrapError \| 0xc0016ee1e0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cluster-zstream-4-20-4cn2wf-k4pbgs" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-zstream-4-20-4cn2wf-k4pbgs/operationStatuses/08584113795723923153 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-20-qt7pwt9lj829/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-mazf27myroa4e\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-00/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:106]: failed to create customer resources for z-stream cluster "cluster-zstream-4-20-4cn2wf" Unexpected error: <fmt.wrapError \| 0xc0016ee1e0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cluster-zstream-4-20-4cn2wf-k4pbgs" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-zstream-4-20-4cn2wf-k4pbgs/operationStatuses/08584113795723923153 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-20-qt7pwt9lj829/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-mazf27myroa4e\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-00/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 05:07:03 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 5.0 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:106]: failed to create customer resources for z-stream cluster "cluster-zstream-5-0-p2llll" Unexpected error: <fmt.wrapError \| 0xc000a69d80>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cluster-zstream-5-0-p2llll-fwvg46" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-zstream-5-0-p2llll-fwvg46/operationStatuses/08584113795512069481 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-5-0-cx8fptjfl78k/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-6tmfgayffrkxs\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-01/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:106]: failed to create customer resources for z-stream cluster "cluster-zstream-5-0-p2llll" Unexpected error: <fmt.wrapError \| 0xc000a69d80>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cluster-zstream-5-0-p2llll-fwvg46" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-zstream-5-0-p2llll-fwvg46/operationStatuses/08584113795512069481 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-5-0-cx8fptjfl78k/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-6tmfgayffrkxs\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-01/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 05:07:03 UTC · failed at e2e · test Customer should be able to create an HCP cluster with back-level version 4.20 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:121]: failed to deploy managed identities for back-level version 4.20 Unexpected error: <fmt.wrapError \| 0xc000fcfae0>: failed waiting for deployment "mi-cluster-ver-4-20-vzkjqw" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-ver-4-20-vzkjqw/operationStatuses/08584113795989168208 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cluster-back-version-vlcf7czw8h2d/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-15/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:121]: failed to deploy managed identities for back-level version 4.20 Unexpected error: <fmt.wrapError \| 0xc000fcfae0>: failed waiting for deployment "mi-cluster-ver-4-20-vzkjqw" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-ver-4-20-vzkjqw/operationStatuses/08584113795989168208 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cluster-back-version-vlcf7czw8h2d/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-15/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 05:07:03 UTC · failed at e2e · test Create HCPOpenShiftCluster with Private KeyVault should create a cluster with private keyvault using v20251223preview API · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:76]: failed to create customer resources with private key vault Unexpected error: <fmt.wrapError \| 0xc0010ad300>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-private-kv-cluster-r9bkv2" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-private-kv-cluster-r9bkv2/operationStatuses/08584113796499725982 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/private-keyvault-9qdz5jkjz5x9/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-had2isw6vkgre\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-22/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:76]: failed to create customer resources with private key vault Unexpected error: <fmt.wrapError \| 0xc0010ad300>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-private-kv-cluster-r9bkv2" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-private-kv-cluster-r9bkv2/operationStatuses/08584113796499725982 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/private-keyvault-9qdz5jkjz5x9/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-had2isw6vkgre\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-22/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 05:07:03 UTC · failed at e2e · test Customer should be able to delete an HCP cluster whose managed identities were deleted first · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delete_missing_identities.go:101]: failed to deploy dedicated managed identities Unexpected error: <fmt.wrapError \| 0xc000f8d460>: failed waiting for deployment "mi-missing-mi-hcp-cluster-962vtg" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-missing-mi-hcp-cluster-962vtg/operationStatuses/08584113797566940213 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/delete-mi-rg-rdxdv7ssp57g/providers/Microsoft.Resources/deployments/clusterMsiScopedAssignments-jaooh6lj24n4o\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/delete-mi-rg-rdxdv7ssp57g/providers/Microsoft.Resources/deployments/clusterNonMsiScopedAssignments-jaooh6lj24n4o\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delete_missing_identities.go:101]: failed to deploy dedicated managed identities Unexpected error: <fmt.wrapError \| 0xc000f8d460>: failed waiting for deployment "mi-missing-mi-hcp-cluster-962vtg" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-missing-mi-hcp-cluster-962vtg/operationStatuses/08584113797566940213 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/delete-mi-rg-rdxdv7ssp57g/providers/Microsoft.Resources/deployments/clusterMsiScopedAssignments-jaooh6lj24n4o\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/delete-mi-rg-rdxdv7ssp57g/providers/Microsoft.Resources/deployments/clusterNonMsiScopedAssignments-jaooh6lj24n4o\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 05:07:03 UTC · failed at e2e · test Customer should be able to successfully upgrade control plane minor version from 4.22 minor to 4.23 minor · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:123]: failed to create customer resources for y-stream upgrade cluster "cp-ystream-upgrade-4-23-zbh5mm" Unexpected error: <fmt.wrapError \| 0xc00096f880>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cp-ystream-upgrade-4-23-zbh5mm-jkzp59" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cp-ystream-upgrade-4-23-zbh5mm-jkzp59/operationStatuses/08584113795257843740 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-11/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cp-ystream-upgrade-4-23-hcnjhtd8r27b/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-cgm32ouf327fy\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:123]: failed to create customer resources for y-stream upgrade cluster "cp-ystream-upgrade-4-23-zbh5mm" Unexpected error: <fmt.wrapError \| 0xc00096f880>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cp-ystream-upgrade-4-23-zbh5mm-jkzp59" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cp-ystream-upgrade-4-23-zbh5mm-jkzp59/operationStatuses/08584113795257843740 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-11/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cp-ystream-upgrade-4-23-hcnjhtd8r27b/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-cgm32ouf327fy\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 05:07:03 UTC · failed at e2e · test Customer should be able to successfully upgrade control plane minor version from 4.22 minor to 5.0 minor · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:123]: failed to create customer resources for y-stream upgrade cluster "cp-ystream-upgrade-5-0-qwwn9c" Unexpected error: <fmt.wrapError \| 0xc000a00a80>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cp-ystream-upgrade-5-0-qwwn9c-wlvz2l" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cp-ystream-upgrade-5-0-qwwn9c-wlvz2l/operationStatuses/08584113795128076410 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cp-ystream-upgrade-5-0-9mtgn4gf7nhc/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-4byxptqgl7j5c\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-12/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:123]: failed to create customer resources for y-stream upgrade cluster "cp-ystream-upgrade-5-0-qwwn9c" Unexpected error: <fmt.wrapError \| 0xc000a00a80>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cp-ystream-upgrade-5-0-qwwn9c-wlvz2l" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cp-ystream-upgrade-5-0-qwwn9c-wlvz2l/operationStatuses/08584113795128076410 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cp-ystream-upgrade-5-0-9mtgn4gf7nhc/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-4byxptqgl7j5c\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-12/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 05:07:03 UTC · failed at e2e · test Customer should create an HCP cluster and validate TLS certificates · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:87]: failed to create customer resources for TLS endpoint cluster Unexpected error: <fmt.wrapError \| 0xc00016fa20>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-tls-endpoint-hcp-cluster-z2sc66" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-tls-endpoint-hcp-cluster-z2sc66/operationStatuses/08584113797558647934 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/tls-endpoint-cluster-5ncqcmwnj49p/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-oo7pyvaeowk44\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-15/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:87]: failed to create customer resources for TLS endpoint cluster Unexpected error: <fmt.wrapError \| 0xc00016fa20>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-tls-endpoint-hcp-cluster-z2sc66" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-tls-endpoint-hcp-cluster-z2sc66/operationStatuses/08584113797558647934 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/tls-endpoint-cluster-5ncqcmwnj49p/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-oo7pyvaeowk44\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-15/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 05:07:03 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 4.21 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:106]: failed to create customer resources for z-stream cluster "cluster-zstream-4-21-dnd4px" Unexpected error: <fmt.wrapError \| 0xc0007c8a80>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cluster-zstream-4-21-dnd4px-lz842d" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-zstream-4-21-dnd4px-lz842d/operationStatuses/08584113795763065278 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-21-dgrk4rrfxmqr/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-h3fh53om2ygkc\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-02/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:106]: failed to create customer resources for z-stream cluster "cluster-zstream-4-21-dnd4px" Unexpected error: <fmt.wrapError \| 0xc0007c8a80>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cluster-zstream-4-21-dnd4px-lz842d" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-zstream-4-21-dnd4px-lz842d/operationStatuses/08584113795763065278 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-21-dgrk4rrfxmqr/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-h3fh53om2ygkc\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-02/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 05:07:03 UTC · failed at e2e · test Customer should create a cluster with private KAS and verify API server is only reachable from VNet while ingress remains public · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:77]: failed to create customer resources for private KAS cluster Unexpected error: <fmt.wrapError \| 0xc000ef3100>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-private-kas-4r9xdd" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-private-kas-4r9xdd/operationStatuses/08584113797510310178 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-14/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/private-kas-jswxb8xbdgc4/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-jxdbnndsp5js2\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:77]: failed to create customer resources for private KAS cluster Unexpected error: <fmt.wrapError \| 0xc000ef3100>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-private-kas-4r9xdd" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-private-kas-4r9xdd/operationStatuses/08584113797510310178 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-14/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/private-kas-jswxb8xbdgc4/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-jxdbnndsp5js2\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 05:07:03 UTC · failed at e2e · test Customer should be able to create a HCP cluster and use cilium CNI plugin · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_cni_cilium.go:78]: failed to create customer resources for cilium CNI cluster Unexpected error: <fmt.wrapError \| 0xc0010bc880>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cilium-cl-2g89zx" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cilium-cl-2g89zx/operationStatuses/08584113797437575156 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-19/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/cni-cilium-rt2rksv85s7p/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-dve5f7wncegny\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_cni_cilium.go:78]: failed to create customer resources for cilium CNI cluster Unexpected error: <fmt.wrapError \| 0xc0010bc880>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cilium-cl-2g89zx" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cilium-cl-2g89zx/operationStatuses/08584113797437575156 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-19/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/cni-cilium-rt2rksv85s7p/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-dve5f7wncegny\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 05:07:03 UTC · failed at e2e · test Customer should upgrade and update a nodepool from 4.20.z to 4.20.zLatest · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:136]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000c29300>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-np-version-upgrade-cluster-kw96zc-5l724p" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-np-version-upgrade-cluster-kw96zc-5l724p/operationStatuses/08584113794227856108 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-version-upgrade-kw96zc-lstdjv66nmvg/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-w2zbgzdrtwari\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-18/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:136]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000c29300>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-np-version-upgrade-cluster-kw96zc-5l724p" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-np-version-upgrade-cluster-kw96zc-5l724p/operationStatuses/08584113794227856108 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-version-upgrade-kw96zc-lstdjv66nmvg/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-w2zbgzdrtwari\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-18/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 05:07:03 UTC · failed at e2e · test Customer should be able to create a cluster and node pools with aggregated advanced features · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:130]: failed to create customer resources for aggregated feature cluster Unexpected error: <fmt.wrapError \| 0xc00067b320>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-agg-cluster-6fqb69" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-agg-cluster-6fqb69/operationStatuses/08584113796356952331 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-09/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/feature-aggregation-gg2tz5wqq5b9/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-zkzqbkxbrvnhu\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:130]: failed to create customer resources for aggregated feature cluster Unexpected error: <fmt.wrapError \| 0xc00067b320>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-agg-cluster-6fqb69" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-agg-cluster-6fqb69/operationStatuses/08584113796356952331 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-09/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/feature-aggregation-gg2tz5wqq5b9/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-zkzqbkxbrvnhu\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 05:07:03 UTC · failed at e2e · test Authorized CIDRs Connectivity should allow API access only from authorized VM IP · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:88]: failed to create customer resources for authorized CIDRs cluster Unexpected error: <fmt.wrapError \| 0xc001258420>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cidr-connectivity-test-f97wpv" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cidr-connectivity-test-f97wpv/operationStatuses/08584113797415943541 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-13/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-tlmdcr2hfq9p/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-wihoigd7cpguy\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:88]: failed to create customer resources for authorized CIDRs cluster Unexpected error: <fmt.wrapError \| 0xc001258420>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cidr-connectivity-test-f97wpv" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cidr-connectivity-test-f97wpv/operationStatuses/08584113797415943541 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-13/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-tlmdcr2hfq9p/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-wihoigd7cpguy\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 05:07:03 UTC · failed at e2e · test Customer should upgrade and update a nodepool from 4.20.z to 4.21.zLatest · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:136]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc0015ca640>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-np-version-upgrade-cluster-bfd6pf-66vswn" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-np-version-upgrade-cluster-bfd6pf-66vswn/operationStatuses/08584113794259884121 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-version-upgrade-bfd6pf-dhltqt8zw7sr/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-dx3utudtged7u\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-07/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:136]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc0015ca640>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-np-version-upgrade-cluster-bfd6pf-66vswn" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-np-version-upgrade-cluster-bfd6pf-66vswn/operationStatuses/08584113794259884121 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-version-upgrade-bfd6pf-dhltqt8zw7sr/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-dx3utudtged7u\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-07/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 05:07:03 UTC · failed at e2e · test ARO HCP Service should successfully provision cluster when MI role assignments are added after cluster creation · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:140]: failed to start cluster creation Unexpected error: <exported.ResponseError \| 0xc000f02720>: PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/delayed-rbac-v5ql2zlqgx5w/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/delayed-rbac-cluster -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:140]: failed to start cluster creation Unexpected error: <exported.ResponseError \| 0xc000f02720>: PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/delayed-rbac-v5ql2zlqgx5w/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/delayed-rbac-cluster -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 05:07:03 UTC · failed at e2e · test Nodepool Ephemeral OS Disk should create a nodepool with ephemeral OS disk when autoRepair is enabled · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_ephemeral_osdisk.go:75]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000716a80>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-ephemeral-disk-vsscw5" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-ephemeral-disk-vsscw5/operationStatuses/08584113794273516182 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/ephemeral-osdisk-6hllwhwnc2ls/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-6xts4ycasve7q\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-09/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_ephemeral_osdisk.go:75]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000716a80>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-ephemeral-disk-vsscw5" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-ephemeral-disk-vsscw5/operationStatuses/08584113794273516182 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/ephemeral-osdisk-6hllwhwnc2ls/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-6xts4ycasve7q\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-09/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 05:07:03 UTC · failed at e2e · test Customer should upgrade a nodepool skipping one minor version (+2) from 4.20.z to 4.22.zLatest · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:431]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc0014fe740>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-np-skip-minor-9fvzll-xr988b" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-np-skip-minor-9fvzll-xr988b/operationStatuses/08584113794038808904 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-skip-minor-9fvzll-2l748x67s24f/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-wlqpvuhemxk5s\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-05/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:431]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc0014fe740>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-np-skip-minor-9fvzll-xr988b" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-np-skip-minor-9fvzll-xr988b/operationStatuses/08584113794038808904 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-skip-minor-9fvzll-2l748x67s24f/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-wlqpvuhemxk5s\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-05/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 05:07:03 UTC · failed at e2e · test Customer should downgrade a nodepool to a lower minor version from 4.22.zLatest to 4.20.zLatest · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:710]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc00140c6c0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-np-dg-minor-vnvrkq-knrpwn" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-np-dg-minor-vnvrkq-knrpwn/operationStatuses/08584113794346095602 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-23/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-dg-minor-vnvrkq-skf4kw8tskx9/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-zyxd247esju2u\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:710]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc00140c6c0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-np-dg-minor-vnvrkq-knrpwn" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-np-dg-minor-vnvrkq-knrpwn/operationStatuses/08584113794346095602 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-23/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-dg-minor-vnvrkq-skf4kw8tskx9/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-zyxd247esju2u\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 05:07:03 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.23 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:112]: failed to create customer resources for cluster "cluster-candidate-4-23-5b4jwd" Unexpected error: <fmt.wrapError \| 0xc000c15860>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cluster-candidate-4-23-5b4jwd-nvrdbw" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-candidate-4-23-5b4jwd-nvrdbw/operationStatuses/08584113795807931830 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-4-23-4jlqndxm4h9r/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-caw3ygfnjzhps\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-07/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:112]: failed to create customer resources for cluster "cluster-candidate-4-23-5b4jwd" Unexpected error: <fmt.wrapError \| 0xc000c15860>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cluster-candidate-4-23-5b4jwd-nvrdbw" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-candidate-4-23-5b4jwd-nvrdbw/operationStatuses/08584113795807931830 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-4-23-4jlqndxm4h9r/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-caw3ygfnjzhps\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-07/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 05:07:03 UTC · failed at e2e · test Customer should be able to create an HCP cluster with Image Registry not present · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/image_registry_cluster_create.go:79]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000a0c900>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-disabled-image-registry-hcp-cluster-gbttsk" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-disabled-image-registry-hcp-cluster-gbttsk/operationStatuses/08584113795153509177 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/disabled-image-registry-mz4fw89j2cdt/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-qmh74zogaodhg\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-16/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/image_registry_cluster_create.go:79]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000a0c900>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-disabled-image-registry-hcp-cluster-gbttsk" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-disabled-image-registry-hcp-cluster-gbttsk/operationStatuses/08584113795153509177 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/disabled-image-registry-mz4fw89j2cdt/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-qmh74zogaodhg\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-16/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 05:07:03 UTC · failed at e2e · test Customer should be able to create an HCP cluster and manage pull secrets · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_pullsecret.go:104]: failed to create customer resources for pull secret cluster Unexpected error: <fmt.wrapError \| 0xc001266860>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-pullsecret-hcp-cluster-9bsxxt" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-pullsecret-hcp-cluster-9bsxxt/operationStatuses/08584113797561738888 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/pullsecret-test-zwnpp7n4fhs9/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-p2h32vyw2l2ma\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-23/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_pullsecret.go:104]: failed to create customer resources for pull secret cluster Unexpected error: <fmt.wrapError \| 0xc001266860>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-pullsecret-hcp-cluster-9bsxxt" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-pullsecret-hcp-cluster-9bsxxt/operationStatuses/08584113797561738888 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/pullsecret-test-zwnpp7n4fhs9/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-p2h32vyw2l2ma\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-23/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 05:07:03 UTC · failed at e2e · test Customer should be able to retrieve cluster and nodepool status active versions · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_nodepool_active_versions.go:73]: failed to create customer resources for v20261001preview cluster Unexpected error: <fmt.wrapError \| 0xc00106f760>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-v20261001-x6gm79" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-v20261001-x6gm79/operationStatuses/08584113797462734569 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/v20261001preview-w8qdl2mz4kg9/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-7e767rdphst3w\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-12/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_nodepool_active_versions.go:73]: failed to create customer resources for v20261001preview cluster Unexpected error: <fmt.wrapError \| 0xc00106f760>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-v20261001-x6gm79" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-v20261001-x6gm79/operationStatuses/08584113797462734569 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/v20261001preview-w8qdl2mz4kg9/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-7e767rdphst3w\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-12/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 05:07:03 UTC · failed at e2e · test Customer should be able to list available HCP OpenShift versions and validate response content · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_versions.go:43]: failed to list HCP OpenShift versions Unexpected error: <exported.ResponseError \| 0xc0014ffda0>: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOpenShiftVersions -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_versions.go:43]: failed to list HCP OpenShift versions Unexpected error: <exported.ResponseError \| 0xc0014ffda0>: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOpenShiftVersions -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 05:07:03 UTC · failed at e2e · test Customer should be able to issue the default ingress serving certificate from Let's Encrypt using cert-manager with ACME DNS-01 challenges · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_ingress_cert_letsencrypt.go:114]: failed to create customer resources Unexpected error: <fmt.wrapError \| 0xc000b13100>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-ingress-cert-le-5tmv7b" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-ingress-cert-le-5tmv7b/operationStatuses/08584113797516953541 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-24/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/ingress-cert-le-qzpjtfdnhg6c/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-nhimtfs3fueps\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_ingress_cert_letsencrypt.go:114]: failed to create customer resources Unexpected error: <fmt.wrapError \| 0xc000b13100>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-ingress-cert-le-5tmv7b" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-ingress-cert-le-5tmv7b/operationStatuses/08584113797516953541 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-24/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/ingress-cert-le-qzpjtfdnhg6c/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-nhimtfs3fueps\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 05:07:03 UTC · failed at e2e · test Customer should be able to create an HCP cluster with custom autoscaling and update the autoscaling configuration · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:97]: failed to create customer resources for autoscaling cluster Unexpected error: <fmt.wrapError \| 0xc001084920>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-autoscaling-hcp-cluster-jw828q" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-autoscaling-hcp-cluster-jw828q/operationStatuses/08584113797516221654 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-16/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/autoscaling-cluster-sk5txhzw7bwk/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-e4ixwyxsfjnje\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:97]: failed to create customer resources for autoscaling cluster Unexpected error: <fmt.wrapError \| 0xc001084920>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-autoscaling-hcp-cluster-jw828q" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-autoscaling-hcp-cluster-jw828q/operationStatuses/08584113797516221654 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-16/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/autoscaling-cluster-sk5txhzw7bwk/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-e4ixwyxsfjnje\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 05:07:03 UTC · failed at e2e · test Customer should be able to create an HCP cluster and manage ImageDigestMirrors · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/idms_lifecycle.go:90]: failed to create customer resources for IDMS cluster Unexpected error: <fmt.wrapError \| 0xc000da13e0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-idms-e2e-hcp-cluster-x7p8lc" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-idms-e2e-hcp-cluster-x7p8lc/operationStatuses/08584113795005880625 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/idms-q68kljjnmbn5/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-lv7dvzspd2e5i\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-19/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/idms_lifecycle.go:90]: failed to create customer resources for IDMS cluster Unexpected error: <fmt.wrapError \| 0xc000da13e0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-idms-e2e-hcp-cluster-x7p8lc" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-idms-e2e-hcp-cluster-x7p8lc/operationStatuses/08584113795005880625 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/idms-q68kljjnmbn5/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-lv7dvzspd2e5i\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-19/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 05:07:03 UTC · failed at e2e · test Customer should not be able to perform various invalid operations on cluster resources · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/simple_negative_cases.go:83]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000c5c080>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-negative-tests-cluster-r7749z" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-negative-tests-cluster-r7749z/operationStatuses/08584113794054293332 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-03/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-negative-tests-mtn6lkwlzgsq/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-oeayaalhj7wjy\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/simple_negative_cases.go:83]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000c5c080>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-negative-tests-cluster-r7749z" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-negative-tests-cluster-r7749z/operationStatuses/08584113794054293332 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-03/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-negative-tests-mtn6lkwlzgsq/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-oeayaalhj7wjy\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 05:07:03 UTC · failed at e2e · test Update HCPOpenShiftCluster Positive creates a cluster and updates tags with a PATCH request · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:151]: failed to create customer resources for patch-tags cluster Unexpected error: <fmt.wrapError \| 0xc000914b80>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-patch-tags-cluster-pw89nk" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-patch-tags-cluster-pw89nk/operationStatuses/08584113795987974979 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-tags-bkph9dv7sn7p/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-fklo75htpxzxc\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-21/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:151]: failed to create customer resources for patch-tags cluster Unexpected error: <fmt.wrapError \| 0xc000914b80>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-patch-tags-cluster-pw89nk" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-patch-tags-cluster-pw89nk/operationStatuses/08584113795987974979 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-tags-bkph9dv7sn7p/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-fklo75htpxzxc\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-21/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 05:07:03 UTC · failed at e2e · test FIPS Mode Support with v2026-09-01 API should create an HCP cluster with FIPS mode enabled via cryptoRestrictions property · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_fips_mode.go:72]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000e9bce0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-fips-enabled-cluster-zpl7rt" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-fips-enabled-cluster-zpl7rt/operationStatuses/08584113797321072088 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-10/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/fips-enabled-gn2b8gjr9kwt/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-3h7v3eeeqgjeu\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_fips_mode.go:72]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000e9bce0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-fips-enabled-cluster-zpl7rt" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-fips-enabled-cluster-zpl7rt/operationStatuses/08584113797321072088 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-10/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/fips-enabled-gn2b8gjr9kwt/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-3h7v3eeeqgjeu\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 05:07:03 UTC · failed at e2e · test Customer should be able to cordon, drain, and uncordon a node in an HCP cluster · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_cordon.go:80]: failed to create cluster customer resources for cordon test Unexpected error: <fmt.wrapError \| 0xc0014307c0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cordon-hcp-cluster-wvnprt" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cordon-hcp-cluster-wvnprt/operationStatuses/08584113795113577924 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-17/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cordon-test-5xnxcpchnklr/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-3wbuyr3ac6kzo\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_cordon.go:80]: failed to create cluster customer resources for cordon test Unexpected error: <fmt.wrapError \| 0xc0014307c0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cordon-hcp-cluster-wvnprt" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cordon-hcp-cluster-wvnprt/operationStatuses/08584113795113577924 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-17/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cordon-test-5xnxcpchnklr/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-3wbuyr3ac6kzo\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 05:07:03 UTC · failed at e2e · test Customer should upgrade and update a nodepool from 4.21.z to 4.21.zLatest · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:136]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000c2cca0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-np-version-upgrade-cluster-cfzv76-5v6stt" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-np-version-upgrade-cluster-cfzv76-5v6stt/operationStatuses/08584113794362466667 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-version-upgrade-cfzv76-v5jm55d72ljj/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-qohedkf2u7zja\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-06/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:136]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000c2cca0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-np-version-upgrade-cluster-cfzv76-5v6stt" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-np-version-upgrade-cluster-cfzv76-5v6stt/operationStatuses/08584113794362466667 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-version-upgrade-cfzv76-v5jm55d72ljj/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-qohedkf2u7zja\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-06/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 05:07:03 UTC · failed at e2e · test Customer should downgrade a nodepool version z-stream downgrade from 4.21.zLatest to 4.21.zPrevious · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:570]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000ffc1c0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-np-downgrade-jsfxcg-g9msqd" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-np-downgrade-jsfxcg-g9msqd/operationStatuses/08584113794199053326 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-downgrade-jsfxcg-wcdlrkd9bvdh/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-ibtiavrd7ygfu\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-21/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:570]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000ffc1c0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-np-downgrade-jsfxcg-g9msqd" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-np-downgrade-jsfxcg-g9msqd/operationStatuses/08584113794199053326 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-downgrade-jsfxcg-wcdlrkd9bvdh/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-ibtiavrd7ygfu\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-21/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 05:07:03 UTC · failed at e2e · test Customer should upgrade a nodepool to a version without Cincinnati upgrade edge z-stream upgrade without Cincinnati edge in 4.20 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:291]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000c6ee20>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-np-noedge-wsdcjf-kmz2kq" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-np-noedge-wsdcjf-kmz2kq/operationStatuses/08584113794289141624 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-noedge-wsdcjf-vd6xl6klg8wv/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-hpoi33muixkmu\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-04/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:291]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000c6ee20>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-np-noedge-wsdcjf-kmz2kq" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-np-noedge-wsdcjf-kmz2kq/operationStatuses/08584113794289141624 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-noedge-wsdcjf-vd6xl6klg8wv/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-hpoi33muixkmu\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-04/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 05:07:03 UTC · failed at e2e · test Customer should be able to successfully upgrade control plane minor version from 4.21 minor to 4.22 minor · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:123]: failed to create customer resources for y-stream upgrade cluster "cp-ystream-upgrade-4-22-xvv6x2" Unexpected error: <fmt.wrapError \| 0xc0014485c0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cp-ystream-upgrade-4-22-xvv6x2-9hs6th" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cp-ystream-upgrade-4-22-xvv6x2-9hs6th/operationStatuses/08584113795530207961 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-10/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cp-ystream-upgrade-4-22-nzqtchg88pjc/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-y4oqvpje3fy3s\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:123]: failed to create customer resources for y-stream upgrade cluster "cp-ystream-upgrade-4-22-xvv6x2" Unexpected error: <fmt.wrapError \| 0xc0014485c0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cp-ystream-upgrade-4-22-xvv6x2-9hs6th" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cp-ystream-upgrade-4-22-xvv6x2-9hs6th/operationStatuses/08584113795530207961 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-10/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cp-ystream-upgrade-4-22-nzqtchg88pjc/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-y4oqvpje3fy3s\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 05:07:03 UTC · failed at e2e · test Update HCPOpenShiftCluster Negative creates a cluster and fails to update its name with a PATCH request · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:74]: failed to create customer resources for patch-name cluster Unexpected error: <fmt.wrapError \| 0xc000c0a7c0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-patch-name-cluster-9sqc9d" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-patch-name-cluster-9sqc9d/operationStatuses/08584113797646453524 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-name-22zzphmvswcv/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-rr7tdjqoygobu\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-21/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:74]: failed to create customer resources for patch-name cluster Unexpected error: <fmt.wrapError \| 0xc000c0a7c0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-patch-name-cluster-9sqc9d" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-patch-name-cluster-9sqc9d/operationStatuses/08584113797646453524 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-name-22zzphmvswcv/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-rr7tdjqoygobu\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-21/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 05:07:03 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.20 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:112]: failed to create customer resources for cluster "cluster-candidate-4-20-2wmzq5" Unexpected error: <fmt.wrapError \| 0xc00117e4c0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cluster-candidate-4-20-2wmzq5-p9lzt6" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-candidate-4-20-2wmzq5-p9lzt6/operationStatuses/08584113795956074537 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-23/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-4-20-f5wgwlh8c4vw/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-ldmq4dx4vhh32\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:112]: failed to create customer resources for cluster "cluster-candidate-4-20-2wmzq5" Unexpected error: <fmt.wrapError \| 0xc00117e4c0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cluster-candidate-4-20-2wmzq5-p9lzt6" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-candidate-4-20-2wmzq5-p9lzt6/operationStatuses/08584113795956074537 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-23/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-4-20-f5wgwlh8c4vw/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-ldmq4dx4vhh32\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 05:07:03 UTC · failed at e2e · test Customer should create a cluster with private ingress using v20260901preview and verify the ingress is internal · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_ingress.go:77]: failed to create customer resources for private ingress cluster Unexpected error: <fmt.wrapError \| 0xc000fd84e0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-private-ingress-kpdmwx" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-private-ingress-kpdmwx/operationStatuses/08584113797618485125 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-03/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/private-ingress-dfpk9s64d99s/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-6rnsb3kbhjy5m\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_ingress.go:77]: failed to create customer resources for private ingress cluster Unexpected error: <fmt.wrapError \| 0xc000fd84e0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-private-ingress-kpdmwx" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-private-ingress-kpdmwx/operationStatuses/08584113797618485125 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-03/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/private-ingress-dfpk9s64d99s/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-6rnsb3kbhjy5m\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 05:07:03 UTC · failed at e2e · test Customer should be able to create node pools with ARM64-based VMs · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/arm64_nodepool.go:78]: failed to create customer resources for ARM64 nodepool cluster Unexpected error: <fmt.wrapError \| 0xc00108d040>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-arm64-vm-hcp-cluster-f6fqrw" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-arm64-vm-hcp-cluster-f6fqrw/operationStatuses/08584113797563017541 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/arm64-vm-cluster-xxmw2kmjpth9/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-6htyoibevj7ye\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-02/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/arm64_nodepool.go:78]: failed to create customer resources for ARM64 nodepool cluster Unexpected error: <fmt.wrapError \| 0xc00108d040>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-arm64-vm-hcp-cluster-f6fqrw" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-arm64-vm-hcp-cluster-f6fqrw/operationStatuses/08584113797563017541 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/arm64-vm-cluster-xxmw2kmjpth9/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-6htyoibevj7ye\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-02/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 05:07:03 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.22 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:112]: failed to create customer resources for cluster "cluster-candidate-4-22-9lrcvj" Unexpected error: <fmt.wrapError \| 0xc0017c5180>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cluster-candidate-4-22-9lrcvj-ltmpzx" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-candidate-4-22-9lrcvj-ltmpzx/operationStatuses/08584113795838273230 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-4-22-8h74rmx9b4nd/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-qbe63cdkmzx74\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-04/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:112]: failed to create customer resources for cluster "cluster-candidate-4-22-9lrcvj" Unexpected error: <fmt.wrapError \| 0xc0017c5180>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cluster-candidate-4-22-9lrcvj-ltmpzx" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-candidate-4-22-9lrcvj-ltmpzx/operationStatuses/08584113795838273230 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-4-22-8h74rmx9b4nd/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-qbe63cdkmzx74\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-04/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 05:07:03 UTC · failed at e2e · test Customer should be able to create a no-CNI private cluster with a private key vault, a nodepool and install cilium CNI successfully · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:84]: failed to create customer resources with private key vault Unexpected error: <fmt.wrapError \| 0xc000d9a000>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cilium-cluster-rsh86k" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cilium-cluster-rsh86k/operationStatuses/08584113796557935367 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/complex-cilium-kv-stmhvp49mf2t/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-4vnk4izirdfzu\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-17/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:84]: failed to create customer resources with private key vault Unexpected error: <fmt.wrapError \| 0xc000d9a000>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cilium-cluster-rsh86k" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cilium-cluster-rsh86k/operationStatuses/08584113796557935367 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/complex-cilium-kv-stmhvp49mf2t/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-4vnk4izirdfzu\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-17/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 05:07:03 UTC · failed at e2e · test Customer should not be able to reuse subnets and NSGs between clusters · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_nsg_subnet_reuse.go:97]: failed to create customer resources for basic-cluster Unexpected error: <fmt.wrapError \| 0xc000fd89e0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-basic-cluster-dthblf" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-basic-cluster-dthblf/operationStatuses/08584113797418741604 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-01/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cluster-nsg-subnet-reuse-5bb59rzmvcrm/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-j3gkygvtvkvjs\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_nsg_subnet_reuse.go:97]: failed to create customer resources for basic-cluster Unexpected error: <fmt.wrapError \| 0xc000fd89e0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-basic-cluster-dthblf" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-basic-cluster-dthblf/operationStatuses/08584113797418741604 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-01/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cluster-nsg-subnet-reuse-5bb59rzmvcrm/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-j3gkygvtvkvjs\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 05:07:03 UTC · failed at e2e · test Customer should be able to lifecycle and confirm external auth on a cluster · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_list_and_verify.go:78]: failed to create customer resources for external auth list test Unexpected error: <fmt.wrapError \| 0xc000571500>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-ea-listfdmjhw-kqcrb8" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-ea-listfdmjhw-kqcrb8/operationStatuses/08584113795169768721 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-14/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/ea-list-65zvssw2sbb2/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-igsi2gfoyf6yg\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_list_and_verify.go:78]: failed to create customer resources for external auth list test Unexpected error: <fmt.wrapError \| 0xc000571500>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-ea-listfdmjhw-kqcrb8" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-ea-listfdmjhw-kqcrb8/operationStatuses/08584113795169768721 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-14/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/ea-list-65zvssw2sbb2/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-igsi2gfoyf6yg\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 05:07:03 UTC · failed at e2e · test Customer should be able to test admin credentials before cluster ready, then full admin credential lifecycle · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:84]: failed to create customer resources for admin credential lifecycle cluster Unexpected error: <fmt.wrapError \| 0xc000b1f100>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-admin-cred-lifecycle-958k92-469hvc" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-admin-cred-lifecycle-958k92-469hvc/operationStatuses/08584113797514814206 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-07/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/admin-credential-lifecycle-test-tbndf5jrz5gk/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-o7e4n5kmatesw\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:84]: failed to create customer resources for admin credential lifecycle cluster Unexpected error: <fmt.wrapError \| 0xc000b1f100>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-admin-cred-lifecycle-958k92-469hvc" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-admin-cred-lifecycle-958k92-469hvc/operationStatuses/08584113797514814206 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-07/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/admin-credential-lifecycle-test-tbndf5jrz5gk/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-o7e4n5kmatesw\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred
04:47:20	28m 41s	branch-ci-Azure-ARO-HCP-main-e2e-stage-e2e-parallel post-goodmerged PR No additional run metadata captured	uksouth	e2e	failed	n/a	59	e2e (59) ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 04:47:20 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 4.21 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:106]: failed to create customer resources for z-stream cluster "cluster-zstream-4-21-vsc5jw" Unexpected error: <fmt.wrapError \| 0xc00070f6a0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cluster-zstream-4-21-vsc5jw-fkccmf" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-zstream-4-21-vsc5jw-fkccmf/operationStatuses/08584113804276958246 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-12/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-21-mzdsvtggrpzt/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-lzgahm6b5jz4u\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:106]: failed to create customer resources for z-stream cluster "cluster-zstream-4-21-vsc5jw" Unexpected error: <fmt.wrapError \| 0xc00070f6a0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cluster-zstream-4-21-vsc5jw-fkccmf" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-zstream-4-21-vsc5jw-fkccmf/operationStatuses/08584113804276958246 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-12/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-21-mzdsvtggrpzt/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-lzgahm6b5jz4u\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 04:47:20 UTC · failed at e2e · test Customer should be able to create a cluster with an external auth config and get the external auth config · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:85]: failed to create customer resources for external auth cluster Unexpected error: <fmt.wrapError \| 0xc00098ec60>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-ea-cluster-2tvq49" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-ea-cluster-2tvq49/operationStatuses/08584113803871976925 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/external-auth-cluster-xkpjgn9p2kgz/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-ltdphft7fgrwg\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-17/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:85]: failed to create customer resources for external auth cluster Unexpected error: <fmt.wrapError \| 0xc00098ec60>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-ea-cluster-2tvq49" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-ea-cluster-2tvq49/operationStatuses/08584113803871976925 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/external-auth-cluster-xkpjgn9p2kgz/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-ltdphft7fgrwg\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-17/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 04:47:20 UTC · failed at e2e · test Customer should upgrade and update a nodepool from 4.21.z to 4.21.zLatest · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:136]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000c51cc0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-np-version-upgrade-cluster-rf4wqp-j4wfnb" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-np-version-upgrade-cluster-rf4wqp-j4wfnb/operationStatuses/08584113803181772614 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-version-upgrade-rf4wqp-8lml4422b28z/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-eugg6mqzqllns\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-08/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:136]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000c51cc0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-np-version-upgrade-cluster-rf4wqp-j4wfnb" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-np-version-upgrade-cluster-rf4wqp-j4wfnb/operationStatuses/08584113803181772614 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-version-upgrade-rf4wqp-8lml4422b28z/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-eugg6mqzqllns\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-08/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 04:47:20 UTC · failed at e2e · test Customer should not be able to perform various invalid operations on cluster resources · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/simple_negative_cases.go:83]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000c78100>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-negative-tests-cluster-hwxkxn" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-negative-tests-cluster-hwxkxn/operationStatuses/08584113802769766971 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-00/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-negative-tests-654tzd4mpbxq/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-oeayaalhj7wjy\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/simple_negative_cases.go:83]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000c78100>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-negative-tests-cluster-hwxkxn" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-negative-tests-cluster-hwxkxn/operationStatuses/08584113802769766971 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-00/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-negative-tests-654tzd4mpbxq/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-oeayaalhj7wjy\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 04:47:20 UTC · failed at e2e · test Customer should be able to create a cluster and node pools with aggregated advanced features · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:130]: failed to create customer resources for aggregated feature cluster Unexpected error: <fmt.wrapError \| 0xc000d1a860>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-agg-cluster-g7sdmh" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-agg-cluster-g7sdmh/operationStatuses/08584113805185771525 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-13/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/feature-aggregation-nc2jr4f2pntk/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-zkzqbkxbrvnhu\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:130]: failed to create customer resources for aggregated feature cluster Unexpected error: <fmt.wrapError \| 0xc000d1a860>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-agg-cluster-g7sdmh" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-agg-cluster-g7sdmh/operationStatuses/08584113805185771525 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-13/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/feature-aggregation-nc2jr4f2pntk/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-zkzqbkxbrvnhu\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 04:47:20 UTC · failed at e2e · test Customer should be able to test admin credentials before cluster ready, then full admin credential lifecycle · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:84]: failed to create customer resources for admin credential lifecycle cluster Unexpected error: <fmt.wrapError \| 0xc000fc1bc0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-admin-cred-lifecycle-2cnwp4-7bpzvl" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-admin-cred-lifecycle-2cnwp4-7bpzvl/operationStatuses/08584113806114353804 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/admin-credential-lifecycle-test-lx842g284ln5/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-vjvkbm7kl4rpg\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-02/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:84]: failed to create customer resources for admin credential lifecycle cluster Unexpected error: <fmt.wrapError \| 0xc000fc1bc0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-admin-cred-lifecycle-2cnwp4-7bpzvl" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-admin-cred-lifecycle-2cnwp4-7bpzvl/operationStatuses/08584113806114353804 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/admin-credential-lifecycle-test-lx842g284ln5/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-vjvkbm7kl4rpg\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-02/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 04:47:20 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 5.0 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:112]: failed to create customer resources for cluster "cluster-candidate-5-0-dhslq8" Unexpected error: <fmt.wrapError \| 0xc00109a1a0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cluster-candidate-5-0-dhslq8-5ptsrb" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-candidate-5-0-dhslq8-5ptsrb/operationStatuses/08584113804398479878 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-5-0-nj5xbnvhxvg4/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-ggun3uwr6kkgc\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-03/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:112]: failed to create customer resources for cluster "cluster-candidate-5-0-dhslq8" Unexpected error: <fmt.wrapError \| 0xc00109a1a0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cluster-candidate-5-0-dhslq8-5ptsrb" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-candidate-5-0-dhslq8-5ptsrb/operationStatuses/08584113804398479878 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-5-0-nj5xbnvhxvg4/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-ggun3uwr6kkgc\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-03/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 04:47:20 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.23 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:112]: failed to create customer resources for cluster "cluster-candidate-4-23-dknk46" Unexpected error: <fmt.wrapError \| 0xc000c470e0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cluster-candidate-4-23-dknk46-dtqz2l" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-candidate-4-23-dknk46-dtqz2l/operationStatuses/08584113804398489673 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-4-23-82xpjxxksvr2/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-szbmn4feb43d2\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-21/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:112]: failed to create customer resources for cluster "cluster-candidate-4-23-dknk46" Unexpected error: <fmt.wrapError \| 0xc000c470e0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cluster-candidate-4-23-dknk46-dtqz2l" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-candidate-4-23-dknk46-dtqz2l/operationStatuses/08584113804398489673 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-4-23-82xpjxxksvr2/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-szbmn4feb43d2\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-21/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 04:47:20 UTC · failed at e2e · test Customer should be able to delete a nodepool · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_delete.go:71]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000e400c0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-np-delete-hcp-cluster-djw8f4" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-np-delete-hcp-cluster-djw8f4/operationStatuses/08584113803554561663 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-24/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/nodepool-delete-6s5wqcfjnx94/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-rcpnvqrcxdqmo\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_delete.go:71]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000e400c0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-np-delete-hcp-cluster-djw8f4" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-np-delete-hcp-cluster-djw8f4/operationStatuses/08584113803554561663 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-24/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/nodepool-delete-6s5wqcfjnx94/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-rcpnvqrcxdqmo\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 04:47:20 UTC · failed at e2e · test Authorized CIDRs Connectivity should allow API access only from authorized VM IP · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:88]: failed to create customer resources for authorized CIDRs cluster Unexpected error: <fmt.wrapError \| 0xc000f50080>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cidr-connectivity-test-qqk9l9" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cidr-connectivity-test-qqk9l9/operationStatuses/08584113806116275101 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-qv6c2vf2b75m/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-wihoigd7cpguy\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-14/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:88]: failed to create customer resources for authorized CIDRs cluster Unexpected error: <fmt.wrapError \| 0xc000f50080>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cidr-connectivity-test-qqk9l9" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cidr-connectivity-test-qqk9l9/operationStatuses/08584113806116275101 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-qv6c2vf2b75m/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-wihoigd7cpguy\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-14/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 04:47:20 UTC · failed at e2e · test Customer should be able to create an HCP cluster with custom autoscaling and update the autoscaling configuration · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:97]: failed to create customer resources for autoscaling cluster Unexpected error: <fmt.wrapError \| 0xc000f14380>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-autoscaling-hcp-cluster-r9t4tf" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-autoscaling-hcp-cluster-r9t4tf/operationStatuses/08584113806116046651 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/autoscaling-cluster-r7frhbmt8bk7/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-e4ixwyxsfjnje\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-00/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:97]: failed to create customer resources for autoscaling cluster Unexpected error: <fmt.wrapError \| 0xc000f14380>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-autoscaling-hcp-cluster-r9t4tf" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-autoscaling-hcp-cluster-r9t4tf/operationStatuses/08584113806116046651 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/autoscaling-cluster-r7frhbmt8bk7/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-e4ixwyxsfjnje\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-00/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 04:47:20 UTC · failed at e2e · test Customer should be able to create an HCP cluster then delete it by deleting the customer resource group · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delete_cx_rg.go:80]: failed to create customer resources for cluster "cx-rg-hcp-cluster" Unexpected error: <fmt.wrapError \| 0xc001284cc0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cx-rg-hcp-cluster-vctv7k" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cx-rg-hcp-cluster-vctv7k/operationStatuses/08584113806113617110 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-23/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/cx-rg-cluster-hbn74zshqfgs/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-fmxs36wu5sow2\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delete_cx_rg.go:80]: failed to create customer resources for cluster "cx-rg-hcp-cluster" Unexpected error: <fmt.wrapError \| 0xc001284cc0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cx-rg-hcp-cluster-vctv7k" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cx-rg-hcp-cluster-vctv7k/operationStatuses/08584113806113617110 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-23/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/cx-rg-cluster-hbn74zshqfgs/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-fmxs36wu5sow2\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 04:47:20 UTC · failed at e2e · test Customer should be able to create a cluster with default autoscaling and a nodepool with autoscaling enabled up to replica limits · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:93]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc0010216e0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-np-autoscale-cluster-szsxcs" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-np-autoscale-cluster-szsxcs/operationStatuses/08584113803873470718 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/np-autoscaling-skcrdsr4h57j/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-ssrwqrnhyyabs\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-22/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:93]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc0010216e0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-np-autoscale-cluster-szsxcs" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-np-autoscale-cluster-szsxcs/operationStatuses/08584113803873470718 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/np-autoscaling-skcrdsr4h57j/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-ssrwqrnhyyabs\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-22/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 04:47:20 UTC · failed at e2e · test Customer should be able to create an HCP cluster with Image Registry not present · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/image_registry_cluster_create.go:79]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000c53600>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-disabled-image-registry-hcp-cluster-knj66q" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-disabled-image-registry-hcp-cluster-knj66q/operationStatuses/08584113803896599497 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-20/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/disabled-image-registry-5zxmx4mxkgnm/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-qmh74zogaodhg\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/image_registry_cluster_create.go:79]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000c53600>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-disabled-image-registry-hcp-cluster-knj66q" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-disabled-image-registry-hcp-cluster-knj66q/operationStatuses/08584113803896599497 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-20/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/disabled-image-registry-5zxmx4mxkgnm/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-qmh74zogaodhg\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 04:47:20 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 4.22 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:106]: failed to create customer resources for z-stream cluster "cluster-zstream-4-22-dqs5k4" Unexpected error: <fmt.wrapError \| 0xc0014a6d40>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cluster-zstream-4-22-dqs5k4-pqc4k8" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-zstream-4-22-dqs5k4-pqc4k8/operationStatuses/08584113804282173996 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-22-rrxd587pzlwh/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-76eghfhmal6jk\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-10/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:106]: failed to create customer resources for z-stream cluster "cluster-zstream-4-22-dqs5k4" Unexpected error: <fmt.wrapError \| 0xc0014a6d40>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cluster-zstream-4-22-dqs5k4-pqc4k8" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-zstream-4-22-dqs5k4-pqc4k8/operationStatuses/08584113804282173996 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-22-rrxd587pzlwh/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-76eghfhmal6jk\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-10/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 04:47:20 UTC · failed at e2e · test Update HCPOpenShiftCluster Negative creates a cluster and fails to update its name with a PATCH request · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:74]: failed to create customer resources for patch-name cluster Unexpected error: <fmt.wrapError \| 0xc000b1a5c0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-patch-name-cluster-q7xjnw" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-patch-name-cluster-q7xjnw/operationStatuses/08584113806111250765 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-10/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-name-cfmdxs2h5mz4/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-rr7tdjqoygobu\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:74]: failed to create customer resources for patch-name cluster Unexpected error: <fmt.wrapError \| 0xc000b1a5c0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-patch-name-cluster-q7xjnw" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-patch-name-cluster-q7xjnw/operationStatuses/08584113806111250765 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-10/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-name-cfmdxs2h5mz4/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-rr7tdjqoygobu\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 04:47:20 UTC · failed at e2e · test Customer should be able to successfully upgrade control plane minor version from 4.21 minor to 4.22 minor · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:123]: failed to create customer resources for y-stream upgrade cluster "cp-ystream-upgrade-4-22-5hfq9c" Unexpected error: <fmt.wrapError \| 0xc001012860>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cp-ystream-upgrade-4-22-5hfq9c-gktmn8" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cp-ystream-upgrade-4-22-5hfq9c-gktmn8/operationStatuses/08584113804128623031 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cp-ystream-upgrade-4-22-k5skrl4wfdrn/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-b7qhh3pt3l6g4\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-00/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:123]: failed to create customer resources for y-stream upgrade cluster "cp-ystream-upgrade-4-22-5hfq9c" Unexpected error: <fmt.wrapError \| 0xc001012860>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cp-ystream-upgrade-4-22-5hfq9c-gktmn8" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cp-ystream-upgrade-4-22-5hfq9c-gktmn8/operationStatuses/08584113804128623031 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cp-ystream-upgrade-4-22-k5skrl4wfdrn/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-b7qhh3pt3l6g4\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-00/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 04:47:20 UTC · failed at e2e · test Customer should upgrade and update a nodepool from 4.20.z to 4.21.zLatest · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:136]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc001000780>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-np-version-upgrade-cluster-fgw5w2-9mhxrf" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-np-version-upgrade-cluster-fgw5w2-9mhxrf/operationStatuses/08584113803282955814 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-version-upgrade-fgw5w2-djf5mhdkmtxn/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-fgbekhquzb3iq\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-06/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:136]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc001000780>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-np-version-upgrade-cluster-fgw5w2-9mhxrf" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-np-version-upgrade-cluster-fgw5w2-9mhxrf/operationStatuses/08584113803282955814 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-version-upgrade-fgw5w2-djf5mhdkmtxn/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-fgbekhquzb3iq\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-06/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 04:47:20 UTC · failed at e2e · test Customer should downgrade a nodepool version z-stream downgrade from 4.21.zLatest to 4.21.zPrevious · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:570]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc0004b8700>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-np-downgrade-rsjgrk-w7nqbj" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-np-downgrade-rsjgrk-w7nqbj/operationStatuses/08584113802912270287 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-downgrade-rsjgrk-crbww5k2htkx/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-tgmdzqh5zlmc2\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-03/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:570]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc0004b8700>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-np-downgrade-rsjgrk-w7nqbj" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-np-downgrade-rsjgrk-w7nqbj/operationStatuses/08584113802912270287 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-downgrade-rsjgrk-crbww5k2htkx/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-tgmdzqh5zlmc2\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-03/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 04:47:20 UTC · failed at e2e · test Customer should be able to update node pool labels and taints · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_labels_taints.go:82]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc001690280>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cluster-np-labels-taints-4spgtm" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-np-labels-taints-4spgtm/operationStatuses/08584113803454899552 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-13/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-labels-taints-jm8859lxjth9/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-r6cuakilyx67k\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_labels_taints.go:82]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc001690280>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cluster-np-labels-taints-4spgtm" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-np-labels-taints-4spgtm/operationStatuses/08584113803454899552 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-13/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-labels-taints-jm8859lxjth9/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-r6cuakilyx67k\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 04:47:20 UTC · failed at e2e · test Customer should upgrade a nodepool skipping one minor version (+2) from 4.20.z to 4.22.zLatest · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:431]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000c89200>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-np-skip-minor-gx8gpn-ntvf2f" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-np-skip-minor-gx8gpn-ntvf2f/operationStatuses/08584113802994503918 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-01/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-skip-minor-gx8gpn-6np5x59hjxbc/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-ikzua3x7s327g\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:431]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000c89200>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-np-skip-minor-gx8gpn-ntvf2f" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-np-skip-minor-gx8gpn-ntvf2f/operationStatuses/08584113802994503918 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-01/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-skip-minor-gx8gpn-6np5x59hjxbc/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-ikzua3x7s327g\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 04:47:20 UTC · failed at e2e · test Customer should create a cluster with private ingress using v20260901preview and verify the ingress is internal · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_ingress.go:77]: failed to create customer resources for private ingress cluster Unexpected error: <fmt.wrapError \| 0xc000e19840>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-private-ingress-s6brt9" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-private-ingress-s6brt9/operationStatuses/08584113806112848721 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/private-ingress-rvz7ml6rm6pn/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-6rnsb3kbhjy5m\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-12/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_ingress.go:77]: failed to create customer resources for private ingress cluster Unexpected error: <fmt.wrapError \| 0xc000e19840>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-private-ingress-s6brt9" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-private-ingress-s6brt9/operationStatuses/08584113806112848721 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/private-ingress-rvz7ml6rm6pn/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-6rnsb3kbhjy5m\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-12/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 04:47:20 UTC · failed at e2e · test FIPS Mode Support with v2026-09-01 API should create an HCP cluster with FIPS mode enabled via cryptoRestrictions property · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_fips_mode.go:72]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000ce6a60>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-fips-enabled-cluster-hj2qcn" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-fips-enabled-cluster-hj2qcn/operationStatuses/08584113806117189919 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-11/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/fips-enabled-jms9dt5jw97h/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-3h7v3eeeqgjeu\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_fips_mode.go:72]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000ce6a60>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-fips-enabled-cluster-hj2qcn" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-fips-enabled-cluster-hj2qcn/operationStatuses/08584113806117189919 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-11/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/fips-enabled-jms9dt5jw97h/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-3h7v3eeeqgjeu\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 04:47:20 UTC · failed at e2e · test Customer should respect cluster-wide node limits with nodepool autoscaling · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:283]: failed to create cluster customer resources for node-limit test Unexpected error: <fmt.wrapError \| 0xc000c8fe40>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-node-limit-cluster-mkw8wg" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-node-limit-cluster-mkw8wg/operationStatuses/08584113803795290145 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/node-limit-test-tknq4cvl4w4l/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-j6nw3ohx7bzo4\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-23/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:283]: failed to create cluster customer resources for node-limit test Unexpected error: <fmt.wrapError \| 0xc000c8fe40>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-node-limit-cluster-mkw8wg" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-node-limit-cluster-mkw8wg/operationStatuses/08584113803795290145 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/node-limit-test-tknq4cvl4w4l/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-j6nw3ohx7bzo4\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-23/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 04:47:20 UTC · failed at e2e · test Customer should be able to create node pools with ARM64-based VMs · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/arm64_nodepool.go:78]: failed to create customer resources for ARM64 nodepool cluster Unexpected error: <fmt.wrapError \| 0xc0015107a0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-arm64-vm-hcp-cluster-2d4bf7" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-arm64-vm-hcp-cluster-2d4bf7/operationStatuses/08584113806114468713 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/arm64-vm-cluster-wrdgbrcf2mft/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-6htyoibevj7ye\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-03/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/arm64_nodepool.go:78]: failed to create customer resources for ARM64 nodepool cluster Unexpected error: <fmt.wrapError \| 0xc0015107a0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-arm64-vm-hcp-cluster-2d4bf7" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-arm64-vm-hcp-cluster-2d4bf7/operationStatuses/08584113806114468713 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/arm64-vm-cluster-wrdgbrcf2mft/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-6htyoibevj7ye\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-03/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 04:47:20 UTC · failed at e2e · test Customer should be able to retrieve cluster and nodepool status active versions · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_nodepool_active_versions.go:73]: failed to create customer resources for v20261001preview cluster Unexpected error: <fmt.wrapError \| 0xc000515120>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-v20261001-tcn8tt" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-v20261001-tcn8tt/operationStatuses/08584113806116665011 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/v20261001preview-nkwfqtlk4dqg/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-7e767rdphst3w\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-01/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_nodepool_active_versions.go:73]: failed to create customer resources for v20261001preview cluster Unexpected error: <fmt.wrapError \| 0xc000515120>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-v20261001-tcn8tt" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-v20261001-tcn8tt/operationStatuses/08584113806116665011 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/v20261001preview-nkwfqtlk4dqg/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-7e767rdphst3w\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-01/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 04:47:20 UTC · failed at e2e · test Customer should upgrade and update a nodepool from 4.20.z to 4.20.zLatest · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:136]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc0008a27a0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-np-version-upgrade-cluster-8l7wc7-twjjnz" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-np-version-upgrade-cluster-8l7wc7-twjjnz/operationStatuses/08584113803024332837 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-02/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-version-upgrade-8l7wc7-jq8k2zdmzm65/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-cltijclxbxmue\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:136]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc0008a27a0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-np-version-upgrade-cluster-8l7wc7-twjjnz" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-np-version-upgrade-cluster-8l7wc7-twjjnz/operationStatuses/08584113803024332837 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-02/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-version-upgrade-8l7wc7-jq8k2zdmzm65/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-cltijclxbxmue\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 04:47:20 UTC · failed at e2e · test Create HCPOpenShiftCluster with Private KeyVault should create a cluster with private keyvault using v20251223preview API · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:76]: failed to create customer resources with private key vault Unexpected error: <fmt.wrapError \| 0xc0011bc1e0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-private-kv-cluster-rbg7tm" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-private-kv-cluster-rbg7tm/operationStatuses/08584113805186906568 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/private-keyvault-t8475hwxlvk4/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-had2isw6vkgre\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-24/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:76]: failed to create customer resources with private key vault Unexpected error: <fmt.wrapError \| 0xc0011bc1e0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-private-kv-cluster-rbg7tm" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-private-kv-cluster-rbg7tm/operationStatuses/08584113805186906568 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/private-keyvault-t8475hwxlvk4/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-had2isw6vkgre\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-24/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 04:47:20 UTC · failed at e2e · test Customer should be able to create an HCP cluster and manage ImageDigestMirrors · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/idms_lifecycle.go:90]: failed to create customer resources for IDMS cluster Unexpected error: <fmt.wrapError \| 0xc000e57d80>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-idms-e2e-hcp-cluster-nc2q4j" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-idms-e2e-hcp-cluster-nc2q4j/operationStatuses/08584113803895991084 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-19/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/idms-mdbl6wxhbzjx/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-lv7dvzspd2e5i\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/idms_lifecycle.go:90]: failed to create customer resources for IDMS cluster Unexpected error: <fmt.wrapError \| 0xc000e57d80>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-idms-e2e-hcp-cluster-nc2q4j" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-idms-e2e-hcp-cluster-nc2q4j/operationStatuses/08584113803895991084 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-19/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/idms-mdbl6wxhbzjx/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-lv7dvzspd2e5i\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 04:47:20 UTC · failed at e2e · test Customer should be able to create an HCP cluster and manage pull secrets · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_pullsecret.go:104]: failed to create customer resources for pull secret cluster Unexpected error: <fmt.wrapError \| 0xc000b32720>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-pullsecret-hcp-cluster-7w52q9" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-pullsecret-hcp-cluster-7w52q9/operationStatuses/08584113806109025158 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/pullsecret-test-v22dzqxkj6qc/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-p2h32vyw2l2ma\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-07/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_pullsecret.go:104]: failed to create customer resources for pull secret cluster Unexpected error: <fmt.wrapError \| 0xc000b32720>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-pullsecret-hcp-cluster-7w52q9" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-pullsecret-hcp-cluster-7w52q9/operationStatuses/08584113806109025158 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/pullsecret-test-v22dzqxkj6qc/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-p2h32vyw2l2ma\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-07/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 04:47:20 UTC · failed at e2e · test Nodepool Ephemeral OS Disk should create a nodepool with ephemeral OS disk when autoRepair is enabled · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_ephemeral_osdisk.go:75]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000894a80>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-ephemeral-disk-f9x9n9" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-ephemeral-disk-f9x9n9/operationStatuses/08584113803560812998 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/ephemeral-osdisk-qsvvp4zd6rfh/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-6xts4ycasve7q\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-04/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_ephemeral_osdisk.go:75]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000894a80>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-ephemeral-disk-f9x9n9" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-ephemeral-disk-f9x9n9/operationStatuses/08584113803560812998 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/ephemeral-osdisk-qsvvp4zd6rfh/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-6xts4ycasve7q\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-04/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 04:47:20 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 5.0 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:106]: failed to create customer resources for z-stream cluster "cluster-zstream-5-0-ncrdgm" Unexpected error: <fmt.wrapError \| 0xc00032fc40>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cluster-zstream-5-0-ncrdgm-fbq9mg" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-zstream-5-0-ncrdgm-fbq9mg/operationStatuses/08584113804280814283 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-5-0-rfdmx8dkwsjv/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-4vtavx33era4k\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-11/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:106]: failed to create customer resources for z-stream cluster "cluster-zstream-5-0-ncrdgm" Unexpected error: <fmt.wrapError \| 0xc00032fc40>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cluster-zstream-5-0-ncrdgm-fbq9mg" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-zstream-5-0-ncrdgm-fbq9mg/operationStatuses/08584113804280814283 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-5-0-rfdmx8dkwsjv/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-4vtavx33era4k\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-11/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 04:47:20 UTC · failed at e2e · test Customer should be able to lifecycle and confirm external auth on a cluster · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_list_and_verify.go:78]: failed to create customer resources for external auth list test Unexpected error: <fmt.wrapError \| 0xc000c0eae0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-ea-list9dpjtf-jf5xkr" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-ea-list9dpjtf-jf5xkr/operationStatuses/08584113803972375281 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-18/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/ea-list-64sp6cvpwd29/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-adn3hfaas23n6\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_list_and_verify.go:78]: failed to create customer resources for external auth list test Unexpected error: <fmt.wrapError \| 0xc000c0eae0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-ea-list9dpjtf-jf5xkr" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-ea-list9dpjtf-jf5xkr/operationStatuses/08584113803972375281 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-18/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/ea-list-64sp6cvpwd29/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-adn3hfaas23n6\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 04:47:20 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.20 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:112]: failed to create customer resources for cluster "cluster-candidate-4-20-xhggfs" Unexpected error: <fmt.wrapError \| 0xc0010561e0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cluster-candidate-4-20-xhggfs-2wkbx6" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-candidate-4-20-xhggfs-2wkbx6/operationStatuses/08584113804398595348 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-4-20-4qk5llt4fz6d/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-a7jlkrfbfsqok\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-09/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:112]: failed to create customer resources for cluster "cluster-candidate-4-20-xhggfs" Unexpected error: <fmt.wrapError \| 0xc0010561e0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cluster-candidate-4-20-xhggfs-2wkbx6" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-candidate-4-20-xhggfs-2wkbx6/operationStatuses/08584113804398595348 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-4-20-4qk5llt4fz6d/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-a7jlkrfbfsqok\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-09/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 04:47:20 UTC · failed at e2e · test Customer should be able to create an HCP cluster and custom node pool osDisk size · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_nodepool_osdisk.go:71]: failed to create customer resources for cluster "hcp-cluster-np-128" Unexpected error: <fmt.wrapError \| 0xc0010f8700>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-hcp-cluster-np-128-j574p2" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-hcp-cluster-np-128-j574p2/operationStatuses/08584113806104875981 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-06/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/clusternp128-kglphbbzhqst/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-3ljyjjx2n7gxo\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_nodepool_osdisk.go:71]: failed to create customer resources for cluster "hcp-cluster-np-128" Unexpected error: <fmt.wrapError \| 0xc0010f8700>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-hcp-cluster-np-128-j574p2" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-hcp-cluster-np-128-j574p2/operationStatuses/08584113806104875981 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-06/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/clusternp128-kglphbbzhqst/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-3ljyjjx2n7gxo\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 04:47:20 UTC · failed at e2e · test Customer should be able to issue the default ingress serving certificate from Let's Encrypt using cert-manager with ACME DNS-01 challenges · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_ingress_cert_letsencrypt.go:114]: failed to create customer resources Unexpected error: <fmt.wrapError \| 0xc0019897c0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-ingress-cert-le-s6pz6g" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-ingress-cert-le-s6pz6g/operationStatuses/08584113806109906954 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/ingress-cert-le-jhxdz7ppd98q/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-nhimtfs3fueps\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-20/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_ingress_cert_letsencrypt.go:114]: failed to create customer resources Unexpected error: <fmt.wrapError \| 0xc0019897c0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-ingress-cert-le-s6pz6g" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-ingress-cert-le-s6pz6g/operationStatuses/08584113806109906954 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/ingress-cert-le-jhxdz7ppd98q/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-nhimtfs3fueps\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-20/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 04:47:20 UTC · failed at e2e · test Customer should create a cluster using v20260901preview API and verify cluster health · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_v20260901preview.go:70]: failed to create customer resources for v20260901preview cluster Unexpected error: <fmt.wrapError \| 0xc000820260>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-v20260901-dp7qzd" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-v20260901-dp7qzd/operationStatuses/08584113806116691335 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/v20260901preview-zrfd45q4c72q/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-xdteugpxef23k\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-09/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_v20260901preview.go:70]: failed to create customer resources for v20260901preview cluster Unexpected error: <fmt.wrapError \| 0xc000820260>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-v20260901-dp7qzd" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-v20260901-dp7qzd/operationStatuses/08584113806116691335 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/v20260901preview-zrfd45q4c72q/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-xdteugpxef23k\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-09/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 04:47:20 UTC · failed at e2e · test Customer should be able to successfully upgrade control plane minor version from 4.22 minor to 4.23 minor · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:123]: failed to create customer resources for y-stream upgrade cluster "cp-ystream-upgrade-4-23-ct2xd4" Unexpected error: <fmt.wrapError \| 0xc000f810c0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cp-ystream-upgrade-4-23-ct2xd4-2pfhjt" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cp-ystream-upgrade-4-23-ct2xd4-2pfhjt/operationStatuses/08584113804018242618 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cp-ystream-upgrade-4-23-5jtfxzb57jbp/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-oko5an6hosdhk\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-14/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:123]: failed to create customer resources for y-stream upgrade cluster "cp-ystream-upgrade-4-23-ct2xd4" Unexpected error: <fmt.wrapError \| 0xc000f810c0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cp-ystream-upgrade-4-23-ct2xd4-2pfhjt" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cp-ystream-upgrade-4-23-ct2xd4-2pfhjt/operationStatuses/08584113804018242618 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cp-ystream-upgrade-4-23-5jtfxzb57jbp/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-oko5an6hosdhk\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-14/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 04:47:20 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 4.20 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:106]: failed to create customer resources for z-stream cluster "cluster-zstream-4-20-7zmhsv" Unexpected error: <fmt.wrapError \| 0xc000a0c420>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cluster-zstream-4-20-7zmhsv-7scgzz" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-zstream-4-20-7zmhsv-7scgzz/operationStatuses/08584113804277874316 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-20-tr2g922h2hqn/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-4udo43sc7vttm\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-07/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:106]: failed to create customer resources for z-stream cluster "cluster-zstream-4-20-7zmhsv" Unexpected error: <fmt.wrapError \| 0xc000a0c420>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cluster-zstream-4-20-7zmhsv-7scgzz" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-zstream-4-20-7zmhsv-7scgzz/operationStatuses/08584113804277874316 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-20-tr2g922h2hqn/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-4udo43sc7vttm\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-07/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 04:47:20 UTC · failed at e2e · test Customer should create an HCP cluster and validate TLS certificates · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:87]: failed to create customer resources for TLS endpoint cluster Unexpected error: <fmt.wrapError \| 0xc0006dda40>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-tls-endpoint-hcp-cluster-79w9fh" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-tls-endpoint-hcp-cluster-79w9fh/operationStatuses/08584113806216845740 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-08/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/tls-endpoint-cluster-h527s8x62fj7/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-oo7pyvaeowk44\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:87]: failed to create customer resources for TLS endpoint cluster Unexpected error: <fmt.wrapError \| 0xc0006dda40>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-tls-endpoint-hcp-cluster-79w9fh" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-tls-endpoint-hcp-cluster-79w9fh/operationStatuses/08584113806216845740 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-08/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/tls-endpoint-cluster-h527s8x62fj7/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-oo7pyvaeowk44\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 04:47:20 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.22 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:112]: failed to create customer resources for cluster "cluster-candidate-4-22-fxdz9q" Unexpected error: <fmt.wrapError \| 0xc001018360>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cluster-candidate-4-22-fxdz9q-vhkwqs" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-candidate-4-22-fxdz9q-vhkwqs/operationStatuses/08584113804540699539 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-4-22-rcz2tr8hxdph/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-iqn6stss77wf6\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-02/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:112]: failed to create customer resources for cluster "cluster-candidate-4-22-fxdz9q" Unexpected error: <fmt.wrapError \| 0xc001018360>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cluster-candidate-4-22-fxdz9q-vhkwqs" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-candidate-4-22-fxdz9q-vhkwqs/operationStatuses/08584113804540699539 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-4-22-rcz2tr8hxdph/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-iqn6stss77wf6\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-02/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 04:47:20 UTC · failed at e2e · test Customer should be able to use workload identity via the cluster OIDC issuer URL · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/oidc_issuer_workload_identity.go:218]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000f99d00>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-oidc-wi-cluster-2xqhrd" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-oidc-wi-cluster-2xqhrd/operationStatuses/08584113802810164928 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-11/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/oidc-wi-s7nl4n9bv5dn/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-s6kwp7xg43dbi\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/oidc_issuer_workload_identity.go:218]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000f99d00>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-oidc-wi-cluster-2xqhrd" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-oidc-wi-cluster-2xqhrd/operationStatuses/08584113802810164928 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-11/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/oidc-wi-s7nl4n9bv5dn/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-s6kwp7xg43dbi\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 04:47:20 UTC · failed at e2e · test Customer should be able to create a HCP cluster and use cilium CNI plugin · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_cni_cilium.go:78]: failed to create customer resources for cilium CNI cluster Unexpected error: <fmt.wrapError \| 0xc0010f48e0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cilium-cl-8ps7nq" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cilium-cl-8ps7nq/operationStatuses/08584113806111815531 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/cni-cilium-xf8c9fwbfv9z/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-dve5f7wncegny\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-21/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_cni_cilium.go:78]: failed to create customer resources for cilium CNI cluster Unexpected error: <fmt.wrapError \| 0xc0010f48e0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cilium-cl-8ps7nq" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cilium-cl-8ps7nq/operationStatuses/08584113806111815531 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/cni-cilium-xf8c9fwbfv9z/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-dve5f7wncegny\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-21/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 04:47:20 UTC · failed at e2e · test Customer should be able to create a no-CNI private cluster with a private key vault, a nodepool and install cilium CNI successfully · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:84]: failed to create customer resources with private key vault Unexpected error: <fmt.wrapError \| 0xc001218300>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cilium-cluster-5pfptn" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cilium-cluster-5pfptn/operationStatuses/08584113805189245935 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-04/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/complex-cilium-kv-z5mnjxckjl2l/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-4vnk4izirdfzu\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:84]: failed to create customer resources with private key vault Unexpected error: <fmt.wrapError \| 0xc001218300>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cilium-cluster-5pfptn" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cilium-cluster-5pfptn/operationStatuses/08584113805189245935 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-04/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/complex-cilium-kv-z5mnjxckjl2l/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-4vnk4izirdfzu\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 04:47:20 UTC · failed at e2e · test Customer should be able to cordon, drain, and uncordon a node in an HCP cluster · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_cordon.go:80]: failed to create cluster customer resources for cordon test Unexpected error: <fmt.wrapError \| 0xc00070df60>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cordon-hcp-cluster-tkt7nm" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cordon-hcp-cluster-tkt7nm/operationStatuses/08584113803709711286 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cordon-test-lxhcqp5g8zsj/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-3wbuyr3ac6kzo\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-05/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_cordon.go:80]: failed to create cluster customer resources for cordon test Unexpected error: <fmt.wrapError \| 0xc00070df60>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cordon-hcp-cluster-tkt7nm" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cordon-hcp-cluster-tkt7nm/operationStatuses/08584113803709711286 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cordon-test-lxhcqp5g8zsj/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-3wbuyr3ac6kzo\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-05/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 04:47:20 UTC · failed at e2e · test Customer should downgrade a nodepool to a lower minor version from 4.22.zLatest to 4.20.zLatest · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:710]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000697420>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-np-dg-minor-ng7fj6-gx6vsk" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-np-dg-minor-ng7fj6-gx6vsk/operationStatuses/08584113802810199751 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-dg-minor-ng7fj6-q2lfhm7n5bk7/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-4k3ew4a3xlt2e\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-10/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:710]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000697420>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-np-dg-minor-ng7fj6-gx6vsk" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-np-dg-minor-ng7fj6-gx6vsk/operationStatuses/08584113802810199751 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-dg-minor-ng7fj6-q2lfhm7n5bk7/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-4k3ew4a3xlt2e\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-10/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 04:47:20 UTC · failed at e2e · test Customer should be able to create an HCP cluster with back-level version 4.20 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:121]: failed to deploy managed identities for back-level version 4.20 Unexpected error: <fmt.wrapError \| 0xc001109220>: failed waiting for deployment "mi-cluster-ver-4-20-sv2j7p" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-ver-4-20-sv2j7p/operationStatuses/08584113804722825144 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-06/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cluster-back-version-mgn4698wxq98/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:121]: failed to deploy managed identities for back-level version 4.20 Unexpected error: <fmt.wrapError \| 0xc001109220>: failed waiting for deployment "mi-cluster-ver-4-20-sv2j7p" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-ver-4-20-sv2j7p/operationStatuses/08584113804722825144 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-06/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cluster-back-version-mgn4698wxq98/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 04:47:20 UTC · failed at e2e · test Customer should upgrade a nodepool to a version without Cincinnati upgrade edge z-stream upgrade without Cincinnati edge in 4.20 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:291]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000e81ee0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-np-noedge-gn68jq-lvkpvg" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-np-noedge-gn68jq-lvkpvg/operationStatuses/08584113803021091151 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-noedge-gn68jq-sz2zh9b526g4/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-ij42gficifcsk\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-21/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:291]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000e81ee0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-np-noedge-gn68jq-lvkpvg" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-np-noedge-gn68jq-lvkpvg/operationStatuses/08584113803021091151 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-noedge-gn68jq-sz2zh9b526g4/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-ij42gficifcsk\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-21/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 04:47:20 UTC · failed at e2e · test Customer should be able to list available HCP OpenShift versions and validate response content · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_versions.go:43]: failed to list HCP OpenShift versions Unexpected error: <exported.ResponseError \| 0xc000ce62a0>: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOpenShiftVersions -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_versions.go:43]: failed to list HCP OpenShift versions Unexpected error: <exported.ResponseError \| 0xc000ce62a0>: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOpenShiftVersions -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 04:47:20 UTC · failed at e2e · test Customer should be able to forward control plane logs to a storage account and Event Hub via shoebox diagnostic settings · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/shoebox_logs.go:268]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000d6a780>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-shoebox-hcp-cluster-dvbk2t" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-shoebox-hcp-cluster-dvbk2t/operationStatuses/08584113802808793038 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/shoebox-cluster-6jlj7g4z6pp2/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-22eudp3sazqeq\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-09/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/shoebox_logs.go:268]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000d6a780>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-shoebox-hcp-cluster-dvbk2t" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-shoebox-hcp-cluster-dvbk2t/operationStatuses/08584113802808793038 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/shoebox-cluster-6jlj7g4z6pp2/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-22eudp3sazqeq\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-09/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 04:47:20 UTC · failed at e2e · test Customer should create a cluster with private KAS and verify API server is only reachable from VNet while ingress remains public · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:77]: failed to create customer resources for private KAS cluster Unexpected error: <fmt.wrapError \| 0xc0004ee660>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-private-kas-s9c9qb" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-private-kas-s9c9qb/operationStatuses/08584113806116772609 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/private-kas-f6vgfpj5z76c/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-jxdbnndsp5js2\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-22/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:77]: failed to create customer resources for private KAS cluster Unexpected error: <fmt.wrapError \| 0xc0004ee660>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-private-kas-s9c9qb" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-private-kas-s9c9qb/operationStatuses/08584113806116772609 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/private-kas-f6vgfpj5z76c/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-jxdbnndsp5js2\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-22/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 04:47:20 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.21 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:112]: failed to create customer resources for cluster "cluster-candidate-4-21-pslr2c" Unexpected error: <fmt.wrapError \| 0xc000c8fa40>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cluster-candidate-4-21-pslr2c-g2769k" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-candidate-4-21-pslr2c-g2769k/operationStatuses/08584113804400098279 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-4-21-5lr6bhjf8l8d/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-fmduqqi6yc6ls\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-01/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:112]: failed to create customer resources for cluster "cluster-candidate-4-21-pslr2c" Unexpected error: <fmt.wrapError \| 0xc000c8fa40>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cluster-candidate-4-21-pslr2c-g2769k" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-candidate-4-21-pslr2c-g2769k/operationStatuses/08584113804400098279 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-4-21-5lr6bhjf8l8d/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-fmduqqi6yc6ls\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-01/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 04:47:20 UTC · failed at e2e · test Customer should be able to successfully upgrade control plane minor version from 4.20 minor to 4.21 minor · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:123]: failed to create customer resources for y-stream upgrade cluster "cp-ystream-upgrade-4-21-69w4xz" Unexpected error: <fmt.wrapError \| 0xc0010734c0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cp-ystream-upgrade-4-21-69w4xz-2th7h6" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cp-ystream-upgrade-4-21-69w4xz-2th7h6/operationStatuses/08584113804026603480 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-15/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cp-ystream-upgrade-4-21-97lmjm8gbhml/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-evvqr3vrpmzr2\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:123]: failed to create customer resources for y-stream upgrade cluster "cp-ystream-upgrade-4-21-69w4xz" Unexpected error: <fmt.wrapError \| 0xc0010734c0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cp-ystream-upgrade-4-21-69w4xz-2th7h6" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cp-ystream-upgrade-4-21-69w4xz-2th7h6/operationStatuses/08584113804026603480 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-15/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cp-ystream-upgrade-4-21-97lmjm8gbhml/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-evvqr3vrpmzr2\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 04:47:20 UTC · failed at e2e · test Customer should be able to successfully upgrade control plane minor version from 4.22 minor to 5.0 minor · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:123]: failed to create customer resources for y-stream upgrade cluster "cp-ystream-upgrade-5-0-2lhggq" Unexpected error: <fmt.wrapError \| 0xc000bd9620>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cp-ystream-upgrade-5-0-2lhggq-7dffjn" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cp-ystream-upgrade-5-0-2lhggq-7dffjn/operationStatuses/08584113804022878880 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cp-ystream-upgrade-5-0-fqhpj4gfndmh/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-zmtivx2uqqptm\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-16/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:123]: failed to create customer resources for y-stream upgrade cluster "cp-ystream-upgrade-5-0-2lhggq" Unexpected error: <fmt.wrapError \| 0xc000bd9620>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cp-ystream-upgrade-5-0-2lhggq-7dffjn" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cp-ystream-upgrade-5-0-2lhggq-7dffjn/operationStatuses/08584113804022878880 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cp-ystream-upgrade-5-0-fqhpj4gfndmh/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-zmtivx2uqqptm\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-16/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 04:47:20 UTC · failed at e2e · test ARO HCP Service should successfully provision cluster when MI role assignments are added after cluster creation · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:140]: failed to start cluster creation Unexpected error: <exported.ResponseError \| 0xc000ef6450>: PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/delayed-rbac-8zb6m4wbcswd/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/delayed-rbac-cluster -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:140]: failed to start cluster creation Unexpected error: <exported.ResponseError \| 0xc000ef6450>: PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/delayed-rbac-8zb6m4wbcswd/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/delayed-rbac-cluster -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 04:47:20 UTC · failed at e2e · test Customer should not be able to reuse subnets and NSGs between clusters · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_nsg_subnet_reuse.go:97]: failed to create customer resources for basic-cluster Unexpected error: <fmt.wrapError \| 0xc0004bf3a0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-basic-cluster-r5dnns" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-basic-cluster-r5dnns/operationStatuses/08584113806114250372 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cluster-nsg-subnet-reuse-8czpd2bttnxj/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-j3gkygvtvkvjs\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-16/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_nsg_subnet_reuse.go:97]: failed to create customer resources for basic-cluster Unexpected error: <fmt.wrapError \| 0xc0004bf3a0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-basic-cluster-r5dnns" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-basic-cluster-r5dnns/operationStatuses/08584113806114250372 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cluster-nsg-subnet-reuse-8czpd2bttnxj/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-j3gkygvtvkvjs\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-16/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 04:47:20 UTC · failed at e2e · test Customer should be able to create several HCP clusters in their customer resource group, but not in the same managed resource group · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:81]: failed to create customer resources for first cluster "basic-hcp-cluster" Unexpected error: <fmt.wrapError \| 0xc0010fb4e0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-basic-hcp-cluster-52qtxz" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-basic-hcp-cluster-52qtxz/operationStatuses/08584113806115129746 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/customer-rg-4fxgzntt4rxq/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-eqbgltuyzhpg4\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-17/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:81]: failed to create customer resources for first cluster "basic-hcp-cluster" Unexpected error: <fmt.wrapError \| 0xc0010fb4e0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-basic-hcp-cluster-52qtxz" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-basic-hcp-cluster-52qtxz/operationStatuses/08584113806115129746 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/customer-rg-4fxgzntt4rxq/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-eqbgltuyzhpg4\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-17/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 04:47:20 UTC · failed at e2e · test Update HCPOpenShiftCluster Positive creates a cluster and updates tags with a PATCH request · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:151]: failed to create customer resources for patch-tags cluster Unexpected error: <fmt.wrapError \| 0xc0006ef8c0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-patch-tags-cluster-ntlx9n" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-patch-tags-cluster-ntlx9n/operationStatuses/08584113804793509594 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-tags-mntvs5bfs2vm/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-fklo75htpxzxc\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-08/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:151]: failed to create customer resources for patch-tags cluster Unexpected error: <fmt.wrapError \| 0xc0006ef8c0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-patch-tags-cluster-ntlx9n" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-patch-tags-cluster-ntlx9n/operationStatuses/08584113804793509594 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-tags-mntvs5bfs2vm/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-fklo75htpxzxc\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-08/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 04:47:20 UTC · failed at e2e · test Customer should be able to delete an HCP cluster whose managed identities were deleted first · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delete_missing_identities.go:101]: failed to deploy dedicated managed identities Unexpected error: <fmt.wrapError \| 0xc0003577a0>: failed waiting for deployment "mi-missing-mi-hcp-cluster-qbdg8q" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-missing-mi-hcp-cluster-qbdg8q/operationStatuses/08584113806112641677 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/delete-mi-rg-k57vjlqvw6kx/providers/Microsoft.Resources/deployments/clusterNonMsiScopedAssignments-jaooh6lj24n4o\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/delete-mi-rg-k57vjlqvw6kx/providers/Microsoft.Resources/deployments/clusterMsiScopedAssignments-jaooh6lj24n4o\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delete_missing_identities.go:101]: failed to deploy dedicated managed identities Unexpected error: <fmt.wrapError \| 0xc0003577a0>: failed waiting for deployment "mi-missing-mi-hcp-cluster-qbdg8q" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-missing-mi-hcp-cluster-qbdg8q/operationStatuses/08584113806112641677 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/delete-mi-rg-k57vjlqvw6kx/providers/Microsoft.Resources/deployments/clusterNonMsiScopedAssignments-jaooh6lj24n4o\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/delete-mi-rg-k57vjlqvw6kx/providers/Microsoft.Resources/deployments/clusterMsiScopedAssignments-jaooh6lj24n4o\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred
02:00:13	10m 21s	periodic-ci-Azure-ARO-HCP-main-periodic-stage-e2e-parallel post-goodmerged PR No additional run metadata captured	uksouth	e2e	failed	n/a	59	e2e (59) ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 02:00:13 UTC · failed at e2e · test Customer should upgrade and update a nodepool from 4.21.z to 4.21.zLatest · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:136]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000c98200>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-np-version-upgrade-cluster-x48fn7-f4mhn6" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-np-version-upgrade-cluster-x48fn7-f4mhn6/operationStatuses/08584113912430214940 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-11/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-version-upgrade-x48fn7-rx4kn7h7l62p/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-igmhstzhz2tea\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:136]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000c98200>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-np-version-upgrade-cluster-x48fn7-f4mhn6" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-np-version-upgrade-cluster-x48fn7-f4mhn6/operationStatuses/08584113912430214940 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-11/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-version-upgrade-x48fn7-rx4kn7h7l62p/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-igmhstzhz2tea\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 02:00:13 UTC · failed at e2e · test Create HCPOpenShiftCluster with Private KeyVault should create a cluster with private keyvault using v20251223preview API · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:76]: failed to create customer resources with private key vault Unexpected error: <fmt.wrapError \| 0xc001036920>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-private-kv-cluster-2xfb72" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-private-kv-cluster-2xfb72/operationStatuses/08584113914359747457 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-09/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/private-keyvault-5dknb5dd5vlk/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-had2isw6vkgre\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:76]: failed to create customer resources with private key vault Unexpected error: <fmt.wrapError \| 0xc001036920>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-private-kv-cluster-2xfb72" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-private-kv-cluster-2xfb72/operationStatuses/08584113914359747457 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-09/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/private-keyvault-5dknb5dd5vlk/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-had2isw6vkgre\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 02:00:13 UTC · failed at e2e · test Customer should be able to list available HCP OpenShift versions and validate response content · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_versions.go:43]: failed to list HCP OpenShift versions Unexpected error: <exported.ResponseError \| 0xc000d230b0>: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOpenShiftVersions -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_versions.go:43]: failed to list HCP OpenShift versions Unexpected error: <exported.ResponseError \| 0xc000d230b0>: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOpenShiftVersions -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 02:00:13 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.21 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:112]: failed to create customer resources for cluster "cluster-candidate-4-21-49mv7b" Unexpected error: <fmt.wrapError \| 0xc000c1b0c0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cluster-candidate-4-21-49mv7b-blspsw" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-candidate-4-21-49mv7b-blspsw/operationStatuses/08584113913904847366 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-4-21-gkmbwrdjpztq/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-qmbekpo5hsspo\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-24/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:112]: failed to create customer resources for cluster "cluster-candidate-4-21-49mv7b" Unexpected error: <fmt.wrapError \| 0xc000c1b0c0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cluster-candidate-4-21-49mv7b-blspsw" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-candidate-4-21-49mv7b-blspsw/operationStatuses/08584113913904847366 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-4-21-gkmbwrdjpztq/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-qmbekpo5hsspo\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-24/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 02:00:13 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 5.0 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:106]: failed to create customer resources for z-stream cluster "cluster-zstream-5-0-z6ssw4" Unexpected error: <fmt.wrapError \| 0xc000f194e0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cluster-zstream-5-0-z6ssw4-vs425l" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-zstream-5-0-z6ssw4-vs425l/operationStatuses/08584113913637551351 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-5-0-8s727jgnk6zr/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-lwwwvssi7r4ue\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-10/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:106]: failed to create customer resources for z-stream cluster "cluster-zstream-5-0-z6ssw4" Unexpected error: <fmt.wrapError \| 0xc000f194e0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cluster-zstream-5-0-z6ssw4-vs425l" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-zstream-5-0-z6ssw4-vs425l/operationStatuses/08584113913637551351 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-5-0-8s727jgnk6zr/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-lwwwvssi7r4ue\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-10/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 02:00:13 UTC · failed at e2e · test Customer should create a cluster using v20260901preview API and verify cluster health · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_v20260901preview.go:70]: failed to create customer resources for v20260901preview cluster Unexpected error: <fmt.wrapError \| 0xc00079da80>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-v20260901-2xzlq4" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-v20260901-2xzlq4/operationStatuses/08584113915367242561 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/v20260901preview-5b2fj7lf5ptx/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-xdteugpxef23k\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-24/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_v20260901preview.go:70]: failed to create customer resources for v20260901preview cluster Unexpected error: <fmt.wrapError \| 0xc00079da80>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-v20260901-2xzlq4" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-v20260901-2xzlq4/operationStatuses/08584113915367242561 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/v20260901preview-5b2fj7lf5ptx/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-xdteugpxef23k\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-24/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 02:00:13 UTC · failed at e2e · test Customer should not be able to reuse subnets and NSGs between clusters · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_nsg_subnet_reuse.go:97]: failed to create customer resources for basic-cluster Unexpected error: <fmt.wrapError \| 0xc000aa6d20>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-basic-cluster-rmgk9x" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-basic-cluster-rmgk9x/operationStatuses/08584113915254253091 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cluster-nsg-subnet-reuse-mcxckz7xhsq9/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-j3gkygvtvkvjs\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-04/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_nsg_subnet_reuse.go:97]: failed to create customer resources for basic-cluster Unexpected error: <fmt.wrapError \| 0xc000aa6d20>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-basic-cluster-rmgk9x" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-basic-cluster-rmgk9x/operationStatuses/08584113915254253091 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cluster-nsg-subnet-reuse-mcxckz7xhsq9/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-j3gkygvtvkvjs\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-04/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 02:00:13 UTC · failed at e2e · test Customer should be able to create a HCP cluster and use cilium CNI plugin · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_cni_cilium.go:78]: failed to create customer resources for cilium CNI cluster Unexpected error: <fmt.wrapError \| 0xc000ad1980>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cilium-cl-xdnwwz" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cilium-cl-xdnwwz/operationStatuses/08584113915369084851 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/cni-cilium-lrh4jzlp524b/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-dve5f7wncegny\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-11/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_cni_cilium.go:78]: failed to create customer resources for cilium CNI cluster Unexpected error: <fmt.wrapError \| 0xc000ad1980>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cilium-cl-xdnwwz" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cilium-cl-xdnwwz/operationStatuses/08584113915369084851 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/cni-cilium-lrh4jzlp524b/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-dve5f7wncegny\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-11/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 02:00:13 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.20 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:112]: failed to create customer resources for cluster "cluster-candidate-4-20-kk49dv" Unexpected error: <fmt.wrapError \| 0xc000c0e960>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cluster-candidate-4-20-kk49dv-srznwg" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-candidate-4-20-kk49dv-srznwg/operationStatuses/08584113913904019311 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-15/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-4-20-cggfsprbwr4j/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-rksty5frww7dw\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:112]: failed to create customer resources for cluster "cluster-candidate-4-20-kk49dv" Unexpected error: <fmt.wrapError \| 0xc000c0e960>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cluster-candidate-4-20-kk49dv-srznwg" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-candidate-4-20-kk49dv-srznwg/operationStatuses/08584113913904019311 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-15/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-4-20-cggfsprbwr4j/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-rksty5frww7dw\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 02:00:13 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.23 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:112]: failed to create customer resources for cluster "cluster-candidate-4-23-8l7ngb" Unexpected error: <fmt.wrapError \| 0xc000b1db60>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cluster-candidate-4-23-8l7ngb-h5lrt2" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-candidate-4-23-8l7ngb-h5lrt2/operationStatuses/08584113913906278616 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-07/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-4-23-rfcmtkcs67tb/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-ozfqahwps5hom\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:112]: failed to create customer resources for cluster "cluster-candidate-4-23-8l7ngb" Unexpected error: <fmt.wrapError \| 0xc000b1db60>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cluster-candidate-4-23-8l7ngb-h5lrt2" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-candidate-4-23-8l7ngb-h5lrt2/operationStatuses/08584113913906278616 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-07/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-4-23-rfcmtkcs67tb/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-ozfqahwps5hom\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 02:00:13 UTC · failed at e2e · test Customer should upgrade a nodepool to a version without Cincinnati upgrade edge z-stream upgrade without Cincinnati edge in 4.20 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:291]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000dac1e0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-np-noedge-s95n9h-cpl7fc" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-np-noedge-s95n9h-cpl7fc/operationStatuses/08584113912343166379 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-02/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-noedge-s95n9h-mm4b4h66xrsd/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-rgyesixzav3z4\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:291]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000dac1e0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-np-noedge-s95n9h-cpl7fc" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-np-noedge-s95n9h-cpl7fc/operationStatuses/08584113912343166379 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-02/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-noedge-s95n9h-mm4b4h66xrsd/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-rgyesixzav3z4\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 02:00:13 UTC · failed at e2e · test Customer should be able to create an HCP cluster then delete it by deleting the customer resource group · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delete_cx_rg.go:80]: failed to create customer resources for cluster "cx-rg-hcp-cluster" Unexpected error: <fmt.wrapError \| 0xc000f84600>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cx-rg-hcp-cluster-279vtk" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cx-rg-hcp-cluster-279vtk/operationStatuses/08584113915372258288 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/cx-rg-cluster-x7gp5n7qjzxd/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-fmxs36wu5sow2\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-14/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delete_cx_rg.go:80]: failed to create customer resources for cluster "cx-rg-hcp-cluster" Unexpected error: <fmt.wrapError \| 0xc000f84600>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cx-rg-hcp-cluster-279vtk" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cx-rg-hcp-cluster-279vtk/operationStatuses/08584113915372258288 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/cx-rg-cluster-x7gp5n7qjzxd/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-fmxs36wu5sow2\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-14/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 02:00:13 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 4.22 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:106]: failed to create customer resources for z-stream cluster "cluster-zstream-4-22-cnb4hw" Unexpected error: <fmt.wrapError \| 0xc000c20820>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cluster-zstream-4-22-cnb4hw-bwtj8b" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-zstream-4-22-cnb4hw-bwtj8b/operationStatuses/08584113913640193299 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-08/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-22-pfd2lgp68w4g/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-n3ogmkaqsinvk\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:106]: failed to create customer resources for z-stream cluster "cluster-zstream-4-22-cnb4hw" Unexpected error: <fmt.wrapError \| 0xc000c20820>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cluster-zstream-4-22-cnb4hw-bwtj8b" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-zstream-4-22-cnb4hw-bwtj8b/operationStatuses/08584113913640193299 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-08/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-22-pfd2lgp68w4g/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-n3ogmkaqsinvk\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 02:00:13 UTC · failed at e2e · test Customer should downgrade a nodepool to a lower minor version from 4.22.zLatest to 4.20.zLatest · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:710]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc0014fe720>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-np-dg-minor-zdzkqx-qhfmmt" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-np-dg-minor-zdzkqx-qhfmmt/operationStatuses/08584113912216912001 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-dg-minor-zdzkqx-l22p2sw54pdw/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-m4n6shodaasxc\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-06/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:710]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc0014fe720>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-np-dg-minor-zdzkqx-qhfmmt" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-np-dg-minor-zdzkqx-qhfmmt/operationStatuses/08584113912216912001 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-dg-minor-zdzkqx-l22p2sw54pdw/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-m4n6shodaasxc\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-06/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 02:00:13 UTC · failed at e2e · test Customer should create a cluster with private ingress using v20260901preview and verify the ingress is internal · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_ingress.go:77]: failed to create customer resources for private ingress cluster Unexpected error: <fmt.wrapError \| 0xc000bd0200>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-private-ingress-rjpgqd" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-private-ingress-rjpgqd/operationStatuses/08584113915370331446 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/private-ingress-c4nx2jgq7mkh/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-6rnsb3kbhjy5m\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-21/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_ingress.go:77]: failed to create customer resources for private ingress cluster Unexpected error: <fmt.wrapError \| 0xc000bd0200>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-private-ingress-rjpgqd" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-private-ingress-rjpgqd/operationStatuses/08584113915370331446 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/private-ingress-c4nx2jgq7mkh/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-6rnsb3kbhjy5m\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-21/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 02:00:13 UTC · failed at e2e · test Nodepool Ephemeral OS Disk should create a nodepool with ephemeral OS disk when autoRepair is enabled · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_ephemeral_osdisk.go:75]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000963640>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-ephemeral-disk-nr8r8x" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-ephemeral-disk-nr8r8x/operationStatuses/08584113912860313395 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/ephemeral-osdisk-9jsgpbvbxf9r/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-6xts4ycasve7q\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-05/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_ephemeral_osdisk.go:75]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000963640>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-ephemeral-disk-nr8r8x" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-ephemeral-disk-nr8r8x/operationStatuses/08584113912860313395 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/ephemeral-osdisk-9jsgpbvbxf9r/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-6xts4ycasve7q\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-05/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 02:00:13 UTC · failed at e2e · test Customer should be able to create a cluster with default autoscaling and a nodepool with autoscaling enabled up to replica limits · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:93]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc001cd1a80>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-np-autoscale-cluster-59q7qk" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-np-autoscale-cluster-59q7qk/operationStatuses/08584113913180683547 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-23/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/np-autoscaling-9dxvcqrwjfqn/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-ssrwqrnhyyabs\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:93]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc001cd1a80>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-np-autoscale-cluster-59q7qk" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-np-autoscale-cluster-59q7qk/operationStatuses/08584113913180683547 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-23/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/np-autoscaling-9dxvcqrwjfqn/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-ssrwqrnhyyabs\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 02:00:13 UTC · failed at e2e · test Update HCPOpenShiftCluster Negative creates a cluster and fails to update its name with a PATCH request · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:74]: failed to create customer resources for patch-name cluster Unexpected error: <fmt.wrapError \| 0xc00070ba60>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-patch-name-cluster-n55mjb" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-patch-name-cluster-n55mjb/operationStatuses/08584113915366639693 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-22/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:74]: failed to create customer resources for patch-name cluster Unexpected error: <fmt.wrapError \| 0xc00070ba60>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-patch-name-cluster-n55mjb" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-patch-name-cluster-n55mjb/operationStatuses/08584113915366639693 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-22/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 02:00:13 UTC · failed at e2e · test Customer should be able to use workload identity via the cluster OIDC issuer URL · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/oidc_issuer_workload_identity.go:218]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000cf2a00>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-oidc-wi-cluster-qvs74t" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-oidc-wi-cluster-qvs74t/operationStatuses/08584113912059631916 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/oidc-wi-m8svn75dghbq/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-s6kwp7xg43dbi\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-04/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/oidc_issuer_workload_identity.go:218]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000cf2a00>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-oidc-wi-cluster-qvs74t" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-oidc-wi-cluster-qvs74t/operationStatuses/08584113912059631916 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/oidc-wi-m8svn75dghbq/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-s6kwp7xg43dbi\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-04/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 02:00:13 UTC · failed at e2e · test Customer should create an HCP cluster and validate TLS certificates · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:87]: failed to create customer resources for TLS endpoint cluster Unexpected error: <fmt.wrapError \| 0xc0010cf380>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-tls-endpoint-hcp-cluster-psgbhg" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-tls-endpoint-hcp-cluster-psgbhg/operationStatuses/08584113915373233860 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-15/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/tls-endpoint-cluster-2kkdh9d9gtp9/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-oo7pyvaeowk44\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:87]: failed to create customer resources for TLS endpoint cluster Unexpected error: <fmt.wrapError \| 0xc0010cf380>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-tls-endpoint-hcp-cluster-psgbhg" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-tls-endpoint-hcp-cluster-psgbhg/operationStatuses/08584113915373233860 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-15/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/tls-endpoint-cluster-2kkdh9d9gtp9/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-oo7pyvaeowk44\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 02:00:13 UTC · failed at e2e · test Customer should be able to lifecycle and confirm external auth on a cluster · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_list_and_verify.go:78]: failed to create customer resources for external auth list test Unexpected error: <fmt.wrapError \| 0xc001109320>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-ea-list2xpmz4-575f9n" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-ea-list2xpmz4-575f9n/operationStatuses/08584113913331185786 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-20/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/ea-list-sq8bchrx6sgb/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-3pwmnglx4zz7q\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_list_and_verify.go:78]: failed to create customer resources for external auth list test Unexpected error: <fmt.wrapError \| 0xc001109320>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-ea-list2xpmz4-575f9n" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-ea-list2xpmz4-575f9n/operationStatuses/08584113913331185786 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-20/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/ea-list-sq8bchrx6sgb/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-3pwmnglx4zz7q\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 02:00:13 UTC · failed at e2e · test Customer should respect cluster-wide node limits with nodepool autoscaling · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:283]: failed to create cluster customer resources for node-limit test Unexpected error: <fmt.wrapError \| 0xc0009ad040>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-node-limit-cluster-5wrht5" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-node-limit-cluster-5wrht5/operationStatuses/08584113913007034136 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-12/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/node-limit-test-hlk6qr5q8zl2/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-j6nw3ohx7bzo4\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:283]: failed to create cluster customer resources for node-limit test Unexpected error: <fmt.wrapError \| 0xc0009ad040>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-node-limit-cluster-5wrht5" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-node-limit-cluster-5wrht5/operationStatuses/08584113913007034136 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-12/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/node-limit-test-hlk6qr5q8zl2/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-j6nw3ohx7bzo4\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 02:00:13 UTC · failed at e2e · test Customer should be able to cordon, drain, and uncordon a node in an HCP cluster · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_cordon.go:80]: failed to create cluster customer resources for cordon test Unexpected error: <fmt.wrapError \| 0xc00089f100>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cordon-hcp-cluster-5jj9fq" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cordon-hcp-cluster-5jj9fq/operationStatuses/08584113912845247458 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-01/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cordon-test-q5nkw446pf9v/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-3wbuyr3ac6kzo\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_cordon.go:80]: failed to create cluster customer resources for cordon test Unexpected error: <fmt.wrapError \| 0xc00089f100>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cordon-hcp-cluster-5jj9fq" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cordon-hcp-cluster-5jj9fq/operationStatuses/08584113912845247458 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-01/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cordon-test-q5nkw446pf9v/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-3wbuyr3ac6kzo\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 02:00:13 UTC · failed at e2e · test Customer should be able to issue the default ingress serving certificate from Let's Encrypt using cert-manager with ACME DNS-01 challenges · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_ingress_cert_letsencrypt.go:114]: failed to create customer resources Unexpected error: <fmt.wrapError \| 0xc000b75c00>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-ingress-cert-le-znhvzv" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-ingress-cert-le-znhvzv/operationStatuses/08584113915367159662 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/ingress-cert-le-4v9jkdchszqt/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-nhimtfs3fueps\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-23/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_ingress_cert_letsencrypt.go:114]: failed to create customer resources Unexpected error: <fmt.wrapError \| 0xc000b75c00>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-ingress-cert-le-znhvzv" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-ingress-cert-le-znhvzv/operationStatuses/08584113915367159662 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/ingress-cert-le-4v9jkdchszqt/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-nhimtfs3fueps\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-23/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 02:00:13 UTC · failed at e2e · test Customer should be able to delete an HCP cluster whose managed identities were deleted first · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delete_missing_identities.go:101]: failed to deploy dedicated managed identities Unexpected error: <fmt.wrapError \| 0xc0010124e0>: failed waiting for deployment "mi-missing-mi-hcp-cluster-fgbmvd" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-missing-mi-hcp-cluster-fgbmvd/operationStatuses/08584113915367121300 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/delete-mi-rg-l28hcpvmt567/providers/Microsoft.Resources/deployments/clusterMsiScopedAssignments-jaooh6lj24n4o\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/delete-mi-rg-l28hcpvmt567/providers/Microsoft.Resources/deployments/clusterNonMsiScopedAssignments-jaooh6lj24n4o\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delete_missing_identities.go:101]: failed to deploy dedicated managed identities Unexpected error: <fmt.wrapError \| 0xc0010124e0>: failed waiting for deployment "mi-missing-mi-hcp-cluster-fgbmvd" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-missing-mi-hcp-cluster-fgbmvd/operationStatuses/08584113915367121300 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/delete-mi-rg-l28hcpvmt567/providers/Microsoft.Resources/deployments/clusterMsiScopedAssignments-jaooh6lj24n4o\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/delete-mi-rg-l28hcpvmt567/providers/Microsoft.Resources/deployments/clusterNonMsiScopedAssignments-jaooh6lj24n4o\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 02:00:13 UTC · failed at e2e · test Customer should be able to create node pools with ARM64-based VMs · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/arm64_nodepool.go:78]: failed to create customer resources for ARM64 nodepool cluster Unexpected error: <fmt.wrapError \| 0xc00057ce00>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-arm64-vm-hcp-cluster-r5vvmz" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-arm64-vm-hcp-cluster-r5vvmz/operationStatuses/08584113915370321334 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-13/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/arm64-vm-cluster-5rbptphj46kh/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-6htyoibevj7ye\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/arm64_nodepool.go:78]: failed to create customer resources for ARM64 nodepool cluster Unexpected error: <fmt.wrapError \| 0xc00057ce00>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-arm64-vm-hcp-cluster-r5vvmz" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-arm64-vm-hcp-cluster-r5vvmz/operationStatuses/08584113915370321334 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-13/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/arm64-vm-cluster-5rbptphj46kh/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-6htyoibevj7ye\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 02:00:13 UTC · failed at e2e · test Customer should be able to test admin credentials before cluster ready, then full admin credential lifecycle · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:84]: failed to create customer resources for admin credential lifecycle cluster Unexpected error: <fmt.wrapError \| 0xc001022100>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-admin-cred-lifecycle-5h9f9s-sdsp8q" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-admin-cred-lifecycle-5h9f9s-sdsp8q/operationStatuses/08584113915370097694 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/admin-credential-lifecycle-test-xlsqsjp7jqbr/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-yrpyalmshkqw2\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-08/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:84]: failed to create customer resources for admin credential lifecycle cluster Unexpected error: <fmt.wrapError \| 0xc001022100>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-admin-cred-lifecycle-5h9f9s-sdsp8q" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-admin-cred-lifecycle-5h9f9s-sdsp8q/operationStatuses/08584113915370097694 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/admin-credential-lifecycle-test-xlsqsjp7jqbr/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-yrpyalmshkqw2\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-08/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 02:00:13 UTC · failed at e2e · test Customer should create a cluster with private KAS and verify API server is only reachable from VNet while ingress remains public · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:77]: failed to create customer resources for private KAS cluster Unexpected error: <fmt.wrapError \| 0xc001129640>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-private-kas-dfpc6t" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-private-kas-dfpc6t/operationStatuses/08584113915370549584 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-02/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/private-kas-dddkkc8rw76v/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-jxdbnndsp5js2\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:77]: failed to create customer resources for private KAS cluster Unexpected error: <fmt.wrapError \| 0xc001129640>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-private-kas-dfpc6t" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-private-kas-dfpc6t/operationStatuses/08584113915370549584 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-02/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/private-kas-dddkkc8rw76v/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-jxdbnndsp5js2\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 02:00:13 UTC · failed at e2e · test Customer should be able to create an HCP cluster and manage ImageDigestMirrors · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/idms_lifecycle.go:90]: failed to create customer resources for IDMS cluster Unexpected error: <fmt.wrapError \| 0xc000686ec0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-idms-e2e-hcp-cluster-54nj9g" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-idms-e2e-hcp-cluster-54nj9g/operationStatuses/08584113913114169636 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/idms-pwtsd6dd5nm5/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-lv7dvzspd2e5i\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-21/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/idms_lifecycle.go:90]: failed to create customer resources for IDMS cluster Unexpected error: <fmt.wrapError \| 0xc000686ec0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-idms-e2e-hcp-cluster-54nj9g" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-idms-e2e-hcp-cluster-54nj9g/operationStatuses/08584113913114169636 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/idms-pwtsd6dd5nm5/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-lv7dvzspd2e5i\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-21/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 02:00:13 UTC · failed at e2e · test Customer should be able to successfully upgrade control plane minor version from 4.20 minor to 4.21 minor · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:123]: failed to create customer resources for y-stream upgrade cluster "cp-ystream-upgrade-4-21-tlkfpt" Unexpected error: <fmt.wrapError \| 0xc0012d42e0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cp-ystream-upgrade-4-21-tlkfpt-dgcfjk" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cp-ystream-upgrade-4-21-tlkfpt-dgcfjk/operationStatuses/08584113913452272222 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-14/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cp-ystream-upgrade-4-21-vfkmrmxhjl5d/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-gwvc3vdjzhp2q\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:123]: failed to create customer resources for y-stream upgrade cluster "cp-ystream-upgrade-4-21-tlkfpt" Unexpected error: <fmt.wrapError \| 0xc0012d42e0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cp-ystream-upgrade-4-21-tlkfpt-dgcfjk" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cp-ystream-upgrade-4-21-tlkfpt-dgcfjk/operationStatuses/08584113913452272222 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-14/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cp-ystream-upgrade-4-21-vfkmrmxhjl5d/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-gwvc3vdjzhp2q\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 02:00:13 UTC · failed at e2e · test ARO HCP Service should successfully provision cluster when MI role assignments are added after cluster creation · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:140]: failed to start cluster creation Unexpected error: <exported.ResponseError \| 0xc0009be4e0>: PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/delayed-rbac-8x9trr4ppbcc/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/delayed-rbac-cluster -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:140]: failed to start cluster creation Unexpected error: <exported.ResponseError \| 0xc0009be4e0>: PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/delayed-rbac-8x9trr4ppbcc/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/delayed-rbac-cluster -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 02:00:13 UTC · failed at e2e · test Customer should be able to create an HCP cluster and manage pull secrets · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_pullsecret.go:104]: failed to create customer resources for pull secret cluster Unexpected error: <fmt.wrapError \| 0xc000bfe780>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-pullsecret-hcp-cluster-qzdltq" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-pullsecret-hcp-cluster-qzdltq/operationStatuses/08584113915375639922 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-16/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/pullsecret-test-ckn8x8pc56zb/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-p2h32vyw2l2ma\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_pullsecret.go:104]: failed to create customer resources for pull secret cluster Unexpected error: <fmt.wrapError \| 0xc000bfe780>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-pullsecret-hcp-cluster-qzdltq" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-pullsecret-hcp-cluster-qzdltq/operationStatuses/08584113915375639922 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-16/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/pullsecret-test-ckn8x8pc56zb/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-p2h32vyw2l2ma\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 02:00:13 UTC · failed at e2e · test Customer should be able to create a cluster and node pools with aggregated advanced features · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:130]: failed to create customer resources for aggregated feature cluster Unexpected error: <fmt.wrapError \| 0xc0010b9ba0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-agg-cluster-fcg4sb" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-agg-cluster-fcg4sb/operationStatuses/08584113914362490457 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/feature-aggregation-tk7ckttkvc2h/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-zkzqbkxbrvnhu\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-00/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:130]: failed to create customer resources for aggregated feature cluster Unexpected error: <fmt.wrapError \| 0xc0010b9ba0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-agg-cluster-fcg4sb" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-agg-cluster-fcg4sb/operationStatuses/08584113914362490457 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/feature-aggregation-tk7ckttkvc2h/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-zkzqbkxbrvnhu\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-00/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 02:00:13 UTC · failed at e2e · test Customer should be able to forward control plane logs to a storage account and Event Hub via shoebox diagnostic settings · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/shoebox_logs.go:268]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000bdb9e0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-shoebox-hcp-cluster-65ml2z" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-shoebox-hcp-cluster-65ml2z/operationStatuses/08584113912169288703 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/shoebox-cluster-ltfr4g8jwwws/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-22eudp3sazqeq\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-08/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/shoebox_logs.go:268]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000bdb9e0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-shoebox-hcp-cluster-65ml2z" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-shoebox-hcp-cluster-65ml2z/operationStatuses/08584113912169288703 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/shoebox-cluster-ltfr4g8jwwws/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-22eudp3sazqeq\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-08/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 02:00:13 UTC · failed at e2e · test Update HCPOpenShiftCluster Positive creates a cluster and updates tags with a PATCH request · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:151]: failed to create customer resources for patch-tags cluster Unexpected error: <fmt.wrapError \| 0xc000eb5680>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-patch-tags-cluster-wshhfl" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-patch-tags-cluster-wshhfl/operationStatuses/08584113913827373414 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-tags-jcg4mtdpvc69/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-fklo75htpxzxc\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-13/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:151]: failed to create customer resources for patch-tags cluster Unexpected error: <fmt.wrapError \| 0xc000eb5680>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-patch-tags-cluster-wshhfl" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-patch-tags-cluster-wshhfl/operationStatuses/08584113913827373414 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-tags-jcg4mtdpvc69/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-fklo75htpxzxc\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-13/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 02:00:13 UTC · failed at e2e · test Customer should be able to create an HCP cluster with Image Registry not present · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/image_registry_cluster_create.go:79]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000d98940>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-disabled-image-registry-hcp-cluster-kjxtrg" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-disabled-image-registry-hcp-cluster-kjxtrg/operationStatuses/08584113913314624478 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-22/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/disabled-image-registry-w7nl4t2fw4sl/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-qmh74zogaodhg\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/image_registry_cluster_create.go:79]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000d98940>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-disabled-image-registry-hcp-cluster-kjxtrg" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-disabled-image-registry-hcp-cluster-kjxtrg/operationStatuses/08584113913314624478 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-22/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/disabled-image-registry-w7nl4t2fw4sl/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-qmh74zogaodhg\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 02:00:13 UTC · failed at e2e · test Customer should be able to delete a nodepool · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_delete.go:71]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc001080a20>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-np-delete-hcp-cluster-wjhgxf" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-np-delete-hcp-cluster-wjhgxf/operationStatuses/08584113912929453892 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/nodepool-delete-779f92ndzf75/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-rcpnvqrcxdqmo\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-09/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_delete.go:71]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc001080a20>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-np-delete-hcp-cluster-wjhgxf" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-np-delete-hcp-cluster-wjhgxf/operationStatuses/08584113912929453892 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/nodepool-delete-779f92ndzf75/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-rcpnvqrcxdqmo\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-09/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 02:00:13 UTC · failed at e2e · test Customer should be able to create an HCP cluster and custom node pool osDisk size · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_nodepool_osdisk.go:71]: failed to create customer resources for cluster "hcp-cluster-np-128" Unexpected error: <fmt.wrapError \| 0xc000ba70c0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-hcp-cluster-np-128-v7nvxr" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-hcp-cluster-np-128-v7nvxr/operationStatuses/08584113915365585990 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-06/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/clusternp128-d7wljqzdp555/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-3ljyjjx2n7gxo\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_nodepool_osdisk.go:71]: failed to create customer resources for cluster "hcp-cluster-np-128" Unexpected error: <fmt.wrapError \| 0xc000ba70c0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-hcp-cluster-np-128-v7nvxr" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-hcp-cluster-np-128-v7nvxr/operationStatuses/08584113915365585990 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-06/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/clusternp128-d7wljqzdp555/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-3ljyjjx2n7gxo\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 02:00:13 UTC · failed at e2e · test Customer should be able to create a no-CNI private cluster with a private key vault, a nodepool and install cilium CNI successfully · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:84]: failed to create customer resources with private key vault Unexpected error: <fmt.wrapError \| 0xc0006e6620>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cilium-cluster-nf72s7" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cilium-cluster-nf72s7/operationStatuses/08584113914453431415 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-01/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/complex-cilium-kv-stcwrwm7gdff/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-4vnk4izirdfzu\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:84]: failed to create customer resources with private key vault Unexpected error: <fmt.wrapError \| 0xc0006e6620>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cilium-cluster-nf72s7" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cilium-cluster-nf72s7/operationStatuses/08584113914453431415 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-01/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/complex-cilium-kv-stcwrwm7gdff/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-4vnk4izirdfzu\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 02:00:13 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 5.0 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:112]: failed to create customer resources for cluster "cluster-candidate-5-0-4b8xtf" Unexpected error: <fmt.wrapError \| 0xc00088c740>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cluster-candidate-5-0-4b8xtf-v47dt6" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-candidate-5-0-4b8xtf-v47dt6/operationStatuses/08584113913751466362 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-5-0-n8kzhs252rzq/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-oowsf3asvtt4c\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-06/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:112]: failed to create customer resources for cluster "cluster-candidate-5-0-4b8xtf" Unexpected error: <fmt.wrapError \| 0xc00088c740>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cluster-candidate-5-0-4b8xtf-v47dt6" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-candidate-5-0-4b8xtf-v47dt6/operationStatuses/08584113913751466362 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-5-0-n8kzhs252rzq/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-oowsf3asvtt4c\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-06/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: BadGatewayConnection; detail message The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request.; provider Microsoft.RedHatOpenShift 02:00:13 UTC · failed at e2e · test Customer should be able to create an HCP cluster with custom autoscaling and update the autoscaling configuration · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:108]: failed to create HCP cluster "autoscaling-hcp-cluster" with custom autoscaling Unexpected error: <fmt.wrapError \| 0xc000c05160>: failed to create HCP cluster autoscaling-hcp-cluster: failed starting cluster creation "autoscaling-hcp-cluster" in resourcegroup="autoscaling-cluster-zsckxd6r8cnx": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/autoscaling-cluster-zsckxd6r8cnx/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/autoscaling-hcp-cluster -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:108]: failed to create HCP cluster "autoscaling-hcp-cluster" with custom autoscaling Unexpected error: <fmt.wrapError \| 0xc000c05160>: failed to create HCP cluster autoscaling-hcp-cluster: failed starting cluster creation "autoscaling-hcp-cluster" in resourcegroup="autoscaling-cluster-zsckxd6r8cnx": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/autoscaling-cluster-zsckxd6r8cnx/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/autoscaling-hcp-cluster -------------------------------------------------------------------------------- RESPONSE 502: 502 Bad Gateway ERROR CODE: BadGatewayConnection -------------------------------------------------------------------------------- { "error": { "code": "BadGatewayConnection", "message": "The network connectivity issue encountered for 'Microsoft.RedHatOpenShift'; cannot fulfill the request." } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 02:00:13 UTC · failed at e2e · test Authorized CIDRs Connectivity should allow API access only from authorized VM IP · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:88]: failed to create customer resources for authorized CIDRs cluster Unexpected error: <fmt.wrapError \| 0xc001012880>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cidr-connectivity-test-lq9dng" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cidr-connectivity-test-lq9dng/operationStatuses/08584113915355806276 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-10/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-ncn82bv9lpkb/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-wihoigd7cpguy\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:88]: failed to create customer resources for authorized CIDRs cluster Unexpected error: <fmt.wrapError \| 0xc001012880>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cidr-connectivity-test-lq9dng" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cidr-connectivity-test-lq9dng/operationStatuses/08584113915355806276 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-10/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-ncn82bv9lpkb/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-wihoigd7cpguy\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 02:00:13 UTC · failed at e2e · test Customer should be able to create an HCP cluster with back-level version 4.20 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:121]: failed to deploy managed identities for back-level version 4.20 Unexpected error: <fmt.wrapError \| 0xc00019efc0>: failed waiting for deployment "mi-cluster-ver-4-20-b2znhh" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-ver-4-20-b2znhh/operationStatuses/08584113913906857154 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cluster-back-version-wmxx2c8c9twx/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-11/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:121]: failed to deploy managed identities for back-level version 4.20 Unexpected error: <fmt.wrapError \| 0xc00019efc0>: failed waiting for deployment "mi-cluster-ver-4-20-b2znhh" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-ver-4-20-b2znhh/operationStatuses/08584113913906857154 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cluster-back-version-wmxx2c8c9twx/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-11/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 02:00:13 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.22 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:112]: failed to create customer resources for cluster "cluster-candidate-4-22-6fxxn7" Unexpected error: <fmt.wrapError \| 0xc000d68380>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cluster-candidate-4-22-6fxxn7-ng8rb6" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-candidate-4-22-6fxxn7-ng8rb6/operationStatuses/08584113913906446639 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-4-22-j8b5hvpzhn5m/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-7frcmwevtsxko\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-02/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:112]: failed to create customer resources for cluster "cluster-candidate-4-22-6fxxn7" Unexpected error: <fmt.wrapError \| 0xc000d68380>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cluster-candidate-4-22-6fxxn7-ng8rb6" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-candidate-4-22-6fxxn7-ng8rb6/operationStatuses/08584113913906446639 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-4-22-j8b5hvpzhn5m/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-7frcmwevtsxko\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-02/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 02:00:13 UTC · failed at e2e · test Customer should downgrade a nodepool version z-stream downgrade from 4.21.zLatest to 4.21.zPrevious · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:570]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000c78700>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-np-downgrade-hfsp4j-mttxxb" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-np-downgrade-hfsp4j-mttxxb/operationStatuses/08584113912282350438 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-24/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-downgrade-hfsp4j-w9pdctvxrqv9/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-7uiwfmbeixp3i\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:570]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000c78700>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-np-downgrade-hfsp4j-mttxxb" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-np-downgrade-hfsp4j-mttxxb/operationStatuses/08584113912282350438 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-24/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-downgrade-hfsp4j-w9pdctvxrqv9/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-7uiwfmbeixp3i\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 02:00:13 UTC · failed at e2e · test Customer should be able to create a cluster with an external auth config and get the external auth config · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:85]: failed to create customer resources for external auth cluster Unexpected error: <fmt.wrapError \| 0xc000baaf40>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-ea-cluster-845ctr" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-ea-cluster-845ctr/operationStatuses/08584113913017973521 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/external-auth-cluster-28pr5rtrjz7f/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-ltdphft7fgrwg\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-19/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:85]: failed to create customer resources for external auth cluster Unexpected error: <fmt.wrapError \| 0xc000baaf40>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-ea-cluster-845ctr" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-ea-cluster-845ctr/operationStatuses/08584113913017973521 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/external-auth-cluster-28pr5rtrjz7f/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-ltdphft7fgrwg\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-19/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 02:00:13 UTC · failed at e2e · test Customer should not be able to perform various invalid operations on cluster resources · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/simple_negative_cases.go:83]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000c96200>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-negative-tests-cluster-pqtw6g" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-negative-tests-cluster-pqtw6g/operationStatuses/08584113912154838800 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-16/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-negative-tests-qhmbvhnrg56r/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-oeayaalhj7wjy\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/simple_negative_cases.go:83]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000c96200>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-negative-tests-cluster-pqtw6g" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-negative-tests-cluster-pqtw6g/operationStatuses/08584113912154838800 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-16/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-negative-tests-qhmbvhnrg56r/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-oeayaalhj7wjy\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 02:00:13 UTC · failed at e2e · test Customer should be able to successfully upgrade control plane minor version from 4.22 minor to 5.0 minor · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:123]: failed to create customer resources for y-stream upgrade cluster "cp-ystream-upgrade-5-0-p8mv48" Unexpected error: <fmt.wrapError \| 0xc000ce0ae0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cp-ystream-upgrade-5-0-p8mv48-rr8c45" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cp-ystream-upgrade-5-0-p8mv48-rr8c45/operationStatuses/08584113913442668481 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-18/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cp-ystream-upgrade-5-0-qmng4jgqjs56/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-6i3b5eapw7ct4\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:123]: failed to create customer resources for y-stream upgrade cluster "cp-ystream-upgrade-5-0-p8mv48" Unexpected error: <fmt.wrapError \| 0xc000ce0ae0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cp-ystream-upgrade-5-0-p8mv48-rr8c45" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cp-ystream-upgrade-5-0-p8mv48-rr8c45/operationStatuses/08584113913442668481 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-18/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cp-ystream-upgrade-5-0-qmng4jgqjs56/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-6i3b5eapw7ct4\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 02:00:13 UTC · failed at e2e · test Customer should be able to successfully upgrade control plane minor version from 4.21 minor to 4.22 minor · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:123]: failed to create customer resources for y-stream upgrade cluster "cp-ystream-upgrade-4-22-t7jcbl" Unexpected error: <fmt.wrapError \| 0xc00071d980>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cp-ystream-upgrade-4-22-t7jcbl-8d776t" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cp-ystream-upgrade-4-22-t7jcbl-8d776t/operationStatuses/08584113913541685590 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cp-ystream-upgrade-4-22-9z48wmwt6pmp/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-zmpandh64xdog\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-16/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:123]: failed to create customer resources for y-stream upgrade cluster "cp-ystream-upgrade-4-22-t7jcbl" Unexpected error: <fmt.wrapError \| 0xc00071d980>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cp-ystream-upgrade-4-22-t7jcbl-8d776t" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cp-ystream-upgrade-4-22-t7jcbl-8d776t/operationStatuses/08584113913541685590 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cp-ystream-upgrade-4-22-9z48wmwt6pmp/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-zmpandh64xdog\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-16/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 02:00:13 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 4.20 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:106]: failed to create customer resources for z-stream cluster "cluster-zstream-4-20-gc4q5q" Unexpected error: <fmt.wrapError \| 0xc0009d6c00>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cluster-zstream-4-20-gc4q5q-z24rh4" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-zstream-4-20-gc4q5q-z24rh4/operationStatuses/08584113913639830680 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-20-s5dpm7kj5krx/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-76i3fdasycsia\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-04/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:106]: failed to create customer resources for z-stream cluster "cluster-zstream-4-20-gc4q5q" Unexpected error: <fmt.wrapError \| 0xc0009d6c00>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cluster-zstream-4-20-gc4q5q-z24rh4" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-zstream-4-20-gc4q5q-z24rh4/operationStatuses/08584113913639830680 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-20-s5dpm7kj5krx/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-76i3fdasycsia\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-04/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 02:00:13 UTC · failed at e2e · test FIPS Mode Support with v2026-09-01 API should create an HCP cluster with FIPS mode enabled via cryptoRestrictions property · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_fips_mode.go:72]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000fa4b80>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-fips-enabled-cluster-zm2g24" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-fips-enabled-cluster-zm2g24/operationStatuses/08584113915371376474 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-07/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/fips-enabled-wckv5dm6jq8x/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-3h7v3eeeqgjeu\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_fips_mode.go:72]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000fa4b80>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-fips-enabled-cluster-zm2g24" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-fips-enabled-cluster-zm2g24/operationStatuses/08584113915371376474 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-07/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/fips-enabled-wckv5dm6jq8x/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-3h7v3eeeqgjeu\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 02:00:13 UTC · failed at e2e · test Customer should be able to successfully upgrade control plane minor version from 4.22 minor to 4.23 minor · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:123]: failed to create customer resources for y-stream upgrade cluster "cp-ystream-upgrade-4-23-dxtnkh" Unexpected error: <fmt.wrapError \| 0xc000a4e9e0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cp-ystream-upgrade-4-23-dxtnkh-qg2zth" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cp-ystream-upgrade-4-23-dxtnkh-qg2zth/operationStatuses/08584113913448474837 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-17/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cp-ystream-upgrade-4-23-zcw9r52tsr2b/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-f5hs6ydcv2ewe\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:123]: failed to create customer resources for y-stream upgrade cluster "cp-ystream-upgrade-4-23-dxtnkh" Unexpected error: <fmt.wrapError \| 0xc000a4e9e0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cp-ystream-upgrade-4-23-dxtnkh-qg2zth" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cp-ystream-upgrade-4-23-dxtnkh-qg2zth/operationStatuses/08584113913448474837 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-17/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cp-ystream-upgrade-4-23-zcw9r52tsr2b/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-f5hs6ydcv2ewe\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 02:00:13 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 4.21 · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:106]: failed to create customer resources for z-stream cluster "cluster-zstream-4-21-jz52ld" Unexpected error: <fmt.wrapError \| 0xc001036ec0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cluster-zstream-4-21-jz52ld-fx8wn6" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-zstream-4-21-jz52ld-fx8wn6/operationStatuses/08584113913641594984 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-21-fpmccnv5bjnp/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-qdnzuabix4bjc\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-03/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:106]: failed to create customer resources for z-stream cluster "cluster-zstream-4-21-jz52ld" Unexpected error: <fmt.wrapError \| 0xc001036ec0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cluster-zstream-4-21-jz52ld-fx8wn6" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-zstream-4-21-jz52ld-fx8wn6/operationStatuses/08584113913641594984 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-21-fpmccnv5bjnp/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-qdnzuabix4bjc\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-03/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 02:00:13 UTC · failed at e2e · test Customer should upgrade and update a nodepool from 4.20.z to 4.21.zLatest · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:136]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000f30a60>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-np-version-upgrade-cluster-kgpnd2-45phqx" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-np-version-upgrade-cluster-kgpnd2-45phqx/operationStatuses/08584113912586089579 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-07/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-version-upgrade-kgpnd2-t5lrjpxgmgpw/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-igjsnmdrgrbxc\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:136]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000f30a60>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-np-version-upgrade-cluster-kgpnd2-45phqx" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-np-version-upgrade-cluster-kgpnd2-45phqx/operationStatuses/08584113912586089579 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-07/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-version-upgrade-kgpnd2-t5lrjpxgmgpw/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-igjsnmdrgrbxc\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 02:00:13 UTC · failed at e2e · test Customer should upgrade and update a nodepool from 4.20.z to 4.20.zLatest · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:136]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc00128ea20>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-np-version-upgrade-cluster-dthbtj-dqjtmg" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-np-version-upgrade-cluster-dthbtj-dqjtmg/operationStatuses/08584113912431610183 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-15/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-version-upgrade-dthbtj-7tbfh95dw7tz/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-wzdmuce4v2olu\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:136]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc00128ea20>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-np-version-upgrade-cluster-dthbtj-dqjtmg" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-np-version-upgrade-cluster-dthbtj-dqjtmg/operationStatuses/08584113912431610183 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-15/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-version-upgrade-dthbtj-7tbfh95dw7tz/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-wzdmuce4v2olu\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 02:00:13 UTC · failed at e2e · test Customer should be able to retrieve cluster and nodepool status active versions · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_nodepool_active_versions.go:73]: failed to create customer resources for v20261001preview cluster Unexpected error: <fmt.wrapError \| 0xc000826c60>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-v20261001-4sckh4" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-v20261001-4sckh4/operationStatuses/08584113915372800941 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/v20261001preview-r6vgswb52dpb/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-7e767rdphst3w\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-17/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_nodepool_active_versions.go:73]: failed to create customer resources for v20261001preview cluster Unexpected error: <fmt.wrapError \| 0xc000826c60>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-v20261001-4sckh4" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-v20261001-4sckh4/operationStatuses/08584113915372800941 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/v20261001preview-r6vgswb52dpb/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-7e767rdphst3w\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-17/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 02:00:13 UTC · failed at e2e · test Customer should be able to create several HCP clusters in their customer resource group, but not in the same managed resource group · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:81]: failed to create customer resources for first cluster "basic-hcp-cluster" Unexpected error: <fmt.wrapError \| 0xc0009e8f80>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-basic-hcp-cluster-kq9qb6" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-basic-hcp-cluster-kq9qb6/operationStatuses/08584113915366174777 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-18/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/customer-rg-qvskb9lgr2hn/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-eqbgltuyzhpg4\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:81]: failed to create customer resources for first cluster "basic-hcp-cluster" Unexpected error: <fmt.wrapError \| 0xc0009e8f80>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-basic-hcp-cluster-kq9qb6" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-basic-hcp-cluster-kq9qb6/operationStatuses/08584113915366174777 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-18/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/customer-rg-qvskb9lgr2hn/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-eqbgltuyzhpg4\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 02:00:13 UTC · failed at e2e · test Customer should upgrade a nodepool skipping one minor version (+2) from 4.20.z to 4.22.zLatest · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:431]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc001402320>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-np-skip-minor-l4vpc6-lszvf9" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-np-skip-minor-l4vpc6-lszvf9/operationStatuses/08584113912334810014 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-13/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-skip-minor-l4vpc6-mxzhwk4xdm5z/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-oajio3ejrfg24\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:431]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc001402320>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-np-skip-minor-l4vpc6-lszvf9" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-np-skip-minor-l4vpc6-lszvf9/operationStatuses/08584113912334810014 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-13/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-skip-minor-l4vpc6-mxzhwk4xdm5z/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-oajio3ejrfg24\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: DeploymentFailed; detail code RoleAssignmentLimitExceeded; detail message No more role assignments can be created. 02:00:13 UTC · failed at e2e · test Customer should be able to update node pool labels and taints · suite stage/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_labels_taints.go:82]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000ed09c0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cluster-np-labels-taints-2jzh8d" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-np-labels-taints-2jzh8d/operationStatuses/08584113912724860835 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-labels-taints-ftnsr25mhp4v/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-r6cuakilyx67k\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-00/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_labels_taints.go:82]: failed to create cluster customer resources Unexpected error: <fmt.wrapError \| 0xc000ed09c0>: failed to create managed identities: failed to create managed identities: failed waiting for deployment "mi-cluster-np-labels-taints-2jzh8d" at subscription scope to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Resources/deployments/mi-cluster-np-labels-taints-2jzh8d/operationStatuses/08584113912724860835 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: DeploymentFailed -------------------------------------------------------------------------------- { "status": "Failed", "error": { "code": "DeploymentFailed", "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.", "details": [ { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-labels-taints-ftnsr25mhp4v/providers/Microsoft.Resources/deployments/pooledNonMsiScopedAssignments-r6cuakilyx67k\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" }, { "code": "Conflict", "message": "{\r\n \"status\": \"Failed\",\r\n \"error\": {\r\n \"code\": \"ResourceDeploymentFailure\",\r\n \"message\": \"The resource write operation failed to complete successfully, because it reached terminal provisioning state 'Failed'.\",\r\n \"details\": [\r\n {\r\n \"code\": \"DeploymentFailed\",\r\n \"target\": \"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/aro-hcp-msi-container-stg-shard0-00-00/providers/Microsoft.Resources/deployments/pooledMsiScopedAssignments\",\r\n \"message\": \"At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.\",\r\n \"details\": [\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n },\r\n {\r\n \"code\": \"RoleAssignmentLimitExceeded\",\r\n \"message\": \"No more role assignments can be created.\"\r\n }\r\n ]\r\n }\r\n ]\r\n }\r\n}" } ] } } -------------------------------------------------------------------------------- ... occurred