CIHealth Run Log

Environment: PROD

Time (UTC)	Runtime	Job	Region	Failed at	Result	PR	Failed tests	Details
23:00:32	1h 57m 37s	periodic-ci-Azure-ARO-HCP-main-periodic-prod-e2e-parallel-ocp-nightly post-goodmerged PR No additional run metadata captured	uksouth	e2e	failed	n/a	12	e2e (12) ERROR CODE: InvalidRequestContent; detail message Invalid value: "4.21": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is "nightly"; provider Microsoft.RedHatOpenShift 23:00:32 UTC · failed at e2e · test Customer should be able to create an HCP cluster and manage ImageDigestMirrors · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/idms_lifecycle.go:175]: failed to update cluster with second ImageDigestMirror set Unexpected error: <exported.ResponseError \| 0xc00114d9e0>: PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/idms-zgncvtrh7b9x/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/idms-e2e-hcp-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"4.21\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"", "target": "properties.version.id" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/idms_lifecycle.go:175]: failed to update cluster with second ImageDigestMirror set Unexpected error: <exported.ResponseError \| 0xc00114d9e0>: PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/idms-zgncvtrh7b9x/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/idms-e2e-hcp-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"4.21\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"", "target": "properties.version.id" } } -------------------------------------------------------------------------------- ... occurred VerifyKubeAPIServerServerVersionUpgraded failed: kube-apiserver ServerVersion not updated (unchanged from pre-upgrade) 23:00:32 UTC · failed at e2e · test Customer should be able to successfully upgrade control plane minor version from 5.0 minor to 5.1 minor · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:176]: Timed out after 2700.002s. Expected success, but got an error: <errors.joinError \| 0xc00055c870>: VerifyKubeAPIServerServerVersionUpgraded failed: kube-apiserver ServerVersion not updated (unchanged from pre-upgrade) VerifyHostedControlPlaneYStreamUpgrade(previousMinor=5.0.0-0.nightly-multi-2026-09-16-151028, targetMinor=5.1.0-0.nightly-multi-2026-09-15-135250) failed: clusterversion status.history has no version in target minor "5.1.0-0.nightly-multi-2026-09-15-135250" ... fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:176]: Timed out after 2700.002s. Expected success, but got an error: <errors.joinError \| 0xc00055c870>: VerifyKubeAPIServerServerVersionUpgraded failed: kube-apiserver ServerVersion not updated (unchanged from pre-upgrade) VerifyHostedControlPlaneYStreamUpgrade(previousMinor=5.0.0-0.nightly-multi-2026-09-16-151028, targetMinor=5.1.0-0.nightly-multi-2026-09-15-135250) failed: clusterversion status.history has no version in target minor "5.1.0-0.nightly-multi-2026-09-15-135250" ... ERROR CODE: InvalidRequestContent; detail message Invalid value: "4.21": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is "nightly"; provider Microsoft.RedHatOpenShift 23:00:32 UTC · failed at e2e · test Customer should be able to create a cluster and node pools with aggregated advanced features · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:274]: failed to begin cluster "agg-cluster" authorized CIDR update Unexpected error: <exported.ResponseError \| 0xc002bb68a0>: PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/feature-aggregation-t78vdzzqpblp/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/agg-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"4.21\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"", "target": "properties.version.id" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:274]: failed to begin cluster "agg-cluster" authorized CIDR update Unexpected error: <exported.ResponseError \| 0xc002bb68a0>: PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/feature-aggregation-t78vdzzqpblp/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/agg-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"4.21\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"", "target": "properties.version.id" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InvalidRequestContent; detail message Invalid value: "4.21": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is "nightly"; provider Microsoft.RedHatOpenShift 23:00:32 UTC · failed at e2e · test Update HCPOpenShiftCluster Positive creates a cluster and updates tags with a PATCH request · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:193]: failed to update HCP cluster tags via PATCH Unexpected error: <exported.ResponseError \| 0xc000b80720>: PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-tags-nz9bqqgzphzd/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-tags-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"4.21\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"", "target": "properties.version.id" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:193]: failed to update HCP cluster tags via PATCH Unexpected error: <exported.ResponseError \| 0xc000b80720>: PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-tags-nz9bqqgzphzd/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-tags-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"4.21\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"", "target": "properties.version.id" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InvalidRequestContent; detail message Invalid value: "4.21": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is "nightly"; provider Microsoft.RedHatOpenShift 23:00:32 UTC · failed at e2e · test Authorized CIDRs Connectivity should allow API access only from authorized VM IP · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:408]: failed to begin updating cluster "cidr-connectivity-test" authorized CIDRs Unexpected error: <exported.ResponseError \| 0xc000f5e360>: PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-cx8sqpwt9s45/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cidr-connectivity-test -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"4.21\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"", "target": "properties.version.id" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:408]: failed to begin updating cluster "cidr-connectivity-test" authorized CIDRs Unexpected error: <exported.ResponseError \| 0xc000f5e360>: PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-cx8sqpwt9s45/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cidr-connectivity-test -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"4.21\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"", "target": "properties.version.id" } } -------------------------------------------------------------------------------- ... occurred failed waiting for nodepool="<nodepool>" for cluster "<cluster>" in resourcegroup="<resource-group>" to finish creating: context deadline exceeded 23:00:32 UTC · failed at e2e · test Customer should be able to create an HCP cluster with back-level version 4.20 · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:198]: failed to create node pool version 4.20.32 for back-level cluster Unexpected error: <fmt.wrapError \| 0xc001376700>: failed waiting for nodepool="np-ver-4-20-32" for cluster "cluster-ver-4-20" in resourcegroup="rg-cluster-back-version-tzdswnvvvrrb" to finish creating: context deadline exceeded ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:198]: failed to create node pool version 4.20.32 for back-level cluster Unexpected error: <fmt.wrapError \| 0xc001376700>: failed waiting for nodepool="np-ver-4-20-32" for cluster "cluster-ver-4-20" in resourcegroup="rg-cluster-back-version-tzdswnvvvrrb" to finish creating: context deadline exceeded ... occurred ERROR CODE: InvalidRequestContent; detail message Invalid value: "4.22": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is "nightly"; provider Microsoft.RedHatOpenShift 23:00:32 UTC · failed at e2e · test Customer should create a cluster with private KAS and verify API server is only reachable from VNet while ingress remains public · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:90]: failed to create HCP cluster "private-kas" with private KAS Unexpected error: <fmt.wrapError \| 0xc00298e7c0>: failed to create HCP cluster private-kas: failed starting cluster creation "private-kas" in resourcegroup="private-kas-9f9tbz46mct7": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/private-kas-9f9tbz46mct7/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/private-kas -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"4.22\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"", "target": "properties.version.id" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:90]: failed to create HCP cluster "private-kas" with private KAS Unexpected error: <fmt.wrapError \| 0xc00298e7c0>: failed to create HCP cluster private-kas: failed starting cluster creation "private-kas" in resourcegroup="private-kas-9f9tbz46mct7": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/private-kas-9f9tbz46mct7/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/private-kas -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"4.22\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"", "target": "properties.version.id" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InvalidRequestContent; detail message Invalid value: "4.21": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is "nightly"; provider Microsoft.RedHatOpenShift 23:00:32 UTC · failed at e2e · test Customer should be able to create an HCP cluster with custom autoscaling and update the autoscaling configuration · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:173]: failed to update cluster "autoscaling-hcp-cluster" autoscaling configuration Unexpected error: <exported.ResponseError \| 0xc002804f90>: PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/autoscaling-cluster-grcsb5ww9xqw/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/autoscaling-hcp-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"4.21\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"", "target": "properties.version.id" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:173]: failed to update cluster "autoscaling-hcp-cluster" autoscaling configuration Unexpected error: <exported.ResponseError \| 0xc002804f90>: PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/autoscaling-cluster-grcsb5ww9xqw/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/autoscaling-hcp-cluster -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"4.21\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"", "target": "properties.version.id" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InvalidRequestContent; detail message Invalid value: "<ocp-nightly>": must be at least 4.20.8; provider Microsoft.RedHatOpenShift 23:00:32 UTC · failed at e2e · test Customer should upgrade a nodepool skipping one minor version (+2) from 4.20.z to 4.22.zLatest · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:459]: failed to create nodepool nps-420-422 at version 4.20.0-0.nightly-multi-2026-09-16-210648 Unexpected error: <fmt.wrapError \| 0xc00011be60>: failed to create NodePool nps-420-422: failed starting nodepool creation "nps-420-422" for cluster "np-skip-minor-tcgt6v" in resourcegroup="rg-np-skip-minor-tcgt6v-cwclsxtppk9g": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-skip-minor-tcgt6v-cwclsxtppk9g/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-skip-minor-tcgt6v/nodePools/nps-420-422 -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"4.20.0-0.nightly-multi-2026-09-16-210648\": must be at least 4.20.8", "target": "properties.version.id" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:459]: failed to create nodepool nps-420-422 at version 4.20.0-0.nightly-multi-2026-09-16-210648 Unexpected error: <fmt.wrapError \| 0xc00011be60>: failed to create NodePool nps-420-422: failed starting nodepool creation "nps-420-422" for cluster "np-skip-minor-tcgt6v" in resourcegroup="rg-np-skip-minor-tcgt6v-cwclsxtppk9g": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-skip-minor-tcgt6v-cwclsxtppk9g/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-skip-minor-tcgt6v/nodePools/nps-420-422 -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"4.20.0-0.nightly-multi-2026-09-16-210648\": must be at least 4.20.8", "target": "properties.version.id" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InvalidRequestContent; detail message Invalid value: "<ocp-nightly>": must be at least 4.20.8; provider Microsoft.RedHatOpenShift 23:00:32 UTC · failed at e2e · test Customer should downgrade a nodepool to a lower minor version from 4.22.zLatest to 4.20.zLatest · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:766]: failed to update nodepool npdg-422-420 to downgrade target 4.20.0-0.nightly-multi-2026-09-16-210648 Unexpected error: <fmt.wrapError \| 0xc0013480c0>: failed to start nodepool "npdg-422-420" update in cluster "np-dg-minor-mpcclt" resourcegroup="rg-np-dg-minor-mpcclt-kpfflj2pg95l": PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-dg-minor-mpcclt-kpfflj2pg95l/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-dg-minor-mpcclt/nodePools/npdg-422-420 -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"4.20.0-0.nightly-multi-2026-09-16-210648\": must be at least 4.20.8", "target": "properties.version.id" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:766]: failed to update nodepool npdg-422-420 to downgrade target 4.20.0-0.nightly-multi-2026-09-16-210648 Unexpected error: <fmt.wrapError \| 0xc0013480c0>: failed to start nodepool "npdg-422-420" update in cluster "np-dg-minor-mpcclt" resourcegroup="rg-np-dg-minor-mpcclt-kpfflj2pg95l": PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-dg-minor-mpcclt-kpfflj2pg95l/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-dg-minor-mpcclt/nodePools/npdg-422-420 -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"4.20.0-0.nightly-multi-2026-09-16-210648\": must be at least 4.20.8", "target": "properties.version.id" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InvalidRequestContent; detail message Invalid value: "<ocp-nightly>": must be at least 4.20.8; provider Microsoft.RedHatOpenShift 23:00:32 UTC · failed at e2e · test Customer should upgrade and update a nodepool from 4.20.z to 4.21.zLatest · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:165]: failed to create node pool npupgrade-4-20 with version 4.20.0-0.nightly-multi-2026-09-16-210648 Unexpected error: <fmt.wrapError \| 0xc00160a220>: failed to create NodePool npupgrade-4-20: failed starting nodepool creation "npupgrade-4-20" for cluster "np-version-upgrade-cluster-ddwqfs" in resourcegroup="rg-np-version-upgrade-ddwqfs-m7xcgsn6lcv6": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-version-upgrade-ddwqfs-m7xcgsn6lcv6/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-version-upgrade-cluster-ddwqfs/nodePools/npupgrade-4-20 -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"4.20.0-0.nightly-multi-2026-09-16-210648\": must be at least 4.20.8", "target": "properties.version.id" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:165]: failed to create node pool npupgrade-4-20 with version 4.20.0-0.nightly-multi-2026-09-16-210648 Unexpected error: <fmt.wrapError \| 0xc00160a220>: failed to create NodePool npupgrade-4-20: failed starting nodepool creation "npupgrade-4-20" for cluster "np-version-upgrade-cluster-ddwqfs" in resourcegroup="rg-np-version-upgrade-ddwqfs-m7xcgsn6lcv6": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-version-upgrade-ddwqfs-m7xcgsn6lcv6/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-version-upgrade-cluster-ddwqfs/nodePools/npupgrade-4-20 -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"4.20.0-0.nightly-multi-2026-09-16-210648\": must be at least 4.20.8", "target": "properties.version.id" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InvalidRequestContent; detail message Invalid value: "<ocp-nightly>": must be at least 4.20.8; provider Microsoft.RedHatOpenShift 23:00:32 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP nightly channel for 4.20 · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:175]: failed to create node pool "np-xrmv7w" for cluster "cluster-nightly-4-20-xrmv7w" Unexpected error: <fmt.wrapError \| 0xc000a98aa0>: failed to create NodePool np-xrmv7w: failed starting nodepool creation "np-xrmv7w" for cluster "cluster-nightly-4-20-xrmv7w" in resourcegroup="rg-nightly-4-20-xx4pb26cwd2j": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-nightly-4-20-xx4pb26cwd2j/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-nightly-4-20-xrmv7w/nodePools/np-xrmv7w -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"4.20.0-0.nightly-multi-2026-09-16-210648\": must be at least 4.20.8", "target": "properties.version.id" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:175]: failed to create node pool "np-xrmv7w" for cluster "cluster-nightly-4-20-xrmv7w" Unexpected error: <fmt.wrapError \| 0xc000a98aa0>: failed to create NodePool np-xrmv7w: failed starting nodepool creation "np-xrmv7w" for cluster "cluster-nightly-4-20-xrmv7w" in resourcegroup="rg-nightly-4-20-xx4pb26cwd2j": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-nightly-4-20-xx4pb26cwd2j/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-nightly-4-20-xrmv7w/nodePools/np-xrmv7w -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: InvalidRequestContent -------------------------------------------------------------------------------- { "error": { "code": "InvalidRequestContent", "message": "Invalid value: \"4.20.0-0.nightly-multi-2026-09-16-210648\": must be at least 4.20.8", "target": "properties.version.id" } } -------------------------------------------------------------------------------- ... occurred
21:10:35	2h 32m 29s	branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel post-goodmerged PR No additional run metadata captured	eastus2	n/a	passed	n/a	0	n/a
17:06:01	3h 11m 28s	branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel post-goodmerged PR No additional run metadata captured	eastus2	e2e	failed	n/a	1	e2e (1) ERROR CODE: InternalServerError; detail message [hypershiftHostedCluster] hosted cluster is not available: KubeconfigWaitingForCreate: Waiting for hosted control plane kubeconfig to be created; hosted cluster degraded: UnavailableReplicas: [capi-provider]; provider Microsoft.RedHatOpenShift 17:06:01 UTC · failed at e2e · test Customer should be able to issue the default ingress serving certificate from Let's Encrypt using cert-manager with ACME DNS-01 challenges · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_ingress_cert_letsencrypt.go:123]: failed to create HCP cluster Unexpected error: <fmt.wrapError \| 0xc0014ced60>: failed to create HCP cluster ingress-cert-le: failed waiting for cluster="ingress-cert-le" in resourcegroup="ingress-cert-le-wvt6tvvldjdg" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2/hcpOperationStatuses/80eaa57a-6c21-4104-9379-5f7032bfe26e -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2/hcpOperationStatuses/80eaa57a-6c21-4104-9379-5f7032bfe26e", "name": "80eaa57a-6c21-4104-9379-5f7032bfe26e", "status": "Failed", "startTime": "2026-09-18T18:06:15.268087105Z", "endTime": "2026-09-18T18:35:15.027123676Z", "error": { "code": "InternalServerError", "message": "[hypershiftHostedCluster] hosted cluster is not available: KubeconfigWaitingForCreate: Waiting for hosted control plane kubeconfig to be created; hosted cluster degraded: UnavailableReplicas: capi-provider deployment has 1 unavailable replicas" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_ingress_cert_letsencrypt.go:123]: failed to create HCP cluster Unexpected error: <fmt.wrapError \| 0xc0014ced60>: failed to create HCP cluster ingress-cert-le: failed waiting for cluster="ingress-cert-le" in resourcegroup="ingress-cert-le-wvt6tvvldjdg" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2/hcpOperationStatuses/80eaa57a-6c21-4104-9379-5f7032bfe26e -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2/hcpOperationStatuses/80eaa57a-6c21-4104-9379-5f7032bfe26e", "name": "80eaa57a-6c21-4104-9379-5f7032bfe26e", "status": "Failed", "startTime": "2026-09-18T18:06:15.268087105Z", "endTime": "2026-09-18T18:35:15.027123676Z", "error": { "code": "InternalServerError", "message": "[hypershiftHostedCluster] hosted cluster is not available: KubeconfigWaitingForCreate: Waiting for hosted control plane kubeconfig to be created; hosted cluster degraded: UnavailableReplicas: capi-provider deployment has 1 unavailable replicas" } } -------------------------------------------------------------------------------- ... occurred
17:05:15	3h 32m 48s	branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel post-goodmerged PR No additional run metadata captured	westeurope	n/a	passed	n/a	0	n/a
17:02:57	3h 50m 1s	branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel post-goodmerged PR No additional run metadata captured	brazilsouth	e2e	failed	n/a	54	e2e (54) ERROR CODE: InternalServerError; detail message cluster placement did not complete before the deadline; provider Microsoft.RedHatOpenShift 17:02:57 UTC · failed at e2e · test Customer should be able to create a cluster with default autoscaling and a nodepool with autoscaling enabled up to replica limits · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:102]: failed to create HCP cluster np-autoscale-cluster Unexpected error: <fmt.wrapError \| 0xc000de50e0>: failed to create HCP cluster np-autoscale-cluster: failed waiting for cluster="np-autoscale-cluster" in resourcegroup="np-autoscaling-bddwvtnwz8x8" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/1de5df02-d5bc-4ea1-8f35-f119420c215b -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/1de5df02-d5bc-4ea1-8f35-f119420c215b", "name": "1de5df02-d5bc-4ea1-8f35-f119420c215b", "status": "Failed", "startTime": "2026-09-18T19:54:45.082839017Z", "endTime": "2026-09-18T20:23:47.024709747Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:102]: failed to create HCP cluster np-autoscale-cluster Unexpected error: <fmt.wrapError \| 0xc000de50e0>: failed to create HCP cluster np-autoscale-cluster: failed waiting for cluster="np-autoscale-cluster" in resourcegroup="np-autoscaling-bddwvtnwz8x8" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/1de5df02-d5bc-4ea1-8f35-f119420c215b -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/1de5df02-d5bc-4ea1-8f35-f119420c215b", "name": "1de5df02-d5bc-4ea1-8f35-f119420c215b", "status": "Failed", "startTime": "2026-09-18T19:54:45.082839017Z", "endTime": "2026-09-18T20:23:47.024709747Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message cluster placement did not complete before the deadline; provider Microsoft.RedHatOpenShift 17:02:57 UTC · failed at e2e · test Customer should be able to create a no-CNI private cluster with a private key vault, a nodepool and install cilium CNI successfully · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:106]: failed to create HCP cluster "cilium-cluster" with no CNI and private etcd Unexpected error: <fmt.wrapError \| 0xc000a60580>: failed waiting for cluster="cilium-cluster" in resourcegroup="complex-cilium-kv-65kllg7cv2w7" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/e3de0bda-9b1d-4eb0-bf9e-927086d1d405 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/e3de0bda-9b1d-4eb0-bf9e-927086d1d405", "name": "e3de0bda-9b1d-4eb0-bf9e-927086d1d405", "status": "Failed", "startTime": "2026-09-18T18:06:07.854642874Z", "endTime": "2026-09-18T18:35:13.922823026Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:106]: failed to create HCP cluster "cilium-cluster" with no CNI and private etcd Unexpected error: <fmt.wrapError \| 0xc000a60580>: failed waiting for cluster="cilium-cluster" in resourcegroup="complex-cilium-kv-65kllg7cv2w7" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/e3de0bda-9b1d-4eb0-bf9e-927086d1d405 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/e3de0bda-9b1d-4eb0-bf9e-927086d1d405", "name": "e3de0bda-9b1d-4eb0-bf9e-927086d1d405", "status": "Failed", "startTime": "2026-09-18T18:06:07.854642874Z", "endTime": "2026-09-18T18:35:13.922823026Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message cluster placement did not complete before the deadline; provider Microsoft.RedHatOpenShift 17:02:57 UTC · failed at e2e · test Customer should be able to successfully upgrade control plane minor version from 4.22 minor to 5.0 minor · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:133]: failed to create HCP cluster "cp-ystream-upgrade-5-0-w7lg9w" at install version 4.22 Unexpected error: <fmt.wrapError \| 0xc0017e2320>: failed to create HCP cluster cp-ystream-upgrade-5-0-w7lg9w: failed waiting for cluster="cp-ystream-upgrade-5-0-w7lg9w" in resourcegroup="rg-cp-ystream-upgrade-5-0-whnjtcclp2gm" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/58ae58c3-0f2b-4f24-99ef-9d3487e0c74e -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/58ae58c3-0f2b-4f24-99ef-9d3487e0c74e", "name": "58ae58c3-0f2b-4f24-99ef-9d3487e0c74e", "status": "Failed", "startTime": "2026-09-18T19:01:40.681093019Z", "endTime": "2026-09-18T19:30:40.290116436Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:133]: failed to create HCP cluster "cp-ystream-upgrade-5-0-w7lg9w" at install version 4.22 Unexpected error: <fmt.wrapError \| 0xc0017e2320>: failed to create HCP cluster cp-ystream-upgrade-5-0-w7lg9w: failed waiting for cluster="cp-ystream-upgrade-5-0-w7lg9w" in resourcegroup="rg-cp-ystream-upgrade-5-0-whnjtcclp2gm" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/58ae58c3-0f2b-4f24-99ef-9d3487e0c74e -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/58ae58c3-0f2b-4f24-99ef-9d3487e0c74e", "name": "58ae58c3-0f2b-4f24-99ef-9d3487e0c74e", "status": "Failed", "startTime": "2026-09-18T19:01:40.681093019Z", "endTime": "2026-09-18T19:30:40.290116436Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message cluster placement did not complete before the deadline; provider Microsoft.RedHatOpenShift 17:02:57 UTC · failed at e2e · test Create HCPOpenShiftCluster with Private KeyVault should create a cluster with private keyvault using v20251223preview API · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:98]: failed to create HCP cluster "private-kv-cluster" with private keyvault Unexpected error: <fmt.wrapError \| 0xc0010085a0>: failed waiting for cluster="private-kv-cluster" in resourcegroup="private-keyvault-266qhn49mfp5" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/724dbb1d-6cfa-4cff-8bf4-5e1e742772df -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/724dbb1d-6cfa-4cff-8bf4-5e1e742772df", "name": "724dbb1d-6cfa-4cff-8bf4-5e1e742772df", "status": "Failed", "startTime": "2026-09-18T18:06:21.25552087Z", "endTime": "2026-09-18T18:35:23.923631594Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:98]: failed to create HCP cluster "private-kv-cluster" with private keyvault Unexpected error: <fmt.wrapError \| 0xc0010085a0>: failed waiting for cluster="private-kv-cluster" in resourcegroup="private-keyvault-266qhn49mfp5" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/724dbb1d-6cfa-4cff-8bf4-5e1e742772df -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/724dbb1d-6cfa-4cff-8bf4-5e1e742772df", "name": "724dbb1d-6cfa-4cff-8bf4-5e1e742772df", "status": "Failed", "startTime": "2026-09-18T18:06:21.25552087Z", "endTime": "2026-09-18T18:35:23.923631594Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message cluster placement did not complete before the deadline; provider Microsoft.RedHatOpenShift 17:02:57 UTC · failed at e2e · test Customer should respect cluster-wide node limits with nodepool autoscaling · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:291]: failed to create HCP cluster node-limit-cluster with MaxNodesTotal limit Unexpected error: <fmt.wrapError \| 0xc000abc600>: failed to create HCP cluster node-limit-cluster: failed waiting for cluster="node-limit-cluster" in resourcegroup="node-limit-test-nhxrjzhdkw4m" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/954f356d-f569-48c9-8a16-ac8774f18b21 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/954f356d-f569-48c9-8a16-ac8774f18b21", "name": "954f356d-f569-48c9-8a16-ac8774f18b21", "status": "Failed", "startTime": "2026-09-18T19:54:57.113094659Z", "endTime": "2026-09-18T20:23:57.027116398Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:291]: failed to create HCP cluster node-limit-cluster with MaxNodesTotal limit Unexpected error: <fmt.wrapError \| 0xc000abc600>: failed to create HCP cluster node-limit-cluster: failed waiting for cluster="node-limit-cluster" in resourcegroup="node-limit-test-nhxrjzhdkw4m" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/954f356d-f569-48c9-8a16-ac8774f18b21 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/954f356d-f569-48c9-8a16-ac8774f18b21", "name": "954f356d-f569-48c9-8a16-ac8774f18b21", "status": "Failed", "startTime": "2026-09-18T19:54:57.113094659Z", "endTime": "2026-09-18T20:23:57.027116398Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message cluster placement did not complete before the deadline; provider Microsoft.RedHatOpenShift 17:02:57 UTC · failed at e2e · test Customer should upgrade and update a nodepool from 4.20.z to 4.21.zLatest · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:146]: failed to create HCP cluster np-version-upgrade-cluster-n6q68g with version 4.21.34 Unexpected error: <fmt.wrapError \| 0xc000be8cc0>: failed to create HCP cluster np-version-upgrade-cluster-n6q68g: failed waiting for cluster="np-version-upgrade-cluster-n6q68g" in resourcegroup="rg-np-version-upgrade-n6q68g-z5wvsgjdr7l9" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/47d83203-d5e7-4af0-adbd-1321b73b5d96 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/47d83203-d5e7-4af0-adbd-1321b73b5d96", "name": "47d83203-d5e7-4af0-adbd-1321b73b5d96", "status": "Failed", "startTime": "2026-09-18T19:55:24.695867176Z", "endTime": "2026-09-18T20:24:27.02816156Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:146]: failed to create HCP cluster np-version-upgrade-cluster-n6q68g with version 4.21.34 Unexpected error: <fmt.wrapError \| 0xc000be8cc0>: failed to create HCP cluster np-version-upgrade-cluster-n6q68g: failed waiting for cluster="np-version-upgrade-cluster-n6q68g" in resourcegroup="rg-np-version-upgrade-n6q68g-z5wvsgjdr7l9" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/47d83203-d5e7-4af0-adbd-1321b73b5d96 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/47d83203-d5e7-4af0-adbd-1321b73b5d96", "name": "47d83203-d5e7-4af0-adbd-1321b73b5d96", "status": "Failed", "startTime": "2026-09-18T19:55:24.695867176Z", "endTime": "2026-09-18T20:24:27.02816156Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message cluster placement did not complete before the deadline; provider Microsoft.RedHatOpenShift 17:02:57 UTC · failed at e2e · test Customer should be able to create an HCP cluster with custom autoscaling and update the autoscaling configuration · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:108]: failed to create HCP cluster "autoscaling-hcp-cluster" with custom autoscaling Unexpected error: <fmt.wrapError \| 0xc0004ef160>: failed to create HCP cluster autoscaling-hcp-cluster: failed waiting for cluster="autoscaling-hcp-cluster" in resourcegroup="autoscaling-cluster-v6jpgxxsfw4m" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/c804e049-d95a-4f1c-8a94-12089c6d4c28 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/c804e049-d95a-4f1c-8a94-12089c6d4c28", "name": "c804e049-d95a-4f1c-8a94-12089c6d4c28", "status": "Failed", "startTime": "2026-09-18T18:04:46.580974029Z", "endTime": "2026-09-18T18:33:53.917669415Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:108]: failed to create HCP cluster "autoscaling-hcp-cluster" with custom autoscaling Unexpected error: <fmt.wrapError \| 0xc0004ef160>: failed to create HCP cluster autoscaling-hcp-cluster: failed waiting for cluster="autoscaling-hcp-cluster" in resourcegroup="autoscaling-cluster-v6jpgxxsfw4m" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/c804e049-d95a-4f1c-8a94-12089c6d4c28 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/c804e049-d95a-4f1c-8a94-12089c6d4c28", "name": "c804e049-d95a-4f1c-8a94-12089c6d4c28", "status": "Failed", "startTime": "2026-09-18T18:04:46.580974029Z", "endTime": "2026-09-18T18:33:53.917669415Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message cluster placement did not complete before the deadline; provider Microsoft.RedHatOpenShift 17:02:57 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.22 · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:122]: HCP cluster rg-candidate-4-22-v8gljqqvz6w4/cluster-candidate-4-22-ts6cw9 should provision Unexpected error: <fmt.wrapError \| 0xc0013bebc0>: failed to create HCP cluster cluster-candidate-4-22-ts6cw9: failed waiting for cluster="cluster-candidate-4-22-ts6cw9" in resourcegroup="rg-candidate-4-22-v8gljqqvz6w4" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/6ecb16b4-c24f-4db6-986c-dc7d0a058ebe -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/6ecb16b4-c24f-4db6-986c-dc7d0a058ebe", "name": "6ecb16b4-c24f-4db6-986c-dc7d0a058ebe", "status": "Failed", "startTime": "2026-09-18T19:00:17.002894711Z", "endTime": "2026-09-18T19:29:20.288780423Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:122]: HCP cluster rg-candidate-4-22-v8gljqqvz6w4/cluster-candidate-4-22-ts6cw9 should provision Unexpected error: <fmt.wrapError \| 0xc0013bebc0>: failed to create HCP cluster cluster-candidate-4-22-ts6cw9: failed waiting for cluster="cluster-candidate-4-22-ts6cw9" in resourcegroup="rg-candidate-4-22-v8gljqqvz6w4" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/6ecb16b4-c24f-4db6-986c-dc7d0a058ebe -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/6ecb16b4-c24f-4db6-986c-dc7d0a058ebe", "name": "6ecb16b4-c24f-4db6-986c-dc7d0a058ebe", "status": "Failed", "startTime": "2026-09-18T19:00:17.002894711Z", "endTime": "2026-09-18T19:29:20.288780423Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message cluster placement did not complete before the deadline; provider Microsoft.RedHatOpenShift 17:02:57 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 4.20 · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:123]: failed to create HCP cluster "cluster-zstream-4-20-hs7c8h" with version 4.20.37 on candidate channel Unexpected error: <fmt.wrapError \| 0xc001204dc0>: failed to create HCP cluster cluster-zstream-4-20-hs7c8h: failed waiting for cluster="cluster-zstream-4-20-hs7c8h" in resourcegroup="rg-zstream-upgrade-4-20-4v6msf6jsvhm" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/fe569f4a-6146-4ebf-90e6-4fef2b13a008 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/fe569f4a-6146-4ebf-90e6-4fef2b13a008", "name": "fe569f4a-6146-4ebf-90e6-4fef2b13a008", "status": "Failed", "startTime": "2026-09-18T19:00:45.232517978Z", "endTime": "2026-09-18T19:29:50.287676961Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:123]: failed to create HCP cluster "cluster-zstream-4-20-hs7c8h" with version 4.20.37 on candidate channel Unexpected error: <fmt.wrapError \| 0xc001204dc0>: failed to create HCP cluster cluster-zstream-4-20-hs7c8h: failed waiting for cluster="cluster-zstream-4-20-hs7c8h" in resourcegroup="rg-zstream-upgrade-4-20-4v6msf6jsvhm" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/fe569f4a-6146-4ebf-90e6-4fef2b13a008 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/fe569f4a-6146-4ebf-90e6-4fef2b13a008", "name": "fe569f4a-6146-4ebf-90e6-4fef2b13a008", "status": "Failed", "startTime": "2026-09-18T19:00:45.232517978Z", "endTime": "2026-09-18T19:29:50.287676961Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message cluster placement did not complete before the deadline; provider Microsoft.RedHatOpenShift 17:02:57 UTC · failed at e2e · test Customer should be able to update node pool labels and taints · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_labels_taints.go:92]: failed to create HCP cluster cluster-np-labels-taints Unexpected error: <fmt.wrapError \| 0xc000ae2e60>: failed to create HCP cluster cluster-np-labels-taints: failed waiting for cluster="cluster-np-labels-taints" in resourcegroup="rg-np-labels-taints-92vpld7m4q55" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/043f530f-1cb5-4b43-ae85-55bc6d808229 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/043f530f-1cb5-4b43-ae85-55bc6d808229", "name": "043f530f-1cb5-4b43-ae85-55bc6d808229", "status": "Failed", "startTime": "2026-09-18T19:55:24.023266499Z", "endTime": "2026-09-18T20:24:27.026850238Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_labels_taints.go:92]: failed to create HCP cluster cluster-np-labels-taints Unexpected error: <fmt.wrapError \| 0xc000ae2e60>: failed to create HCP cluster cluster-np-labels-taints: failed waiting for cluster="cluster-np-labels-taints" in resourcegroup="rg-np-labels-taints-92vpld7m4q55" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/043f530f-1cb5-4b43-ae85-55bc6d808229 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/043f530f-1cb5-4b43-ae85-55bc6d808229", "name": "043f530f-1cb5-4b43-ae85-55bc6d808229", "status": "Failed", "startTime": "2026-09-18T19:55:24.023266499Z", "endTime": "2026-09-18T20:24:27.026850238Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred timeout during CreateHCPClusterAndWait; context deadline exceeded 17:02:57 UTC · failed at e2e · test Customer should be able to create an HCP cluster with back-level version 4.20 · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:152]: failed to create HCP cluster version 4.20 Unexpected error: <fmt.wrapErrors \| 0xc0010a86c0>: failed waiting for cluster="cluster-ver-4-20" in resourcegroup="rg-cluster-back-version-ql4cwwsm948w" to finish creating, caused by: timeout '30.000000' minutes exceeded during CreateHCPClusterAndWait for cluster cluster-ver-4-20 in resource group rg-cluster-back-version-ql4cwwsm948w, error: context deadline exceeded ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:152]: failed to create HCP cluster version 4.20 Unexpected error: <fmt.wrapErrors \| 0xc0010a86c0>: failed waiting for cluster="cluster-ver-4-20" in resourcegroup="rg-cluster-back-version-ql4cwwsm948w" to finish creating, caused by: timeout '30.000000' minutes exceeded during CreateHCPClusterAndWait for cluster cluster-ver-4-20 in resource group rg-cluster-back-version-ql4cwwsm948w, error: context deadline exceeded ... occurred Customer should be able to test admin credentials before cluster ready, then full admin credential lifecycle: Cluster provisioning failed 17:02:57 UTC · failed at e2e · test Customer should be able to test admin credentials before cluster ready, then full admin credential lifecycle · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:158]: Cluster provisioning failed fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:158]: Cluster provisioning failed ERROR CODE: InternalServerError; detail message cluster placement did not complete before the deadline; provider Microsoft.RedHatOpenShift 17:02:57 UTC · failed at e2e · test Customer should create an HCP cluster and validate TLS certificates · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:96]: failed to create HCP cluster for TLS endpoint test Unexpected error: <fmt.wrapError \| 0xc000872e40>: failed to create HCP cluster tls-endpoint-hcp-cluster: failed waiting for cluster="tls-endpoint-hcp-cluster" in resourcegroup="tls-endpoint-cluster-9pzlkssjh8rk" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/32dd996c-52a3-42f2-90b1-59c3004ebd0e -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/32dd996c-52a3-42f2-90b1-59c3004ebd0e", "name": "32dd996c-52a3-42f2-90b1-59c3004ebd0e", "status": "Failed", "startTime": "2026-09-18T18:59:49.139622647Z", "endTime": "2026-09-18T19:28:50.286578702Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:96]: failed to create HCP cluster for TLS endpoint test Unexpected error: <fmt.wrapError \| 0xc000872e40>: failed to create HCP cluster tls-endpoint-hcp-cluster: failed waiting for cluster="tls-endpoint-hcp-cluster" in resourcegroup="tls-endpoint-cluster-9pzlkssjh8rk" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/32dd996c-52a3-42f2-90b1-59c3004ebd0e -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/32dd996c-52a3-42f2-90b1-59c3004ebd0e", "name": "32dd996c-52a3-42f2-90b1-59c3004ebd0e", "status": "Failed", "startTime": "2026-09-18T18:59:49.139622647Z", "endTime": "2026-09-18T19:28:50.286578702Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message cluster placement did not complete before the deadline; provider Microsoft.RedHatOpenShift 17:02:57 UTC · failed at e2e · test Customer should be able to lifecycle and confirm external auth on a cluster · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_list_and_verify.go:87]: failed to create HCP cluster for external auth list test Unexpected error: <fmt.wrapError \| 0xc000730dc0>: failed to create HCP cluster ea-listht4pz6: failed waiting for cluster="ea-listht4pz6" in resourcegroup="ea-list-zfzjn95sp6z4" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/1e50a71b-2480-4fde-a3f4-65f1ffbe4dbf -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/1e50a71b-2480-4fde-a3f4-65f1ffbe4dbf", "name": "1e50a71b-2480-4fde-a3f4-65f1ffbe4dbf", "status": "Failed", "startTime": "2026-09-18T19:02:22.470708216Z", "endTime": "2026-09-18T19:31:30.292183292Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_list_and_verify.go:87]: failed to create HCP cluster for external auth list test Unexpected error: <fmt.wrapError \| 0xc000730dc0>: failed to create HCP cluster ea-listht4pz6: failed waiting for cluster="ea-listht4pz6" in resourcegroup="ea-list-zfzjn95sp6z4" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/1e50a71b-2480-4fde-a3f4-65f1ffbe4dbf -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/1e50a71b-2480-4fde-a3f4-65f1ffbe4dbf", "name": "1e50a71b-2480-4fde-a3f4-65f1ffbe4dbf", "status": "Failed", "startTime": "2026-09-18T19:02:22.470708216Z", "endTime": "2026-09-18T19:31:30.292183292Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message cluster placement did not complete before the deadline; provider Microsoft.RedHatOpenShift 17:02:57 UTC · failed at e2e · test Customer should create a cluster with private KAS and verify API server is only reachable from VNet while ingress remains public · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:90]: failed to create HCP cluster "private-kas" with private KAS Unexpected error: <fmt.wrapError \| 0xc000d0e460>: failed to create HCP cluster private-kas: failed waiting for cluster="private-kas" in resourcegroup="private-kas-gq5kj4t7v9lj" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/33d2aa70-41e7-4e37-bdad-c413801367c0 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/33d2aa70-41e7-4e37-bdad-c413801367c0", "name": "33d2aa70-41e7-4e37-bdad-c413801367c0", "status": "Failed", "startTime": "2026-09-18T18:05:54.936239942Z", "endTime": "2026-09-18T18:35:03.923586817Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:90]: failed to create HCP cluster "private-kas" with private KAS Unexpected error: <fmt.wrapError \| 0xc000d0e460>: failed to create HCP cluster private-kas: failed waiting for cluster="private-kas" in resourcegroup="private-kas-gq5kj4t7v9lj" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/33d2aa70-41e7-4e37-bdad-c413801367c0 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/33d2aa70-41e7-4e37-bdad-c413801367c0", "name": "33d2aa70-41e7-4e37-bdad-c413801367c0", "status": "Failed", "startTime": "2026-09-18T18:05:54.936239942Z", "endTime": "2026-09-18T18:35:03.923586817Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message cluster placement did not complete before the deadline; provider Microsoft.RedHatOpenShift 17:02:57 UTC · failed at e2e · test Customer should be able to issue the default ingress serving certificate from Let's Encrypt using cert-manager with ACME DNS-01 challenges · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_ingress_cert_letsencrypt.go:123]: failed to create HCP cluster Unexpected error: <fmt.wrapError \| 0xc0007e7760>: failed to create HCP cluster ingress-cert-le: failed waiting for cluster="ingress-cert-le" in resourcegroup="ingress-cert-le-krl8lpfgmlzn" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/4dc5125d-f2b4-4bbf-8e4e-933cde635436 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/4dc5125d-f2b4-4bbf-8e4e-933cde635436", "name": "4dc5125d-f2b4-4bbf-8e4e-933cde635436", "status": "Failed", "startTime": "2026-09-18T18:04:32.707548645Z", "endTime": "2026-09-18T18:33:33.918055388Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_ingress_cert_letsencrypt.go:123]: failed to create HCP cluster Unexpected error: <fmt.wrapError \| 0xc0007e7760>: failed to create HCP cluster ingress-cert-le: failed waiting for cluster="ingress-cert-le" in resourcegroup="ingress-cert-le-krl8lpfgmlzn" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/4dc5125d-f2b4-4bbf-8e4e-933cde635436 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/4dc5125d-f2b4-4bbf-8e4e-933cde635436", "name": "4dc5125d-f2b4-4bbf-8e4e-933cde635436", "status": "Failed", "startTime": "2026-09-18T18:04:32.707548645Z", "endTime": "2026-09-18T18:33:33.918055388Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message cluster placement did not complete before the deadline; provider Microsoft.RedHatOpenShift 17:02:57 UTC · failed at e2e · test Customer should be able to create a cluster with an external auth config and get the external auth config · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:94]: failed to create HCP cluster for external auth test Unexpected error: <fmt.wrapError \| 0xc00077bdc0>: failed to create HCP cluster ea-cluster: failed waiting for cluster="ea-cluster" in resourcegroup="external-auth-cluster-tsw82p5n5s4n" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/5ff71734-3c8e-483d-9262-a12ff7b3f337 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/5ff71734-3c8e-483d-9262-a12ff7b3f337", "name": "5ff71734-3c8e-483d-9262-a12ff7b3f337", "status": "Failed", "startTime": "2026-09-18T19:02:11.953070547Z", "endTime": "2026-09-18T19:31:20.290227685Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:94]: failed to create HCP cluster for external auth test Unexpected error: <fmt.wrapError \| 0xc00077bdc0>: failed to create HCP cluster ea-cluster: failed waiting for cluster="ea-cluster" in resourcegroup="external-auth-cluster-tsw82p5n5s4n" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/5ff71734-3c8e-483d-9262-a12ff7b3f337 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/5ff71734-3c8e-483d-9262-a12ff7b3f337", "name": "5ff71734-3c8e-483d-9262-a12ff7b3f337", "status": "Failed", "startTime": "2026-09-18T19:02:11.953070547Z", "endTime": "2026-09-18T19:31:20.290227685Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message cluster placement did not complete before the deadline; provider Microsoft.RedHatOpenShift 17:02:57 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.21 · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:122]: HCP cluster rg-candidate-4-21-vnd6pm4rt5md/cluster-candidate-4-21-gkgzcb should provision Unexpected error: <fmt.wrapError \| 0xc0013ec140>: failed to create HCP cluster cluster-candidate-4-21-gkgzcb: failed waiting for cluster="cluster-candidate-4-21-gkgzcb" in resourcegroup="rg-candidate-4-21-vnd6pm4rt5md" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/fefb2d3d-5d17-4ba0-9f25-ca66939b6a12 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/fefb2d3d-5d17-4ba0-9f25-ca66939b6a12", "name": "fefb2d3d-5d17-4ba0-9f25-ca66939b6a12", "status": "Failed", "startTime": "2026-09-18T19:00:14.815434003Z", "endTime": "2026-09-18T19:29:20.286956387Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:122]: HCP cluster rg-candidate-4-21-vnd6pm4rt5md/cluster-candidate-4-21-gkgzcb should provision Unexpected error: <fmt.wrapError \| 0xc0013ec140>: failed to create HCP cluster cluster-candidate-4-21-gkgzcb: failed waiting for cluster="cluster-candidate-4-21-gkgzcb" in resourcegroup="rg-candidate-4-21-vnd6pm4rt5md" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/fefb2d3d-5d17-4ba0-9f25-ca66939b6a12 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/fefb2d3d-5d17-4ba0-9f25-ca66939b6a12", "name": "fefb2d3d-5d17-4ba0-9f25-ca66939b6a12", "status": "Failed", "startTime": "2026-09-18T19:00:14.815434003Z", "endTime": "2026-09-18T19:29:20.286956387Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message cluster placement did not complete before the deadline; provider Microsoft.RedHatOpenShift 17:02:57 UTC · failed at e2e · test Customer should be able to delete an HCP cluster whose managed identities were deleted first · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delete_missing_identities.go:113]: failed to create HCP cluster "missing-mi-hcp-cluster" Unexpected error: <fmt.wrapError \| 0xc000d2c500>: failed to create HCP cluster missing-mi-hcp-cluster: failed waiting for cluster="missing-mi-hcp-cluster" in resourcegroup="delete-mi-rg-djhd95nkq4ln" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/f1f0eeaa-4da5-4317-9297-5a813aab6f33 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/f1f0eeaa-4da5-4317-9297-5a813aab6f33", "name": "f1f0eeaa-4da5-4317-9297-5a813aab6f33", "status": "Failed", "startTime": "2026-09-18T19:57:48.39283405Z", "endTime": "2026-09-18T20:26:57.037154264Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delete_missing_identities.go:113]: failed to create HCP cluster "missing-mi-hcp-cluster" Unexpected error: <fmt.wrapError \| 0xc000d2c500>: failed to create HCP cluster missing-mi-hcp-cluster: failed waiting for cluster="missing-mi-hcp-cluster" in resourcegroup="delete-mi-rg-djhd95nkq4ln" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/f1f0eeaa-4da5-4317-9297-5a813aab6f33 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/f1f0eeaa-4da5-4317-9297-5a813aab6f33", "name": "f1f0eeaa-4da5-4317-9297-5a813aab6f33", "status": "Failed", "startTime": "2026-09-18T19:57:48.39283405Z", "endTime": "2026-09-18T20:26:57.037154264Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message cluster placement did not complete before the deadline; provider Microsoft.RedHatOpenShift 17:02:57 UTC · failed at e2e · test Customer should create a cluster using v20260901preview API and verify cluster health · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_v20260901preview.go:96]: failed to create HCP cluster "v20260901" via v20260901preview Unexpected error: <fmt.wrapError \| 0xc001314da0>: failed to create HCP cluster v20260901: failed waiting for cluster="v20260901" in resourcegroup="v20260901preview-rgw6mwjd2jdp" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/09ad358c-d8b2-4b77-b6e1-79ace9bdc096 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/09ad358c-d8b2-4b77-b6e1-79ace9bdc096", "name": "09ad358c-d8b2-4b77-b6e1-79ace9bdc096", "status": "Failed", "startTime": "2026-09-18T18:04:35.503440276Z", "endTime": "2026-09-18T18:33:43.918161334Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_v20260901preview.go:96]: failed to create HCP cluster "v20260901" via v20260901preview Unexpected error: <fmt.wrapError \| 0xc001314da0>: failed to create HCP cluster v20260901: failed waiting for cluster="v20260901" in resourcegroup="v20260901preview-rgw6mwjd2jdp" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/09ad358c-d8b2-4b77-b6e1-79ace9bdc096 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/09ad358c-d8b2-4b77-b6e1-79ace9bdc096", "name": "09ad358c-d8b2-4b77-b6e1-79ace9bdc096", "status": "Failed", "startTime": "2026-09-18T18:04:35.503440276Z", "endTime": "2026-09-18T18:33:43.918161334Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message cluster placement did not complete before the deadline; provider Microsoft.RedHatOpenShift 17:02:57 UTC · failed at e2e · test Customer should be able to create an HCP cluster then delete it by deleting the customer resource group · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delete_cx_rg.go:90]: failed to create HCP cluster "cx-rg-hcp-cluster" Unexpected error: <fmt.wrapError \| 0xc0010724a0>: failed to create HCP cluster cx-rg-hcp-cluster: failed waiting for cluster="cx-rg-hcp-cluster" in resourcegroup="cx-rg-cluster-fp6bsqq92kn8" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/55377e80-bcab-47a5-9c3e-4350a26d1727 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/55377e80-bcab-47a5-9c3e-4350a26d1727", "name": "55377e80-bcab-47a5-9c3e-4350a26d1727", "status": "Failed", "startTime": "2026-09-18T18:04:22.41327514Z", "endTime": "2026-09-18T18:33:23.916940698Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delete_cx_rg.go:90]: failed to create HCP cluster "cx-rg-hcp-cluster" Unexpected error: <fmt.wrapError \| 0xc0010724a0>: failed to create HCP cluster cx-rg-hcp-cluster: failed waiting for cluster="cx-rg-hcp-cluster" in resourcegroup="cx-rg-cluster-fp6bsqq92kn8" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/55377e80-bcab-47a5-9c3e-4350a26d1727 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/55377e80-bcab-47a5-9c3e-4350a26d1727", "name": "55377e80-bcab-47a5-9c3e-4350a26d1727", "status": "Failed", "startTime": "2026-09-18T18:04:22.41327514Z", "endTime": "2026-09-18T18:33:23.916940698Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message cluster placement did not complete before the deadline; provider Microsoft.RedHatOpenShift 17:02:57 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 4.21 · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:123]: failed to create HCP cluster "cluster-zstream-4-21-cjg6sk" with version 4.21.33 on candidate channel Unexpected error: <fmt.wrapError \| 0xc000c6ed60>: failed to create HCP cluster cluster-zstream-4-21-cjg6sk: failed waiting for cluster="cluster-zstream-4-21-cjg6sk" in resourcegroup="rg-zstream-upgrade-4-21-h8qwfrzsn2zp" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/e4023d65-ddb6-4faf-abf2-0efb01d18c22 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/e4023d65-ddb6-4faf-abf2-0efb01d18c22", "name": "e4023d65-ddb6-4faf-abf2-0efb01d18c22", "status": "Failed", "startTime": "2026-09-18T19:00:38.211424873Z", "endTime": "2026-09-18T19:29:40.287386024Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:123]: failed to create HCP cluster "cluster-zstream-4-21-cjg6sk" with version 4.21.33 on candidate channel Unexpected error: <fmt.wrapError \| 0xc000c6ed60>: failed to create HCP cluster cluster-zstream-4-21-cjg6sk: failed waiting for cluster="cluster-zstream-4-21-cjg6sk" in resourcegroup="rg-zstream-upgrade-4-21-h8qwfrzsn2zp" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/e4023d65-ddb6-4faf-abf2-0efb01d18c22 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/e4023d65-ddb6-4faf-abf2-0efb01d18c22", "name": "e4023d65-ddb6-4faf-abf2-0efb01d18c22", "status": "Failed", "startTime": "2026-09-18T19:00:38.211424873Z", "endTime": "2026-09-18T19:29:40.287386024Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message cluster placement did not complete before the deadline; provider Microsoft.RedHatOpenShift 17:02:57 UTC · failed at e2e · test Customer should not be able to perform various invalid operations on cluster resources · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/simple_negative_cases.go:110]: failed to create HCP cluster negative-tests-cluster Unexpected error: <fmt.wrapError \| 0xc00107a4a0>: failed to create HCP cluster negative-tests-cluster: failed waiting for cluster="negative-tests-cluster" in resourcegroup="rg-negative-tests-nsq6nncs9jx5" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/d18659c0-ccd6-4533-914c-00e40e166e6f -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/d18659c0-ccd6-4533-914c-00e40e166e6f", "name": "d18659c0-ccd6-4533-914c-00e40e166e6f", "status": "Failed", "startTime": "2026-09-18T19:56:51.185500713Z", "endTime": "2026-09-18T20:25:57.031163299Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/simple_negative_cases.go:110]: failed to create HCP cluster negative-tests-cluster Unexpected error: <fmt.wrapError \| 0xc00107a4a0>: failed to create HCP cluster negative-tests-cluster: failed waiting for cluster="negative-tests-cluster" in resourcegroup="rg-negative-tests-nsq6nncs9jx5" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/d18659c0-ccd6-4533-914c-00e40e166e6f -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/d18659c0-ccd6-4533-914c-00e40e166e6f", "name": "d18659c0-ccd6-4533-914c-00e40e166e6f", "status": "Failed", "startTime": "2026-09-18T19:56:51.185500713Z", "endTime": "2026-09-18T20:25:57.031163299Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message cluster placement did not complete before the deadline; provider Microsoft.RedHatOpenShift 17:02:57 UTC · failed at e2e · test Nodepool Ephemeral OS Disk should create a nodepool with ephemeral OS disk when autoRepair is enabled · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_ephemeral_osdisk.go:98]: failed to create HCP cluster ephemeral-disk Unexpected error: <fmt.wrapError \| 0xc0016ca440>: failed to create HCP cluster ephemeral-disk: failed waiting for cluster="ephemeral-disk" in resourcegroup="ephemeral-osdisk-hx4h2xzxb7kg" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/e5c499c9-a7e3-4879-bf2c-b0c897b22258 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/e5c499c9-a7e3-4879-bf2c-b0c897b22258", "name": "e5c499c9-a7e3-4879-bf2c-b0c897b22258", "status": "Failed", "startTime": "2026-09-18T19:55:09.060511648Z", "endTime": "2026-09-18T20:24:17.026518831Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_ephemeral_osdisk.go:98]: failed to create HCP cluster ephemeral-disk Unexpected error: <fmt.wrapError \| 0xc0016ca440>: failed to create HCP cluster ephemeral-disk: failed waiting for cluster="ephemeral-disk" in resourcegroup="ephemeral-osdisk-hx4h2xzxb7kg" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/e5c499c9-a7e3-4879-bf2c-b0c897b22258 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/e5c499c9-a7e3-4879-bf2c-b0c897b22258", "name": "e5c499c9-a7e3-4879-bf2c-b0c897b22258", "status": "Failed", "startTime": "2026-09-18T19:55:09.060511648Z", "endTime": "2026-09-18T20:24:17.026518831Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message cluster placement did not complete before the deadline; provider Microsoft.RedHatOpenShift 17:02:57 UTC · failed at e2e · test Customer should be able to forward control plane logs to a storage account and Event Hub via shoebox diagnostic settings · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/shoebox_logs.go:272]: failed to create HCP cluster shoebox-hcp-cluster Unexpected error: <fmt.wrapError \| 0xc000b7c320>: failed to create HCP cluster shoebox-hcp-cluster: failed waiting for cluster="shoebox-hcp-cluster" in resourcegroup="shoebox-cluster-ntmzhdz9dn8b" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/304a6a3a-9f5b-4eef-9550-d0ba73b6ef78 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/304a6a3a-9f5b-4eef-9550-d0ba73b6ef78", "name": "304a6a3a-9f5b-4eef-9550-d0ba73b6ef78", "status": "Failed", "startTime": "2026-09-18T19:57:03.328284727Z", "endTime": "2026-09-18T20:26:07.032768851Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/shoebox_logs.go:272]: failed to create HCP cluster shoebox-hcp-cluster Unexpected error: <fmt.wrapError \| 0xc000b7c320>: failed to create HCP cluster shoebox-hcp-cluster: failed waiting for cluster="shoebox-hcp-cluster" in resourcegroup="shoebox-cluster-ntmzhdz9dn8b" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/304a6a3a-9f5b-4eef-9550-d0ba73b6ef78 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/304a6a3a-9f5b-4eef-9550-d0ba73b6ef78", "name": "304a6a3a-9f5b-4eef-9550-d0ba73b6ef78", "status": "Failed", "startTime": "2026-09-18T19:57:03.328284727Z", "endTime": "2026-09-18T20:26:07.032768851Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message cluster placement did not complete before the deadline; provider Microsoft.RedHatOpenShift 17:02:57 UTC · failed at e2e · test Authorized CIDRs Connectivity should allow API access only from authorized VM IP · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:111]: failed to create HCP cluster "cidr-connectivity-test" with authorized CIDRs Unexpected error: <fmt.wrapError \| 0xc000e42a80>: failed to create HCP cluster cidr-connectivity-test: failed waiting for cluster="cidr-connectivity-test" in resourcegroup="e2e-cidr-connectivity-x6ksv4qd5hh4" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/0d11158a-c610-4ba0-b070-6f011685df27 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/0d11158a-c610-4ba0-b070-6f011685df27", "name": "0d11158a-c610-4ba0-b070-6f011685df27", "status": "Failed", "startTime": "2026-09-18T18:06:15.656789631Z", "endTime": "2026-09-18T18:35:23.922596176Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:111]: failed to create HCP cluster "cidr-connectivity-test" with authorized CIDRs Unexpected error: <fmt.wrapError \| 0xc000e42a80>: failed to create HCP cluster cidr-connectivity-test: failed waiting for cluster="cidr-connectivity-test" in resourcegroup="e2e-cidr-connectivity-x6ksv4qd5hh4" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/0d11158a-c610-4ba0-b070-6f011685df27 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/0d11158a-c610-4ba0-b070-6f011685df27", "name": "0d11158a-c610-4ba0-b070-6f011685df27", "status": "Failed", "startTime": "2026-09-18T18:06:15.656789631Z", "endTime": "2026-09-18T18:35:23.922596176Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message cluster placement did not complete before the deadline; provider Microsoft.RedHatOpenShift 17:02:57 UTC · failed at e2e · test Customer should be able to create an HCP cluster and manage ImageDigestMirrors · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/idms_lifecycle.go:113]: failed to create HCP cluster with ImageDigestMirrors Unexpected error: <fmt.wrapError \| 0xc000d3e4e0>: failed to create HCP cluster idms-e2e-hcp-cluster: failed waiting for cluster="idms-e2e-hcp-cluster" in resourcegroup="idms-5l7s7j7vcxkh" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/6488d30f-ab0b-48ea-a0ec-7c581c0e717f -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/6488d30f-ab0b-48ea-a0ec-7c581c0e717f", "name": "6488d30f-ab0b-48ea-a0ec-7c581c0e717f", "status": "Failed", "startTime": "2026-09-18T19:02:09.380870167Z", "endTime": "2026-09-18T19:31:10.29248977Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/idms_lifecycle.go:113]: failed to create HCP cluster with ImageDigestMirrors Unexpected error: <fmt.wrapError \| 0xc000d3e4e0>: failed to create HCP cluster idms-e2e-hcp-cluster: failed waiting for cluster="idms-e2e-hcp-cluster" in resourcegroup="idms-5l7s7j7vcxkh" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/6488d30f-ab0b-48ea-a0ec-7c581c0e717f -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/6488d30f-ab0b-48ea-a0ec-7c581c0e717f", "name": "6488d30f-ab0b-48ea-a0ec-7c581c0e717f", "status": "Failed", "startTime": "2026-09-18T19:02:09.380870167Z", "endTime": "2026-09-18T19:31:10.29248977Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message cluster placement did not complete before the deadline; provider Microsoft.RedHatOpenShift 17:02:57 UTC · failed at e2e · test Customer should be able to delete a nodepool · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_delete.go:80]: failed to create HCP cluster np-delete-hcp-cluster Unexpected error: <fmt.wrapError \| 0xc000ab4bc0>: failed to create HCP cluster np-delete-hcp-cluster: failed waiting for cluster="np-delete-hcp-cluster" in resourcegroup="nodepool-delete-9x5xbv4nzw8b" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/33146f6d-4733-4ec4-b15d-3e685b83f09f -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/33146f6d-4733-4ec4-b15d-3e685b83f09f", "name": "33146f6d-4733-4ec4-b15d-3e685b83f09f", "status": "Failed", "startTime": "2026-09-18T19:55:03.940822361Z", "endTime": "2026-09-18T20:24:07.0250673Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_delete.go:80]: failed to create HCP cluster np-delete-hcp-cluster Unexpected error: <fmt.wrapError \| 0xc000ab4bc0>: failed to create HCP cluster np-delete-hcp-cluster: failed waiting for cluster="np-delete-hcp-cluster" in resourcegroup="nodepool-delete-9x5xbv4nzw8b" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/33146f6d-4733-4ec4-b15d-3e685b83f09f -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/33146f6d-4733-4ec4-b15d-3e685b83f09f", "name": "33146f6d-4733-4ec4-b15d-3e685b83f09f", "status": "Failed", "startTime": "2026-09-18T19:55:03.940822361Z", "endTime": "2026-09-18T20:24:07.0250673Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message cluster placement did not complete before the deadline; provider Microsoft.RedHatOpenShift 17:02:57 UTC · failed at e2e · test Update HCPOpenShiftCluster Negative creates a cluster and fails to update its name with a PATCH request · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:84]: failed to create HCP cluster for patch-name test Unexpected error: <fmt.wrapError \| 0xc000e90f60>: failed to create HCP cluster patch-name-cluster: failed waiting for cluster="patch-name-cluster" in resourcegroup="patch-name-z57bbtc664tj" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/5f403839-210b-4649-acab-f3cebf6c199e -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/5f403839-210b-4649-acab-f3cebf6c199e", "name": "5f403839-210b-4649-acab-f3cebf6c199e", "status": "Failed", "startTime": "2026-09-18T18:59:56.042833825Z", "endTime": "2026-09-18T19:29:00.285060799Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:84]: failed to create HCP cluster for patch-name test Unexpected error: <fmt.wrapError \| 0xc000e90f60>: failed to create HCP cluster patch-name-cluster: failed waiting for cluster="patch-name-cluster" in resourcegroup="patch-name-z57bbtc664tj" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/5f403839-210b-4649-acab-f3cebf6c199e -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/5f403839-210b-4649-acab-f3cebf6c199e", "name": "5f403839-210b-4649-acab-f3cebf6c199e", "status": "Failed", "startTime": "2026-09-18T18:59:56.042833825Z", "endTime": "2026-09-18T19:29:00.285060799Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message cluster placement did not complete before the deadline; provider Microsoft.RedHatOpenShift 17:02:57 UTC · failed at e2e · test Customer should create a cluster with private ingress using v20260901preview and verify the ingress is internal · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_ingress.go:87]: failed to create HCP cluster "private-ingress" with private ingress Unexpected error: <fmt.wrapError \| 0xc0004e17a0>: failed to create HCP cluster private-ingress: failed waiting for cluster="private-ingress" in resourcegroup="private-ingress-qcdwg6ktcch6" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/b8db3dd6-3acf-4a8a-8882-bacb6f6fb9d0 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/b8db3dd6-3acf-4a8a-8882-bacb6f6fb9d0", "name": "b8db3dd6-3acf-4a8a-8882-bacb6f6fb9d0", "status": "Failed", "startTime": "2026-09-18T18:04:24.536461143Z", "endTime": "2026-09-18T18:33:33.916870368Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_ingress.go:87]: failed to create HCP cluster "private-ingress" with private ingress Unexpected error: <fmt.wrapError \| 0xc0004e17a0>: failed to create HCP cluster private-ingress: failed waiting for cluster="private-ingress" in resourcegroup="private-ingress-qcdwg6ktcch6" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/b8db3dd6-3acf-4a8a-8882-bacb6f6fb9d0 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/b8db3dd6-3acf-4a8a-8882-bacb6f6fb9d0", "name": "b8db3dd6-3acf-4a8a-8882-bacb6f6fb9d0", "status": "Failed", "startTime": "2026-09-18T18:04:24.536461143Z", "endTime": "2026-09-18T18:33:33.916870368Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred cluster entered terminal Failed state after role assignment deployment 17:02:57 UTC · failed at e2e · test ARO HCP Service should successfully provision cluster when MI role assignments are added after cluster creation · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:227]: Timed out after 2580.001s. cluster should eventually succeed after role assignments are created The function passed to Eventually failed at /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:223 with: cluster entered terminal Failed state after role assignment deployment Expected <armredhatopenshifthcp.ProvisioningState>: ... not to equal <armredhatopenshifthcp.ProvisioningState>: ... fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:227]: Timed out after 2580.001s. cluster should eventually succeed after role assignments are created The function passed to Eventually failed at /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:223 with: cluster entered terminal Failed state after role assignment deployment Expected <armredhatopenshifthcp.ProvisioningState>: ... not to equal <armredhatopenshifthcp.ProvisioningState>: ... ERROR CODE: InternalServerError; detail message cluster placement did not complete before the deadline; provider Microsoft.RedHatOpenShift 17:02:57 UTC · failed at e2e · test Customer should be able to use workload identity via the cluster OIDC issuer URL · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/oidc_issuer_workload_identity.go:227]: failed to create HCP cluster oidc-wi-cluster Unexpected error: <fmt.wrapError \| 0xc000e9ab80>: failed to create HCP cluster oidc-wi-cluster: failed waiting for cluster="oidc-wi-cluster" in resourcegroup="oidc-wi-qp94zrf9dxjw" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/1869d89d-6245-485d-8118-d3fc6e9af9cc -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/1869d89d-6245-485d-8118-d3fc6e9af9cc", "name": "1869d89d-6245-485d-8118-d3fc6e9af9cc", "status": "Failed", "startTime": "2026-09-18T19:56:31.378329777Z", "endTime": "2026-09-18T20:25:37.031824413Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/oidc_issuer_workload_identity.go:227]: failed to create HCP cluster oidc-wi-cluster Unexpected error: <fmt.wrapError \| 0xc000e9ab80>: failed to create HCP cluster oidc-wi-cluster: failed waiting for cluster="oidc-wi-cluster" in resourcegroup="oidc-wi-qp94zrf9dxjw" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/1869d89d-6245-485d-8118-d3fc6e9af9cc -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/1869d89d-6245-485d-8118-d3fc6e9af9cc", "name": "1869d89d-6245-485d-8118-d3fc6e9af9cc", "status": "Failed", "startTime": "2026-09-18T19:56:31.378329777Z", "endTime": "2026-09-18T20:25:37.031824413Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message cluster placement did not complete before the deadline; provider Microsoft.RedHatOpenShift 17:02:57 UTC · failed at e2e · test Customer should not be able to reuse subnets and NSGs between clusters · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_nsg_subnet_reuse.go:107]: failed to create seed HCP cluster basic-cluster Unexpected error: <fmt.wrapError \| 0xc000fc6480>: failed to create HCP cluster basic-cluster: failed waiting for cluster="basic-cluster" in resourcegroup="rg-cluster-nsg-subnet-reuse-p4nctxb27lkf" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/47247d9a-d966-4134-b361-76bb41af047f -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/47247d9a-d966-4134-b361-76bb41af047f", "name": "47247d9a-d966-4134-b361-76bb41af047f", "status": "Failed", "startTime": "2026-09-18T18:04:22.12537559Z", "endTime": "2026-09-18T18:33:23.915581975Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_nsg_subnet_reuse.go:107]: failed to create seed HCP cluster basic-cluster Unexpected error: <fmt.wrapError \| 0xc000fc6480>: failed to create HCP cluster basic-cluster: failed waiting for cluster="basic-cluster" in resourcegroup="rg-cluster-nsg-subnet-reuse-p4nctxb27lkf" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/47247d9a-d966-4134-b361-76bb41af047f -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/47247d9a-d966-4134-b361-76bb41af047f", "name": "47247d9a-d966-4134-b361-76bb41af047f", "status": "Failed", "startTime": "2026-09-18T18:04:22.12537559Z", "endTime": "2026-09-18T18:33:23.915581975Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message cluster placement did not complete before the deadline; provider Microsoft.RedHatOpenShift 17:02:57 UTC · failed at e2e · test Customer should be able to successfully upgrade control plane minor version from 4.20 minor to 4.21 minor · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:133]: failed to create HCP cluster "cp-ystream-upgrade-4-21-njqww7" at install version 4.20 Unexpected error: <fmt.wrapError \| 0xc000af10c0>: failed to create HCP cluster cp-ystream-upgrade-4-21-njqww7: failed waiting for cluster="cp-ystream-upgrade-4-21-njqww7" in resourcegroup="rg-cp-ystream-upgrade-4-21-jpqnx2lrkvjw" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/b4fccd14-460f-43de-9e90-738bd6033ce0 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/b4fccd14-460f-43de-9e90-738bd6033ce0", "name": "b4fccd14-460f-43de-9e90-738bd6033ce0", "status": "Failed", "startTime": "2026-09-18T19:01:31.270212929Z", "endTime": "2026-09-18T19:30:40.29151616Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:133]: failed to create HCP cluster "cp-ystream-upgrade-4-21-njqww7" at install version 4.20 Unexpected error: <fmt.wrapError \| 0xc000af10c0>: failed to create HCP cluster cp-ystream-upgrade-4-21-njqww7: failed waiting for cluster="cp-ystream-upgrade-4-21-njqww7" in resourcegroup="rg-cp-ystream-upgrade-4-21-jpqnx2lrkvjw" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/b4fccd14-460f-43de-9e90-738bd6033ce0 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/b4fccd14-460f-43de-9e90-738bd6033ce0", "name": "b4fccd14-460f-43de-9e90-738bd6033ce0", "status": "Failed", "startTime": "2026-09-18T19:01:31.270212929Z", "endTime": "2026-09-18T19:30:40.29151616Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message cluster placement did not complete before the deadline; provider Microsoft.RedHatOpenShift 17:02:57 UTC · failed at e2e · test Customer should be able to create a cluster and node pools with aggregated advanced features · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:207]: failed to create HCP cluster "agg-cluster" with aggregated settings Unexpected error: <fmt.wrapError \| 0xc0009b0380>: failed waiting for cluster="agg-cluster" in resourcegroup="feature-aggregation-8dhk5572s4k2" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/0b13dfc6-e74b-46ab-86f5-cfed93dba41f -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/0b13dfc6-e74b-46ab-86f5-cfed93dba41f", "name": "0b13dfc6-e74b-46ab-86f5-cfed93dba41f", "status": "Failed", "startTime": "2026-09-18T18:07:16.233683149Z", "endTime": "2026-09-18T18:36:20.211533932Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:207]: failed to create HCP cluster "agg-cluster" with aggregated settings Unexpected error: <fmt.wrapError \| 0xc0009b0380>: failed waiting for cluster="agg-cluster" in resourcegroup="feature-aggregation-8dhk5572s4k2" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/0b13dfc6-e74b-46ab-86f5-cfed93dba41f -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/0b13dfc6-e74b-46ab-86f5-cfed93dba41f", "name": "0b13dfc6-e74b-46ab-86f5-cfed93dba41f", "status": "Failed", "startTime": "2026-09-18T18:07:16.233683149Z", "endTime": "2026-09-18T18:36:20.211533932Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message cluster placement did not complete before the deadline; provider Microsoft.RedHatOpenShift 17:02:57 UTC · failed at e2e · test Customer should downgrade a nodepool to a lower minor version from 4.22.zLatest to 4.20.zLatest · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:720]: failed to create HCP cluster Unexpected error: <fmt.wrapError \| 0xc000b14ec0>: failed to create HCP cluster np-dg-minor-xwhdp5: failed waiting for cluster="np-dg-minor-xwhdp5" in resourcegroup="rg-np-dg-minor-xwhdp5-592m9fw6ltl5" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/35a3b7f4-5677-4ee3-8047-d4e91fdf91e0 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/35a3b7f4-5677-4ee3-8047-d4e91fdf91e0", "name": "35a3b7f4-5677-4ee3-8047-d4e91fdf91e0", "status": "Failed", "startTime": "2026-09-18T19:56:39.90066671Z", "endTime": "2026-09-18T20:25:47.031092558Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:720]: failed to create HCP cluster Unexpected error: <fmt.wrapError \| 0xc000b14ec0>: failed to create HCP cluster np-dg-minor-xwhdp5: failed waiting for cluster="np-dg-minor-xwhdp5" in resourcegroup="rg-np-dg-minor-xwhdp5-592m9fw6ltl5" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/35a3b7f4-5677-4ee3-8047-d4e91fdf91e0 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/35a3b7f4-5677-4ee3-8047-d4e91fdf91e0", "name": "35a3b7f4-5677-4ee3-8047-d4e91fdf91e0", "status": "Failed", "startTime": "2026-09-18T19:56:39.90066671Z", "endTime": "2026-09-18T20:25:47.031092558Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message cluster placement did not complete before the deadline; provider Microsoft.RedHatOpenShift 17:02:57 UTC · failed at e2e · test Customer should be able to create an HCP cluster and custom node pool osDisk size · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_nodepool_osdisk.go:80]: failed to create HCP cluster "hcp-cluster-np-128" Unexpected error: <fmt.wrapError \| 0xc0010da5a0>: failed to create HCP cluster hcp-cluster-np-128: failed waiting for cluster="hcp-cluster-np-128" in resourcegroup="clusternp128-6ppzp5frfmgz" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/18010207-11d3-4bad-8b26-bfe3951f9a05 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/18010207-11d3-4bad-8b26-bfe3951f9a05", "name": "18010207-11d3-4bad-8b26-bfe3951f9a05", "status": "Failed", "startTime": "2026-09-18T18:04:25.031159118Z", "endTime": "2026-09-18T18:33:33.919612215Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_nodepool_osdisk.go:80]: failed to create HCP cluster "hcp-cluster-np-128" Unexpected error: <fmt.wrapError \| 0xc0010da5a0>: failed to create HCP cluster hcp-cluster-np-128: failed waiting for cluster="hcp-cluster-np-128" in resourcegroup="clusternp128-6ppzp5frfmgz" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/18010207-11d3-4bad-8b26-bfe3951f9a05 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/18010207-11d3-4bad-8b26-bfe3951f9a05", "name": "18010207-11d3-4bad-8b26-bfe3951f9a05", "status": "Failed", "startTime": "2026-09-18T18:04:25.031159118Z", "endTime": "2026-09-18T18:33:33.919612215Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message cluster placement did not complete before the deadline; provider Microsoft.RedHatOpenShift 17:02:57 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 4.22 · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:123]: failed to create HCP cluster "cluster-zstream-4-22-7ggpqb" with version 4.22.13 on candidate channel Unexpected error: <fmt.wrapError \| 0xc001088300>: failed to create HCP cluster cluster-zstream-4-22-7ggpqb: failed waiting for cluster="cluster-zstream-4-22-7ggpqb" in resourcegroup="rg-zstream-upgrade-4-22-nfmkkc97chvl" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/14449ce9-b1ea-49dc-820c-31ab5444c62f -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/14449ce9-b1ea-49dc-820c-31ab5444c62f", "name": "14449ce9-b1ea-49dc-820c-31ab5444c62f", "status": "Failed", "startTime": "2026-09-18T19:00:37.092484844Z", "endTime": "2026-09-18T19:29:40.290112371Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:123]: failed to create HCP cluster "cluster-zstream-4-22-7ggpqb" with version 4.22.13 on candidate channel Unexpected error: <fmt.wrapError \| 0xc001088300>: failed to create HCP cluster cluster-zstream-4-22-7ggpqb: failed waiting for cluster="cluster-zstream-4-22-7ggpqb" in resourcegroup="rg-zstream-upgrade-4-22-nfmkkc97chvl" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/14449ce9-b1ea-49dc-820c-31ab5444c62f -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/14449ce9-b1ea-49dc-820c-31ab5444c62f", "name": "14449ce9-b1ea-49dc-820c-31ab5444c62f", "status": "Failed", "startTime": "2026-09-18T19:00:37.092484844Z", "endTime": "2026-09-18T19:29:40.290112371Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message cluster placement did not complete before the deadline; provider Microsoft.RedHatOpenShift 17:02:57 UTC · failed at e2e · test Customer should be able to create node pools with ARM64-based VMs · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/arm64_nodepool.go:88]: failed to create HCP cluster "arm64-vm-hcp-cluster" Unexpected error: <fmt.wrapError \| 0xc0011faaa0>: failed to create HCP cluster arm64-vm-hcp-cluster: failed waiting for cluster="arm64-vm-hcp-cluster" in resourcegroup="arm64-vm-cluster-smxqhhn278sn" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/ecd24d8c-9b96-49ef-9ee9-932851b8cc59 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/ecd24d8c-9b96-49ef-9ee9-932851b8cc59", "name": "ecd24d8c-9b96-49ef-9ee9-932851b8cc59", "status": "Failed", "startTime": "2026-09-18T18:04:22.39543422Z", "endTime": "2026-09-18T18:33:23.917604609Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/arm64_nodepool.go:88]: failed to create HCP cluster "arm64-vm-hcp-cluster" Unexpected error: <fmt.wrapError \| 0xc0011faaa0>: failed to create HCP cluster arm64-vm-hcp-cluster: failed waiting for cluster="arm64-vm-hcp-cluster" in resourcegroup="arm64-vm-cluster-smxqhhn278sn" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/ecd24d8c-9b96-49ef-9ee9-932851b8cc59 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/ecd24d8c-9b96-49ef-9ee9-932851b8cc59", "name": "ecd24d8c-9b96-49ef-9ee9-932851b8cc59", "status": "Failed", "startTime": "2026-09-18T18:04:22.39543422Z", "endTime": "2026-09-18T18:33:23.917604609Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message cluster placement did not complete before the deadline; provider Microsoft.RedHatOpenShift 17:02:57 UTC · failed at e2e · test Customer should be able to successfully upgrade control plane minor version from 4.21 minor to 4.22 minor · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:133]: failed to create HCP cluster "cp-ystream-upgrade-4-22-kg5zmv" at install version 4.21 Unexpected error: <fmt.wrapError \| 0xc000dfa6c0>: failed to create HCP cluster cp-ystream-upgrade-4-22-kg5zmv: failed waiting for cluster="cp-ystream-upgrade-4-22-kg5zmv" in resourcegroup="rg-cp-ystream-upgrade-4-22-gsppncf9n78j" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/6811d4f8-987c-40e9-a8fb-87314864cef2 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/6811d4f8-987c-40e9-a8fb-87314864cef2", "name": "6811d4f8-987c-40e9-a8fb-87314864cef2", "status": "Failed", "startTime": "2026-09-18T19:01:34.659094453Z", "endTime": "2026-09-18T19:30:40.290599044Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:133]: failed to create HCP cluster "cp-ystream-upgrade-4-22-kg5zmv" at install version 4.21 Unexpected error: <fmt.wrapError \| 0xc000dfa6c0>: failed to create HCP cluster cp-ystream-upgrade-4-22-kg5zmv: failed waiting for cluster="cp-ystream-upgrade-4-22-kg5zmv" in resourcegroup="rg-cp-ystream-upgrade-4-22-gsppncf9n78j" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/6811d4f8-987c-40e9-a8fb-87314864cef2 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/6811d4f8-987c-40e9-a8fb-87314864cef2", "name": "6811d4f8-987c-40e9-a8fb-87314864cef2", "status": "Failed", "startTime": "2026-09-18T19:01:34.659094453Z", "endTime": "2026-09-18T19:30:40.290599044Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message cluster placement did not complete before the deadline; provider Microsoft.RedHatOpenShift 17:02:57 UTC · failed at e2e · test Customer should be able to create a HCP cluster and use cilium CNI plugin · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_cni_cilium.go:88]: failed to create HCP cluster "cilium-cl" without CNI Unexpected error: <fmt.wrapError \| 0xc000dcf3c0>: failed to create HCP cluster cilium-cl: failed waiting for cluster="cilium-cl" in resourcegroup="cni-cilium-2v27j8w6flzw" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/3a82c0c6-4a42-4a86-a966-622844d30b20 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/3a82c0c6-4a42-4a86-a966-622844d30b20", "name": "3a82c0c6-4a42-4a86-a966-622844d30b20", "status": "Failed", "startTime": "2026-09-18T18:04:31.190318654Z", "endTime": "2026-09-18T18:33:33.918230691Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_cni_cilium.go:88]: failed to create HCP cluster "cilium-cl" without CNI Unexpected error: <fmt.wrapError \| 0xc000dcf3c0>: failed to create HCP cluster cilium-cl: failed waiting for cluster="cilium-cl" in resourcegroup="cni-cilium-2v27j8w6flzw" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/3a82c0c6-4a42-4a86-a966-622844d30b20 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/3a82c0c6-4a42-4a86-a966-622844d30b20", "name": "3a82c0c6-4a42-4a86-a966-622844d30b20", "status": "Failed", "startTime": "2026-09-18T18:04:31.190318654Z", "endTime": "2026-09-18T18:33:33.918230691Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message cluster placement did not complete before the deadline; provider Microsoft.RedHatOpenShift 17:02:57 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.20 · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:122]: HCP cluster rg-candidate-4-20-jspqshnm6b8s/cluster-candidate-4-20-26p7kj should provision Unexpected error: <fmt.wrapError \| 0xc000d98400>: failed to create HCP cluster cluster-candidate-4-20-26p7kj: failed waiting for cluster="cluster-candidate-4-20-26p7kj" in resourcegroup="rg-candidate-4-20-jspqshnm6b8s" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/670cbc64-10c9-40e8-ab17-ba6edf9a8da6 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/670cbc64-10c9-40e8-ab17-ba6edf9a8da6", "name": "670cbc64-10c9-40e8-ab17-ba6edf9a8da6", "status": "Failed", "startTime": "2026-09-18T19:00:14.619385936Z", "endTime": "2026-09-18T19:29:20.288515017Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:122]: HCP cluster rg-candidate-4-20-jspqshnm6b8s/cluster-candidate-4-20-26p7kj should provision Unexpected error: <fmt.wrapError \| 0xc000d98400>: failed to create HCP cluster cluster-candidate-4-20-26p7kj: failed waiting for cluster="cluster-candidate-4-20-26p7kj" in resourcegroup="rg-candidate-4-20-jspqshnm6b8s" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/670cbc64-10c9-40e8-ab17-ba6edf9a8da6 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/670cbc64-10c9-40e8-ab17-ba6edf9a8da6", "name": "670cbc64-10c9-40e8-ab17-ba6edf9a8da6", "status": "Failed", "startTime": "2026-09-18T19:00:14.619385936Z", "endTime": "2026-09-18T19:29:20.288515017Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message cluster placement did not complete before the deadline; provider Microsoft.RedHatOpenShift 17:02:57 UTC · failed at e2e · test FIPS Mode Support with v2026-09-01 API should create an HCP cluster with FIPS mode enabled via cryptoRestrictions property · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_fips_mode.go:87]: failed to create HCP cluster "fips-enabled-cluster" with cryptoRestrictions set to FIPS Unexpected error: <fmt.wrapError \| 0xc000b6d540>: failed waiting for cluster="fips-enabled-cluster" in resourcegroup="fips-enabled-rc84t6g5h54p" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/33766b12-eb33-46f0-b3b0-eea3719fedb2 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/33766b12-eb33-46f0-b3b0-eea3719fedb2", "name": "33766b12-eb33-46f0-b3b0-eea3719fedb2", "status": "Failed", "startTime": "2026-09-18T18:04:32.507928047Z", "endTime": "2026-09-18T18:33:33.91760498Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_fips_mode.go:87]: failed to create HCP cluster "fips-enabled-cluster" with cryptoRestrictions set to FIPS Unexpected error: <fmt.wrapError \| 0xc000b6d540>: failed waiting for cluster="fips-enabled-cluster" in resourcegroup="fips-enabled-rc84t6g5h54p" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/33766b12-eb33-46f0-b3b0-eea3719fedb2 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/33766b12-eb33-46f0-b3b0-eea3719fedb2", "name": "33766b12-eb33-46f0-b3b0-eea3719fedb2", "status": "Failed", "startTime": "2026-09-18T18:04:32.507928047Z", "endTime": "2026-09-18T18:33:33.91760498Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message cluster placement did not complete before the deadline; provider Microsoft.RedHatOpenShift 17:02:57 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 5.0 · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:122]: HCP cluster rg-candidate-5-0-pc4nmpfmfd8n/cluster-candidate-5-0-8k9kgf should provision Unexpected error: <fmt.wrapError \| 0xc0008977c0>: failed to create HCP cluster cluster-candidate-5-0-8k9kgf: failed waiting for cluster="cluster-candidate-5-0-8k9kgf" in resourcegroup="rg-candidate-5-0-pc4nmpfmfd8n" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/b151a448-87ca-4125-a955-ff96a58ce701 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/b151a448-87ca-4125-a955-ff96a58ce701", "name": "b151a448-87ca-4125-a955-ff96a58ce701", "status": "Failed", "startTime": "2026-09-18T19:00:04.863718426Z", "endTime": "2026-09-18T19:29:10.287329661Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:122]: HCP cluster rg-candidate-5-0-pc4nmpfmfd8n/cluster-candidate-5-0-8k9kgf should provision Unexpected error: <fmt.wrapError \| 0xc0008977c0>: failed to create HCP cluster cluster-candidate-5-0-8k9kgf: failed waiting for cluster="cluster-candidate-5-0-8k9kgf" in resourcegroup="rg-candidate-5-0-pc4nmpfmfd8n" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/b151a448-87ca-4125-a955-ff96a58ce701 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/b151a448-87ca-4125-a955-ff96a58ce701", "name": "b151a448-87ca-4125-a955-ff96a58ce701", "status": "Failed", "startTime": "2026-09-18T19:00:04.863718426Z", "endTime": "2026-09-18T19:29:10.287329661Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message cluster placement did not complete before the deadline; provider Microsoft.RedHatOpenShift 17:02:57 UTC · failed at e2e · test Customer should upgrade a nodepool skipping one minor version (+2) from 4.20.z to 4.22.zLatest · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:441]: failed to create HCP cluster np-skip-minor-tpqdn5 at version 4.22.14 Unexpected error: <fmt.wrapError \| 0xc000d0e760>: failed to create HCP cluster np-skip-minor-tpqdn5: failed waiting for cluster="np-skip-minor-tpqdn5" in resourcegroup="rg-np-skip-minor-tpqdn5-tc97l4b65b5k" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/06ddea4f-de67-40c5-a480-9d00159e56c4 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/06ddea4f-de67-40c5-a480-9d00159e56c4", "name": "06ddea4f-de67-40c5-a480-9d00159e56c4", "status": "Failed", "startTime": "2026-09-18T19:55:52.428957278Z", "endTime": "2026-09-18T20:24:57.029688909Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:441]: failed to create HCP cluster np-skip-minor-tpqdn5 at version 4.22.14 Unexpected error: <fmt.wrapError \| 0xc000d0e760>: failed to create HCP cluster np-skip-minor-tpqdn5: failed waiting for cluster="np-skip-minor-tpqdn5" in resourcegroup="rg-np-skip-minor-tpqdn5-tc97l4b65b5k" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/06ddea4f-de67-40c5-a480-9d00159e56c4 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/06ddea4f-de67-40c5-a480-9d00159e56c4", "name": "06ddea4f-de67-40c5-a480-9d00159e56c4", "status": "Failed", "startTime": "2026-09-18T19:55:52.428957278Z", "endTime": "2026-09-18T20:24:57.029688909Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message cluster placement did not complete before the deadline; provider Microsoft.RedHatOpenShift 17:02:57 UTC · failed at e2e · test Service Provider should upgrade the control plane z-stream automatically on behalf of the customer for 5.0 · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:123]: failed to create HCP cluster "cluster-zstream-5-0-wlmbkw" with version 5.0.0-rc.1 on candidate channel Unexpected error: <fmt.wrapError \| 0xc000a24ce0>: failed to create HCP cluster cluster-zstream-5-0-wlmbkw: failed waiting for cluster="cluster-zstream-5-0-wlmbkw" in resourcegroup="rg-zstream-upgrade-5-0-88f465g4v8qg" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/8399e46e-e9e9-480e-b60a-122ad9e52531 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/8399e46e-e9e9-480e-b60a-122ad9e52531", "name": "8399e46e-e9e9-480e-b60a-122ad9e52531", "status": "Failed", "startTime": "2026-09-18T19:00:37.645326719Z", "endTime": "2026-09-18T19:29:40.288068036Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:123]: failed to create HCP cluster "cluster-zstream-5-0-wlmbkw" with version 5.0.0-rc.1 on candidate channel Unexpected error: <fmt.wrapError \| 0xc000a24ce0>: failed to create HCP cluster cluster-zstream-5-0-wlmbkw: failed waiting for cluster="cluster-zstream-5-0-wlmbkw" in resourcegroup="rg-zstream-upgrade-5-0-88f465g4v8qg" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/8399e46e-e9e9-480e-b60a-122ad9e52531 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/8399e46e-e9e9-480e-b60a-122ad9e52531", "name": "8399e46e-e9e9-480e-b60a-122ad9e52531", "status": "Failed", "startTime": "2026-09-18T19:00:37.645326719Z", "endTime": "2026-09-18T19:29:40.288068036Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message cluster placement did not complete before the deadline; provider Microsoft.RedHatOpenShift 17:02:57 UTC · failed at e2e · test Customer should upgrade a nodepool to a version without Cincinnati upgrade edge z-stream upgrade without Cincinnati edge in 4.20 · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:301]: failed to create HCP cluster np-noedge-phwsxm Unexpected error: <fmt.wrapError \| 0xc0010dc9c0>: failed to create HCP cluster np-noedge-phwsxm: failed waiting for cluster="np-noedge-phwsxm" in resourcegroup="rg-np-noedge-phwsxm-4bv8jxxfz58h" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/ac9b5fdb-7eb8-486a-9321-c2c553703d36 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/ac9b5fdb-7eb8-486a-9321-c2c553703d36", "name": "ac9b5fdb-7eb8-486a-9321-c2c553703d36", "status": "Failed", "startTime": "2026-09-18T19:55:33.085250323Z", "endTime": "2026-09-18T20:24:37.028572057Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:301]: failed to create HCP cluster np-noedge-phwsxm Unexpected error: <fmt.wrapError \| 0xc0010dc9c0>: failed to create HCP cluster np-noedge-phwsxm: failed waiting for cluster="np-noedge-phwsxm" in resourcegroup="rg-np-noedge-phwsxm-4bv8jxxfz58h" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/ac9b5fdb-7eb8-486a-9321-c2c553703d36 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/ac9b5fdb-7eb8-486a-9321-c2c553703d36", "name": "ac9b5fdb-7eb8-486a-9321-c2c553703d36", "status": "Failed", "startTime": "2026-09-18T19:55:33.085250323Z", "endTime": "2026-09-18T20:24:37.028572057Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message cluster placement did not complete before the deadline; provider Microsoft.RedHatOpenShift 17:02:57 UTC · failed at e2e · test Customer should be able to create an HCP cluster with Image Registry not present · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/image_registry_cluster_create.go:89]: failed to create HCP cluster with image registry disabled Unexpected error: <fmt.wrapError \| 0xc000b6e260>: failed to create HCP cluster disabled-image-registry-hcp-cluster: failed waiting for cluster="disabled-image-registry-hcp-cluster" in resourcegroup="disabled-image-registry-ld4s6tgxr5vg" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/a000d2ab-54f0-4334-b41d-9e9451140d91 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/a000d2ab-54f0-4334-b41d-9e9451140d91", "name": "a000d2ab-54f0-4334-b41d-9e9451140d91", "status": "Failed", "startTime": "2026-09-18T19:16:16.855336614Z", "endTime": "2026-09-18T19:45:20.354805698Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/image_registry_cluster_create.go:89]: failed to create HCP cluster with image registry disabled Unexpected error: <fmt.wrapError \| 0xc000b6e260>: failed to create HCP cluster disabled-image-registry-hcp-cluster: failed waiting for cluster="disabled-image-registry-hcp-cluster" in resourcegroup="disabled-image-registry-ld4s6tgxr5vg" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/a000d2ab-54f0-4334-b41d-9e9451140d91 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/a000d2ab-54f0-4334-b41d-9e9451140d91", "name": "a000d2ab-54f0-4334-b41d-9e9451140d91", "status": "Failed", "startTime": "2026-09-18T19:16:16.855336614Z", "endTime": "2026-09-18T19:45:20.354805698Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message cluster placement did not complete before the deadline; provider Microsoft.RedHatOpenShift 17:02:57 UTC · failed at e2e · test Customer should downgrade a nodepool version z-stream downgrade from 4.21.zLatest to 4.21.zPrevious · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:580]: failed to create HCP cluster Unexpected error: <fmt.wrapError \| 0xc000c425a0>: failed to create HCP cluster np-downgrade-7d26bl: failed waiting for cluster="np-downgrade-7d26bl" in resourcegroup="rg-np-downgrade-7d26bl-7qmvc9mdqdmp" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/02ee3c59-eff4-48f9-8e78-44b142853e94 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/02ee3c59-eff4-48f9-8e78-44b142853e94", "name": "02ee3c59-eff4-48f9-8e78-44b142853e94", "status": "Failed", "startTime": "2026-09-18T19:55:45.411002197Z", "endTime": "2026-09-18T20:24:47.028298292Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:580]: failed to create HCP cluster Unexpected error: <fmt.wrapError \| 0xc000c425a0>: failed to create HCP cluster np-downgrade-7d26bl: failed waiting for cluster="np-downgrade-7d26bl" in resourcegroup="rg-np-downgrade-7d26bl-7qmvc9mdqdmp" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/02ee3c59-eff4-48f9-8e78-44b142853e94 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/02ee3c59-eff4-48f9-8e78-44b142853e94", "name": "02ee3c59-eff4-48f9-8e78-44b142853e94", "status": "Failed", "startTime": "2026-09-18T19:55:45.411002197Z", "endTime": "2026-09-18T20:24:47.028298292Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message cluster placement did not complete before the deadline; provider Microsoft.RedHatOpenShift 17:02:57 UTC · failed at e2e · test Customer should be able to create several HCP clusters in their customer resource group, but not in the same managed resource group · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:131]: failed to wait for first cluster "basic-hcp-cluster" to complete creation (timeout '30.000000' minutes) Unexpected error: <exported.ResponseError \| 0xc0011ed1a0>: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/d4795f3f-29d6-4f65-a818-0470837029c0 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/d4795f3f-29d6-4f65-a818-0470837029c0", "name": "d4795f3f-29d6-4f65-a818-0470837029c0", "status": "Failed", "startTime": "2026-09-18T18:05:57.828166643Z", "endTime": "2026-09-18T18:35:03.924193327Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:131]: failed to wait for first cluster "basic-hcp-cluster" to complete creation (timeout '30.000000' minutes) Unexpected error: <exported.ResponseError \| 0xc0011ed1a0>: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/d4795f3f-29d6-4f65-a818-0470837029c0 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/d4795f3f-29d6-4f65-a818-0470837029c0", "name": "d4795f3f-29d6-4f65-a818-0470837029c0", "status": "Failed", "startTime": "2026-09-18T18:05:57.828166643Z", "endTime": "2026-09-18T18:35:03.924193327Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message cluster placement did not complete before the deadline; provider Microsoft.RedHatOpenShift 17:02:57 UTC · failed at e2e · test Customer should upgrade and update a nodepool from 4.20.z to 4.20.zLatest · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:146]: failed to create HCP cluster np-version-upgrade-cluster-59kkxj with version 4.20.38 Unexpected error: <fmt.wrapError \| 0xc000f90940>: failed to create HCP cluster np-version-upgrade-cluster-59kkxj: failed waiting for cluster="np-version-upgrade-cluster-59kkxj" in resourcegroup="rg-np-version-upgrade-59kkxj-strjcq5cf28t" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/c6cb01a7-ddc6-483a-83c4-b3f161303a6f -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/c6cb01a7-ddc6-483a-83c4-b3f161303a6f", "name": "c6cb01a7-ddc6-483a-83c4-b3f161303a6f", "status": "Failed", "startTime": "2026-09-18T19:55:19.582015951Z", "endTime": "2026-09-18T20:24:27.027192644Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:146]: failed to create HCP cluster np-version-upgrade-cluster-59kkxj with version 4.20.38 Unexpected error: <fmt.wrapError \| 0xc000f90940>: failed to create HCP cluster np-version-upgrade-cluster-59kkxj: failed waiting for cluster="np-version-upgrade-cluster-59kkxj" in resourcegroup="rg-np-version-upgrade-59kkxj-strjcq5cf28t" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/c6cb01a7-ddc6-483a-83c4-b3f161303a6f -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/c6cb01a7-ddc6-483a-83c4-b3f161303a6f", "name": "c6cb01a7-ddc6-483a-83c4-b3f161303a6f", "status": "Failed", "startTime": "2026-09-18T19:55:19.582015951Z", "endTime": "2026-09-18T20:24:27.027192644Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message cluster placement did not complete before the deadline; provider Microsoft.RedHatOpenShift 17:02:57 UTC · failed at e2e · test Customer should be able to create an HCP cluster and manage pull secrets · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_pullsecret.go:113]: failed to create HCP cluster for pull secret test Unexpected error: <fmt.wrapError \| 0xc0016acc40>: failed to create HCP cluster pullsecret-hcp-cluster: failed waiting for cluster="pullsecret-hcp-cluster" in resourcegroup="pullsecret-test-t8z82lzs48sr" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/ee4e2ffd-758b-447c-95c9-acf961276e55 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/ee4e2ffd-758b-447c-95c9-acf961276e55", "name": "ee4e2ffd-758b-447c-95c9-acf961276e55", "status": "Failed", "startTime": "2026-09-18T18:04:32.431481456Z", "endTime": "2026-09-18T18:33:33.917230374Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_pullsecret.go:113]: failed to create HCP cluster for pull secret test Unexpected error: <fmt.wrapError \| 0xc0016acc40>: failed to create HCP cluster pullsecret-hcp-cluster: failed waiting for cluster="pullsecret-hcp-cluster" in resourcegroup="pullsecret-test-t8z82lzs48sr" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/ee4e2ffd-758b-447c-95c9-acf961276e55 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/ee4e2ffd-758b-447c-95c9-acf961276e55", "name": "ee4e2ffd-758b-447c-95c9-acf961276e55", "status": "Failed", "startTime": "2026-09-18T18:04:32.431481456Z", "endTime": "2026-09-18T18:33:33.917230374Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message cluster placement did not complete before the deadline; provider Microsoft.RedHatOpenShift 17:02:57 UTC · failed at e2e · test Update HCPOpenShiftCluster Positive creates a cluster and updates tags with a PATCH request · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:161]: failed to create HCP cluster for patch-tags test Unexpected error: <fmt.wrapError \| 0xc0015aed60>: failed to create HCP cluster patch-tags-cluster: failed waiting for cluster="patch-tags-cluster" in resourcegroup="patch-tags-vxq45g66n2fb" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/1aa4f569-8351-4ab6-8cbb-fa86fa721f61 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/1aa4f569-8351-4ab6-8cbb-fa86fa721f61", "name": "1aa4f569-8351-4ab6-8cbb-fa86fa721f61", "status": "Failed", "startTime": "2026-09-18T18:59:43.251936528Z", "endTime": "2026-09-18T19:28:50.284453068Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:161]: failed to create HCP cluster for patch-tags test Unexpected error: <fmt.wrapError \| 0xc0015aed60>: failed to create HCP cluster patch-tags-cluster: failed waiting for cluster="patch-tags-cluster" in resourcegroup="patch-tags-vxq45g66n2fb" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/1aa4f569-8351-4ab6-8cbb-fa86fa721f61 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/1aa4f569-8351-4ab6-8cbb-fa86fa721f61", "name": "1aa4f569-8351-4ab6-8cbb-fa86fa721f61", "status": "Failed", "startTime": "2026-09-18T18:59:43.251936528Z", "endTime": "2026-09-18T19:28:50.284453068Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message cluster placement did not complete before the deadline; provider Microsoft.RedHatOpenShift 17:02:57 UTC · failed at e2e · test Customer should upgrade and update a nodepool from 4.21.z to 4.21.zLatest · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:146]: failed to create HCP cluster np-version-upgrade-cluster-gs8fc2 with version 4.21.34 Unexpected error: <fmt.wrapError \| 0xc000bbc920>: failed to create HCP cluster np-version-upgrade-cluster-gs8fc2: failed waiting for cluster="np-version-upgrade-cluster-gs8fc2" in resourcegroup="rg-np-version-upgrade-gs8fc2-flflx6nnvpbv" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/602c1354-35fa-4664-9d86-220f8f644a46 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/602c1354-35fa-4664-9d86-220f8f644a46", "name": "602c1354-35fa-4664-9d86-220f8f644a46", "status": "Failed", "startTime": "2026-09-18T19:55:14.360447552Z", "endTime": "2026-09-18T20:24:17.027918855Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:146]: failed to create HCP cluster np-version-upgrade-cluster-gs8fc2 with version 4.21.34 Unexpected error: <fmt.wrapError \| 0xc000bbc920>: failed to create HCP cluster np-version-upgrade-cluster-gs8fc2: failed waiting for cluster="np-version-upgrade-cluster-gs8fc2" in resourcegroup="rg-np-version-upgrade-gs8fc2-flflx6nnvpbv" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/602c1354-35fa-4664-9d86-220f8f644a46 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/602c1354-35fa-4664-9d86-220f8f644a46", "name": "602c1354-35fa-4664-9d86-220f8f644a46", "status": "Failed", "startTime": "2026-09-18T19:55:14.360447552Z", "endTime": "2026-09-18T20:24:17.027918855Z", "error": { "code": "InternalServerError", "message": "cluster placement did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred
13:27:14	2h 36m 21s	branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel post-goodmerged PR No additional run metadata captured	centralindia	n/a	passed	n/a	0	n/a
13:26:34	2h 28m 10s	branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel post-goodmerged PR No additional run metadata captured	australiaeast	n/a	passed	n/a	0	n/a
13:25:59	2h 31m 20s	branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel post-goodmerged PR No additional run metadata captured	canadacentral	n/a	passed	n/a	0	n/a
10:26:04	2h 28m 36s	branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel post-goodmerged PR No additional run metadata captured	centralindia	e2e	failed	n/a	1	e2e (1) ERROR CODE: SubscriptionNotRegistered; provider Microsoft.Insights 10:26:04 UTC · failed at e2e · test Customer should be able to forward control plane logs to a storage account and Event Hub via shoebox diagnostic settings · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/shoebox_logs.go:319]: failed to create storage account diagnostic setting for cluster /subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/shoebox-cluster-d9mbtgrlrxw2/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/shoebox-hcp-cluster Unexpected error: <exported.ResponseError \| 0xc000a45890>: PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/shoebox-cluster-d9mbtgrlrxw2/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/shoebox-hcp-cluster/providers/Microsoft.Insights/diagnosticSettings/shoebox-storage-diag -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: SubscriptionNotRegistered -------------------------------------------------------------------------------- { "code": "SubscriptionNotRegistered", "message": "The subscription 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX' is not registered to use microsoft.insights." } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/shoebox_logs.go:319]: failed to create storage account diagnostic setting for cluster /subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/shoebox-cluster-d9mbtgrlrxw2/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/shoebox-hcp-cluster Unexpected error: <exported.ResponseError \| 0xc000a45890>: PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/shoebox-cluster-d9mbtgrlrxw2/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/shoebox-hcp-cluster/providers/Microsoft.Insights/diagnosticSettings/shoebox-storage-diag -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: SubscriptionNotRegistered -------------------------------------------------------------------------------- { "code": "SubscriptionNotRegistered", "message": "The subscription 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX' is not registered to use microsoft.insights." } -------------------------------------------------------------------------------- ... occurred
10:20:35	2h 26m 45s	branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel post-goodmerged PR No additional run metadata captured	australiaeast	e2e	failed	n/a	1	e2e (1) ERROR CODE: SubscriptionNotRegistered; provider Microsoft.Insights 10:20:35 UTC · failed at e2e · test Customer should be able to forward control plane logs to a storage account and Event Hub via shoebox diagnostic settings · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/shoebox_logs.go:319]: failed to create storage account diagnostic setting for cluster /subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/shoebox-cluster-986dm5j578hd/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/shoebox-hcp-cluster Unexpected error: <exported.ResponseError \| 0xc00117a2a0>: PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/shoebox-cluster-986dm5j578hd/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/shoebox-hcp-cluster/providers/Microsoft.Insights/diagnosticSettings/shoebox-storage-diag -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: SubscriptionNotRegistered -------------------------------------------------------------------------------- { "code": "SubscriptionNotRegistered", "message": "The subscription 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX' is not registered to use microsoft.insights." } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/shoebox_logs.go:319]: failed to create storage account diagnostic setting for cluster /subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/shoebox-cluster-986dm5j578hd/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/shoebox-hcp-cluster Unexpected error: <exported.ResponseError \| 0xc00117a2a0>: PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/shoebox-cluster-986dm5j578hd/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/shoebox-hcp-cluster/providers/Microsoft.Insights/diagnosticSettings/shoebox-storage-diag -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: SubscriptionNotRegistered -------------------------------------------------------------------------------- { "code": "SubscriptionNotRegistered", "message": "The subscription 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX' is not registered to use microsoft.insights." } -------------------------------------------------------------------------------- ... occurred
10:14:55	2h 35m 22s	branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel post-goodmerged PR No additional run metadata captured	switzerlandnorth	n/a	passed	n/a	0	n/a
10:10:35	2h 25m 49s	branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel post-goodmerged PR No additional run metadata captured	canadacentral	e2e	failed	n/a	1	e2e (1) ERROR CODE: SubscriptionNotRegistered; provider Microsoft.Insights 10:10:35 UTC · failed at e2e · test Customer should be able to forward control plane logs to a storage account and Event Hub via shoebox diagnostic settings · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/shoebox_logs.go:319]: failed to create storage account diagnostic setting for cluster /subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/shoebox-cluster-92rgrk84fv8d/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/shoebox-hcp-cluster Unexpected error: <exported.ResponseError \| 0xc00088e570>: PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/shoebox-cluster-92rgrk84fv8d/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/shoebox-hcp-cluster/providers/Microsoft.Insights/diagnosticSettings/shoebox-storage-diag -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: SubscriptionNotRegistered -------------------------------------------------------------------------------- { "code": "SubscriptionNotRegistered", "message": "The subscription 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX' is not registered to use microsoft.insights." } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/shoebox_logs.go:319]: failed to create storage account diagnostic setting for cluster /subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/shoebox-cluster-92rgrk84fv8d/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/shoebox-hcp-cluster Unexpected error: <exported.ResponseError \| 0xc00088e570>: PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/shoebox-cluster-92rgrk84fv8d/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/shoebox-hcp-cluster/providers/Microsoft.Insights/diagnosticSettings/shoebox-storage-diag -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: SubscriptionNotRegistered -------------------------------------------------------------------------------- { "code": "SubscriptionNotRegistered", "message": "The subscription 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX' is not registered to use microsoft.insights." } -------------------------------------------------------------------------------- ... occurred
05:58:50	2h 29m 43s	branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel post-goodmerged PR No additional run metadata captured	uksouth	e2e	failed	n/a	1	e2e (1) ERROR CODE: InternalServerError; detail message node pool creation did not complete before the deadline; provider Microsoft.RedHatOpenShift 05:58:50 UTC · failed at e2e · test Customer should be able to use workload identity via the cluster OIDC issuer URL · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/oidc_issuer_workload_identity.go:270]: failed to create node pool oidc-wi-np Unexpected error: <fmt.wrapError \| 0xc002d105c0>: failed to create NodePool oidc-wi-np: failed waiting for nodepool="oidc-wi-np" for cluster "oidc-wi-cluster" in resourcegroup="oidc-wi-wbftvl6pxn6v" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/9f5be950-3d76-400d-bba8-80953e9b85fb -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/9f5be950-3d76-400d-bba8-80953e9b85fb", "name": "9f5be950-3d76-400d-bba8-80953e9b85fb", "status": "Failed", "startTime": "2026-09-18T07:41:11.725184381Z", "endTime": "2026-09-18T08:00:14.656645268Z", "error": { "code": "InternalServerError", "message": "node pool creation did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/oidc_issuer_workload_identity.go:270]: failed to create node pool oidc-wi-np Unexpected error: <fmt.wrapError \| 0xc002d105c0>: failed to create NodePool oidc-wi-np: failed waiting for nodepool="oidc-wi-np" for cluster "oidc-wi-cluster" in resourcegroup="oidc-wi-wbftvl6pxn6v" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/9f5be950-3d76-400d-bba8-80953e9b85fb -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/9f5be950-3d76-400d-bba8-80953e9b85fb", "name": "9f5be950-3d76-400d-bba8-80953e9b85fb", "status": "Failed", "startTime": "2026-09-18T07:41:11.725184381Z", "endTime": "2026-09-18T08:00:14.656645268Z", "error": { "code": "InternalServerError", "message": "node pool creation did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred
02:00:17	2h 18m 49s	periodic-ci-Azure-ARO-HCP-main-periodic-prod-e2e-parallel post-goodmerged PR No additional run metadata captured	uksouth	e2e	failed	n/a	4	e2e (4) ERROR CODE: InternalServerError; detail message node pool creation did not complete before the deadline; provider Microsoft.RedHatOpenShift 02:00:17 UTC · failed at e2e · test Customer should be able to create a HCP cluster and use cilium CNI plugin · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_cni_cilium.go:182]: failed to verify cilium is running and nodes are Ready for cluster "cilium-cl" Unexpected error: <errors.joinError \| 0xc001092648>: failed to create NodePool cilium-np: failed waiting for nodepool="cilium-np" for cluster "cilium-cl" in resourcegroup="cni-cilium-tgqlbhrjcjzd" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/f4e2d962-cb10-4c5d-b5e9-88c9efab7468 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/f4e2d962-cb10-4c5d-b5e9-88c9efab7468", "name": "f4e2d962-cb10-4c5d-b5e9-88c9efab7468", "status": "Failed", "startTime": "2026-09-18T02:15:45.572313063Z", "endTime": "2026-09-18T02:34:50.003063171Z", "error": { "code": "InternalServerError", "message": "node pool creation did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_cni_cilium.go:182]: failed to verify cilium is running and nodes are Ready for cluster "cilium-cl" Unexpected error: <errors.joinError \| 0xc001092648>: failed to create NodePool cilium-np: failed waiting for nodepool="cilium-np" for cluster "cilium-cl" in resourcegroup="cni-cilium-tgqlbhrjcjzd" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/f4e2d962-cb10-4c5d-b5e9-88c9efab7468 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/f4e2d962-cb10-4c5d-b5e9-88c9efab7468", "name": "f4e2d962-cb10-4c5d-b5e9-88c9efab7468", "status": "Failed", "startTime": "2026-09-18T02:15:45.572313063Z", "endTime": "2026-09-18T02:34:50.003063171Z", "error": { "code": "InternalServerError", "message": "node pool creation did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message node pool creation did not complete before the deadline; provider Microsoft.RedHatOpenShift 02:00:17 UTC · failed at e2e · test Customer should upgrade and update a nodepool from 4.21.z to 4.21.zLatest · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:165]: failed to create node pool npupgrade-4-21 with version 4.21.32 Unexpected error: <fmt.wrapError \| 0xc000efdee0>: failed to create NodePool npupgrade-4-21: failed waiting for nodepool="npupgrade-4-21" for cluster "np-version-upgrade-cluster-dd2djv" in resourcegroup="rg-np-version-upgrade-dd2djv-f5pt9zl6csx7" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/34fcd870-f035-4d43-a07e-056005a0cd12 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/34fcd870-f035-4d43-a07e-056005a0cd12", "name": "34fcd870-f035-4d43-a07e-056005a0cd12", "status": "Failed", "startTime": "2026-09-18T03:23:36.136458591Z", "endTime": "2026-09-18T03:42:38.777220989Z", "error": { "code": "InternalServerError", "message": "node pool creation did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:165]: failed to create node pool npupgrade-4-21 with version 4.21.32 Unexpected error: <fmt.wrapError \| 0xc000efdee0>: failed to create NodePool npupgrade-4-21: failed waiting for nodepool="npupgrade-4-21" for cluster "np-version-upgrade-cluster-dd2djv" in resourcegroup="rg-np-version-upgrade-dd2djv-f5pt9zl6csx7" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/34fcd870-f035-4d43-a07e-056005a0cd12 -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/34fcd870-f035-4d43-a07e-056005a0cd12", "name": "34fcd870-f035-4d43-a07e-056005a0cd12", "status": "Failed", "startTime": "2026-09-18T03:23:36.136458591Z", "endTime": "2026-09-18T03:42:38.777220989Z", "error": { "code": "InternalServerError", "message": "node pool creation did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: SubscriptionNotRegistered; provider Microsoft.Insights 02:00:17 UTC · failed at e2e · test Customer should be able to forward control plane logs to a storage account and Event Hub via shoebox diagnostic settings · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/shoebox_logs.go:319]: failed to create storage account diagnostic setting for cluster /subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/shoebox-cluster-sxsc7bxhvj2n/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/shoebox-hcp-cluster Unexpected error: <exported.ResponseError \| 0xc0012e0990>: PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/shoebox-cluster-sxsc7bxhvj2n/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/shoebox-hcp-cluster/providers/Microsoft.Insights/diagnosticSettings/shoebox-storage-diag -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: SubscriptionNotRegistered -------------------------------------------------------------------------------- { "code": "SubscriptionNotRegistered", "message": "The subscription 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX' is not registered to use microsoft.insights." } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/shoebox_logs.go:319]: failed to create storage account diagnostic setting for cluster /subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/shoebox-cluster-sxsc7bxhvj2n/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/shoebox-hcp-cluster Unexpected error: <exported.ResponseError \| 0xc0012e0990>: PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/shoebox-cluster-sxsc7bxhvj2n/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/shoebox-hcp-cluster/providers/Microsoft.Insights/diagnosticSettings/shoebox-storage-diag -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: SubscriptionNotRegistered -------------------------------------------------------------------------------- { "code": "SubscriptionNotRegistered", "message": "The subscription 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX' is not registered to use microsoft.insights." } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message node pool creation did not complete before the deadline; provider Microsoft.RedHatOpenShift 02:00:17 UTC · failed at e2e · test Customer should downgrade a nodepool to a lower minor version from 4.22.zLatest to 4.20.zLatest · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:738]: failed to create nodepool npdg-422-420 Unexpected error: <fmt.wrapError \| 0xc000d0cd20>: failed to create NodePool npdg-422-420: failed waiting for nodepool="npdg-422-420" for cluster "np-dg-minor-hg2tcz" in resourcegroup="rg-np-dg-minor-hg2tcz-n7pnlmn2w4pv" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/bd7e5bf4-841e-4289-9a4b-a34d164ba80f -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/bd7e5bf4-841e-4289-9a4b-a34d164ba80f", "name": "bd7e5bf4-841e-4289-9a4b-a34d164ba80f", "status": "Failed", "startTime": "2026-09-18T03:32:13.01856929Z", "endTime": "2026-09-18T03:51:16.292258995Z", "error": { "code": "InternalServerError", "message": "node pool creation did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:738]: failed to create nodepool npdg-422-420 Unexpected error: <fmt.wrapError \| 0xc000d0cd20>: failed to create NodePool npdg-422-420: failed waiting for nodepool="npdg-422-420" for cluster "np-dg-minor-hg2tcz" in resourcegroup="rg-np-dg-minor-hg2tcz-n7pnlmn2w4pv" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/bd7e5bf4-841e-4289-9a4b-a34d164ba80f -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/bd7e5bf4-841e-4289-9a4b-a34d164ba80f", "name": "bd7e5bf4-841e-4289-9a4b-a34d164ba80f", "status": "Failed", "startTime": "2026-09-18T03:32:13.01856929Z", "endTime": "2026-09-18T03:51:16.292258995Z", "error": { "code": "InternalServerError", "message": "node pool creation did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred
01:44:31	2h 40m 51s	branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel post-goodmerged PR No additional run metadata captured	uksouth	e2e	failed	n/a	3	e2e (3) ERROR CODE: InternalServerError; detail message [hypershiftHostedCluster] hosted cluster is not available: KubeconfigWaitingForCreate: Waiting for hosted control plane kubeconfig to be created; hosted cluster degraded: UnavailableReplicas: [capi-provider]; provider Microsoft.RedHatOpenShift 01:44:31 UTC · failed at e2e · test Customer should be able to create an HCP cluster with custom autoscaling and update the autoscaling configuration · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:108]: failed to create HCP cluster "autoscaling-hcp-cluster" with custom autoscaling Unexpected error: <fmt.wrapError \| 0xc000860220>: failed to create HCP cluster autoscaling-hcp-cluster: failed waiting for cluster="autoscaling-hcp-cluster" in resourcegroup="autoscaling-cluster-rm9vjp5gmczk" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/df01aa2b-a5c5-4899-b645-5ce58f5f8cee -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/df01aa2b-a5c5-4899-b645-5ce58f5f8cee", "name": "df01aa2b-a5c5-4899-b645-5ce58f5f8cee", "status": "Failed", "startTime": "2026-09-18T02:08:07.344249043Z", "endTime": "2026-09-18T02:37:10.01628355Z", "error": { "code": "InternalServerError", "message": "[hypershiftHostedCluster] hosted cluster is not available: KubeconfigWaitingForCreate: Waiting for hosted control plane kubeconfig to be created; hosted cluster degraded: UnavailableReplicas: capi-provider deployment has 1 unavailable replicas" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:108]: failed to create HCP cluster "autoscaling-hcp-cluster" with custom autoscaling Unexpected error: <fmt.wrapError \| 0xc000860220>: failed to create HCP cluster autoscaling-hcp-cluster: failed waiting for cluster="autoscaling-hcp-cluster" in resourcegroup="autoscaling-cluster-rm9vjp5gmczk" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/df01aa2b-a5c5-4899-b645-5ce58f5f8cee -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/df01aa2b-a5c5-4899-b645-5ce58f5f8cee", "name": "df01aa2b-a5c5-4899-b645-5ce58f5f8cee", "status": "Failed", "startTime": "2026-09-18T02:08:07.344249043Z", "endTime": "2026-09-18T02:37:10.01628355Z", "error": { "code": "InternalServerError", "message": "[hypershiftHostedCluster] hosted cluster is not available: KubeconfigWaitingForCreate: Waiting for hosted control plane kubeconfig to be created; hosted cluster degraded: UnavailableReplicas: capi-provider deployment has 1 unavailable replicas" } } -------------------------------------------------------------------------------- ... occurred ERROR CODE: SubscriptionNotRegistered; provider Microsoft.Insights 01:44:31 UTC · failed at e2e · test Customer should be able to forward control plane logs to a storage account and Event Hub via shoebox diagnostic settings · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/shoebox_logs.go:319]: failed to create storage account diagnostic setting for cluster /subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/shoebox-cluster-cxcwb4twxbz4/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/shoebox-hcp-cluster Unexpected error: <exported.ResponseError \| 0xc001176ff0>: PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/shoebox-cluster-cxcwb4twxbz4/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/shoebox-hcp-cluster/providers/Microsoft.Insights/diagnosticSettings/shoebox-storage-diag -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: SubscriptionNotRegistered -------------------------------------------------------------------------------- { "code": "SubscriptionNotRegistered", "message": "The subscription 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX' is not registered to use microsoft.insights." } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/shoebox_logs.go:319]: failed to create storage account diagnostic setting for cluster /subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/shoebox-cluster-cxcwb4twxbz4/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/shoebox-hcp-cluster Unexpected error: <exported.ResponseError \| 0xc001176ff0>: PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/shoebox-cluster-cxcwb4twxbz4/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/shoebox-hcp-cluster/providers/Microsoft.Insights/diagnosticSettings/shoebox-storage-diag -------------------------------------------------------------------------------- RESPONSE 400: 400 Bad Request ERROR CODE: SubscriptionNotRegistered -------------------------------------------------------------------------------- { "code": "SubscriptionNotRegistered", "message": "The subscription 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX' is not registered to use microsoft.insights." } -------------------------------------------------------------------------------- ... occurred ERROR CODE: InternalServerError; detail message node pool creation did not complete before the deadline; provider Microsoft.RedHatOpenShift 01:44:31 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.20 · suite prod/parallel Show raw failure fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:175]: failed to create node pool "np-pd9r72" for cluster "cluster-candidate-4-20-pd9r72" Unexpected error: <fmt.wrapError \| 0xc001005b20>: failed to create NodePool np-pd9r72: failed waiting for nodepool="np-pd9r72" for cluster "cluster-candidate-4-20-pd9r72" in resourcegroup="rg-candidate-4-20-7nvxhnmpb6qr" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/e62533b3-e87f-4306-b7fa-7e8f2f79508a -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/e62533b3-e87f-4306-b7fa-7e8f2f79508a", "name": "e62533b3-e87f-4306-b7fa-7e8f2f79508a", "status": "Failed", "startTime": "2026-09-18T02:55:35.193527856Z", "endTime": "2026-09-18T03:14:42.13127378Z", "error": { "code": "InternalServerError", "message": "node pool creation did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:175]: failed to create node pool "np-pd9r72" for cluster "cluster-candidate-4-20-pd9r72" Unexpected error: <fmt.wrapError \| 0xc001005b20>: failed to create NodePool np-pd9r72: failed waiting for nodepool="np-pd9r72" for cluster "cluster-candidate-4-20-pd9r72" in resourcegroup="rg-candidate-4-20-7nvxhnmpb6qr" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/e62533b3-e87f-4306-b7fa-7e8f2f79508a -------------------------------------------------------------------------------- RESPONSE 200: 200 OK ERROR CODE: InternalServerError -------------------------------------------------------------------------------- { "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/e62533b3-e87f-4306-b7fa-7e8f2f79508a", "name": "e62533b3-e87f-4306-b7fa-7e8f2f79508a", "status": "Failed", "startTime": "2026-09-18T02:55:35.193527856Z", "endTime": "2026-09-18T03:14:42.13127378Z", "error": { "code": "InternalServerError", "message": "node pool creation did not complete before the deadline" } } -------------------------------------------------------------------------------- ... occurred