No runs match your search.
Environment: PROD
| Time (UTC) | Runtime | Job | Region | Failed at | Result | PR | Failed tests | Details |
|---|---|---|---|---|---|---|---|---|
| 2h 47m 39s | branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel post-goodmerged PR |
brazilsouth | n/a | passed | n/a | 0 | n/a | |
| 2h 40m 51s | branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel post-goodmerged PR |
westeurope | e2e | failed | n/a | 2 | e2e (2)ERROR CODE: DeadlineExceeded; provider Microsoft.Compute Show raw failurefail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-jcbqlv" for cluster "cluster-candidate-4-20-jcbqlv"
Unexpected error:
<*fmt.wrapError | 0xc002b94f00>:
failed to create NodePool np-jcbqlv: failed waiting for nodepool="np-jcbqlv" for cluster "cluster-candidate-4-20-jcbqlv" in resourcegroup="rg-candidate-4-20-lvv7b2lssbpl" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/84cb018b-4347-4f56-b1c7-88d7604492f9
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: DeadlineExceeded
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/84cb018b-4347-4f56-b1c7-88d7604492f9",
"name": "84cb018b-4347-4f56-b1c7-88d7604492f9",
"status": "Failed",
"startTime": "2026-10-03T00:32:09.437716177Z",
"endTime": "2026-10-03T00:51:11.761764992Z",
"error": {
"code": "DeadlineExceeded",
"message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 1, want 2; node pool condition AllNodesHealthy is False: 1 of 2 machines are not healthy\nMachine c1t3y9h8f3v9w7t-np-jcbqlv-zlpbm-ww6xn: InspectionFailed: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-4-20-lvv7b2lssbpl-candidate-jcbqlv--managed/providers/Microsoft.Compute/virtualMachines/c1t3y9h8f3v9w7t-np-jcbqlv-zlpbm-ww6xn to exist\n; node pool condition AllMachinesReady is False: 1 of 2 machines are not ready\nMachine c1t3y9h8f3v9w7t-np-jcbqlv-zlpbm-ww6xn: ReadyUnknown: * NodeHealthy: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-4-20-lvv7b2lssbpl-candidate-jcbqlv--managed/providers/Microsoft.Compute/virtualMachines/c1t3y9h8f3v9w7t-np-jcbqlv-zlpbm-ww6xn to exist\n"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-jcbqlv" for cluster "cluster-candidate-4-20-jcbqlv"
Unexpected error:
<*fmt.wrapError | 0xc002b94f00>:
failed to create NodePool np-jcbqlv: failed waiting for nodepool="np-jcbqlv" for cluster "cluster-candidate-4-20-jcbqlv" in resourcegroup="rg-candidate-4-20-lvv7b2lssbpl" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/84cb018b-4347-4f56-b1c7-88d7604492f9
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: DeadlineExceeded
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/84cb018b-4347-4f56-b1c7-88d7604492f9",
"name": "84cb018b-4347-4f56-b1c7-88d7604492f9",
"status": "Failed",
"startTime": "2026-10-03T00:32:09.437716177Z",
"endTime": "2026-10-03T00:51:11.761764992Z",
"error": {
"code": "DeadlineExceeded",
"message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 1, want 2; node pool condition AllNodesHealthy is False: 1 of 2 machines are not healthy\nMachine c1t3y9h8f3v9w7t-np-jcbqlv-zlpbm-ww6xn: InspectionFailed: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-4-20-lvv7b2lssbpl-candidate-jcbqlv--managed/providers/Microsoft.Compute/virtualMachines/c1t3y9h8f3v9w7t-np-jcbqlv-zlpbm-ww6xn to exist\n; node pool condition AllMachinesReady is False: 1 of 2 machines are not ready\nMachine c1t3y9h8f3v9w7t-np-jcbqlv-zlpbm-ww6xn: ReadyUnknown: * NodeHealthy: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-4-20-lvv7b2lssbpl-candidate-jcbqlv--managed/providers/Microsoft.Compute/virtualMachines/c1t3y9h8f3v9w7t-np-jcbqlv-zlpbm-ww6xn to exist\n"
}
}
--------------------------------------------------------------------------------
...
occurredfailed to create nodepool <nodepool>: failed waiting for nodepool="<nodepool>" for cluster "<cluster>" in resourcegroup="<resource-group>" to finish creating: GET <url> Show raw failurefail [github.com/Azure/ARO-HCP/test/e2e/cluster_ingress_cert_letsencrypt.go:138]: failed to create node pool
Expected success, but got an error:
<*fmt.wrapError | 0xc00113e680>:
failed to create NodePool np-1: failed waiting for nodepool="np-1" for cluster "ingress-cert-le" in resourcegroup="ingress-cert-le-9tnt4zgp45ww" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/84e6a04c-39e2-4627-85f5-80cd1669879f
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: DeadlineExceeded
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/84e6a04c-39e2-4627-85f5-80cd1669879f",
"name": "84e6a04c-39e2-4627-85f5-80cd1669879f",
"status": "Failed",
"startTime": "2026-10-02T23:58:26.609352403Z",
"endTime": "2026-10-03T00:17:31.644821105Z",
"error": {
"code": "DeadlineExceeded",
"message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 1, want 2; node pool condition AllNodesHealthy is False: 1 of 2 machines are not healthy\nMachine ingress-cert-le-np-1-mmbgh-jld78: InspectionFailed: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/ingress-cert-le-9tnt4zgp45ww--managed/providers/Microsoft.Compute/virtualMachines/ingress-cert-le-np-1-mmbgh-jld78 to exist\n; node pool condition AllMachinesReady is False: 1 of 2 machines are not ready\nMachine ingress-cert-le-np-1-mmbgh-jld78: ReadyUnknown: * NodeHealthy: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/ingress-cert-le-9tnt4zgp45ww--managed/providers/Microsoft.Compute/virtualMachines/ingress-cert-le-np-1-mmbgh-jld78 to exist\n"
}
}
--------------------------------------------------------------------------------
...
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_ingress_cert_letsencrypt.go:138]: failed to create node pool
Expected success, but got an error:
<*fmt.wrapError | 0xc00113e680>:
failed to create NodePool np-1: failed waiting for nodepool="np-1" for cluster "ingress-cert-le" in resourcegroup="ingress-cert-le-9tnt4zgp45ww" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/84e6a04c-39e2-4627-85f5-80cd1669879f
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: DeadlineExceeded
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/84e6a04c-39e2-4627-85f5-80cd1669879f",
"name": "84e6a04c-39e2-4627-85f5-80cd1669879f",
"status": "Failed",
"startTime": "2026-10-02T23:58:26.609352403Z",
"endTime": "2026-10-03T00:17:31.644821105Z",
"error": {
"code": "DeadlineExceeded",
"message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 1, want 2; node pool condition AllNodesHealthy is False: 1 of 2 machines are not healthy\nMachine ingress-cert-le-np-1-mmbgh-jld78: InspectionFailed: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/ingress-cert-le-9tnt4zgp45ww--managed/providers/Microsoft.Compute/virtualMachines/ingress-cert-le-np-1-mmbgh-jld78 to exist\n; node pool condition AllMachinesReady is False: 1 of 2 machines are not ready\nMachine ingress-cert-le-np-1-mmbgh-jld78: ReadyUnknown: * NodeHealthy: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/ingress-cert-le-9tnt4zgp45ww--managed/providers/Microsoft.Compute/virtualMachines/ingress-cert-le-np-1-mmbgh-jld78 to exist\n"
}
}
--------------------------------------------------------------------------------
... |
|
| 2h 25m 1s | branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel post-goodmerged PR |
centralindia | e2e | failed | n/a | 1 | e2e (1)client rate limiter Wait returned an error: context deadline exceeded Show raw failurefail [github.com/Azure/ARO-HCP/test/e2e/cluster_pullsecret.go:286]: failed to wait for image from registry.redhat.io to be pulled successfully with the added pull secret
Unexpected error:
<*fmt.wrapError | 0xc000a814a0>:
VerifyImagePulled(registry.redhat.io/ubi-minimal) timed out after 3m0.001s: failed to list pods in namespace pullsecret-image-test: client rate limiter Wait returned an error: context deadline exceeded
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_pullsecret.go:286]: failed to wait for image from registry.redhat.io to be pulled successfully with the added pull secret
Unexpected error:
<*fmt.wrapError | 0xc000a814a0>:
VerifyImagePulled(registry.redhat.io/ubi-minimal) timed out after 3m0.001s: failed to list pods in namespace pullsecret-image-test: client rate limiter Wait returned an error: context deadline exceeded
...
occurred |
|
| 2h 0m 39s | periodic-ci-Azure-ARO-HCP-main-periodic-prod-e2e-parallel-ocp-nightly post-goodmerged PR |
canadacentral | e2e | failed | n/a | 18 | e2e (18)ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-update-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift Show raw failurefail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:167]: failed to trigger y-stream upgrade of cluster "cp-ystream-upgrade-4-21-kf8zjf" to 4.21.0-0.nightly-multi-2026-10-02-021649
Unexpected error:
<*exported.ResponseError | 0xc000d32ae0>:
PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cp-ystream-upgrade-4-21-9wcjt7cq5sl5/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cp-ystream-upgrade-4-21-kf8zjf
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"aro-hcp.experimental.cluster.max-update-duration\": unrecognized experimental tag",
"target": "tags[aro-hcp.experimental.cluster.max-update-duration]"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:167]: failed to trigger y-stream upgrade of cluster "cp-ystream-upgrade-4-21-kf8zjf" to 4.21.0-0.nightly-multi-2026-10-02-021649
Unexpected error:
<*exported.ResponseError | 0xc000d32ae0>:
PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cp-ystream-upgrade-4-21-9wcjt7cq5sl5/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cp-ystream-upgrade-4-21-kf8zjf
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"aro-hcp.experimental.cluster.max-update-duration\": unrecognized experimental tag",
"target": "tags[aro-hcp.experimental.cluster.max-update-duration]"
}
}
--------------------------------------------------------------------------------
...
occurredERROR CODE: InvalidRequestContent; detail message Invalid value: "4.21": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is "nightly"; provider Microsoft.RedHatOpenShift Show raw failurefail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:275]: failed to begin cluster "agg-cluster" authorized CIDR update
Unexpected error:
<*exported.ResponseError | 0xc000cdbad0>:
PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/feature-aggregation-hbvfkpmwjx97/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/agg-cluster
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"4.21\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"",
"target": "properties.version.id"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:275]: failed to begin cluster "agg-cluster" authorized CIDR update
Unexpected error:
<*exported.ResponseError | 0xc000cdbad0>:
PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/feature-aggregation-hbvfkpmwjx97/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/agg-cluster
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"4.21\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"",
"target": "properties.version.id"
}
}
--------------------------------------------------------------------------------
...
occurredERROR CODE: InvalidRequestContent; detail message Invalid value: "<ocp-nightly>": must be at least 4.20.8; provider Microsoft.RedHatOpenShift Show raw failurefail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-s49xkm" for cluster "cluster-nightly-4-20-s49xkm"
Unexpected error:
<*fmt.wrapError | 0xc000ad8640>:
failed to create NodePool np-s49xkm: failed starting nodepool creation "np-s49xkm" for cluster "cluster-nightly-4-20-s49xkm" in resourcegroup="rg-nightly-4-20-mhnxwl8ndm9g": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-nightly-4-20-mhnxwl8ndm9g/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-nightly-4-20-s49xkm/nodePools/np-s49xkm
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"4.20.0-0.nightly-multi-2026-10-01-125400\": must be at least 4.20.8",
"target": "properties.version.id"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-s49xkm" for cluster "cluster-nightly-4-20-s49xkm"
Unexpected error:
<*fmt.wrapError | 0xc000ad8640>:
failed to create NodePool np-s49xkm: failed starting nodepool creation "np-s49xkm" for cluster "cluster-nightly-4-20-s49xkm" in resourcegroup="rg-nightly-4-20-mhnxwl8ndm9g": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-nightly-4-20-mhnxwl8ndm9g/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-nightly-4-20-s49xkm/nodePools/np-s49xkm
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"4.20.0-0.nightly-multi-2026-10-01-125400\": must be at least 4.20.8",
"target": "properties.version.id"
}
}
--------------------------------------------------------------------------------
...
occurredERROR CODE: InvalidRequestContent; detail message Invalid value: "4.21": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is "nightly"; provider Microsoft.RedHatOpenShift Show raw failurefail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:408]: failed to begin updating cluster "cidr-connectivity-test" authorized CIDRs
Unexpected error:
<*exported.ResponseError | 0xc0023c62d0>:
PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-w5hzr8g77c9w/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cidr-connectivity-test
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"4.21\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"",
"target": "properties.version.id"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:408]: failed to begin updating cluster "cidr-connectivity-test" authorized CIDRs
Unexpected error:
<*exported.ResponseError | 0xc0023c62d0>:
PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-w5hzr8g77c9w/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cidr-connectivity-test
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"4.21\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"",
"target": "properties.version.id"
}
}
--------------------------------------------------------------------------------
...
occurredERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.nodepool.max-update-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift Show raw failurefail [github.com/Azure/ARO-HCP/test/e2e/nodepool_labels_taints.go:186]: failed to update node pool np-1 with new taints and scale up
Unexpected error:
<*fmt.wrapError | 0xc0012cc100>:
failed to start nodepool "np-1" update in cluster "cluster-np-labels-taints" resourcegroup="rg-np-labels-taints-qmt425cc9vkv": PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-labels-taints-qmt425cc9vkv/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-np-labels-taints/nodePools/np-1
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"aro-hcp.experimental.nodepool.max-update-duration\": unrecognized experimental tag",
"target": "tags[aro-hcp.experimental.nodepool.max-update-duration]"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_labels_taints.go:186]: failed to update node pool np-1 with new taints and scale up
Unexpected error:
<*fmt.wrapError | 0xc0012cc100>:
failed to start nodepool "np-1" update in cluster "cluster-np-labels-taints" resourcegroup="rg-np-labels-taints-qmt425cc9vkv": PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-labels-taints-qmt425cc9vkv/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-np-labels-taints/nodePools/np-1
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"aro-hcp.experimental.nodepool.max-update-duration\": unrecognized experimental tag",
"target": "tags[aro-hcp.experimental.nodepool.max-update-duration]"
}
}
--------------------------------------------------------------------------------
...
occurredERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.nodepool.max-update-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift Show raw failurefail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:189]: failed to update AZ nodepool autoscale-az max replicas to 4
Unexpected error:
<*fmt.wrapError | 0xc0006110a0>:
failed to start nodepool "autoscale-az" update in cluster "np-autoscale-cluster" resourcegroup="np-autoscaling-bh8wcncrztgb": PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/np-autoscaling-bh8wcncrztgb/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-autoscale-cluster/nodePools/autoscale-az
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"aro-hcp.experimental.nodepool.max-update-duration\": unrecognized experimental tag",
"target": "tags[aro-hcp.experimental.nodepool.max-update-duration]"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:189]: failed to update AZ nodepool autoscale-az max replicas to 4
Unexpected error:
<*fmt.wrapError | 0xc0006110a0>:
failed to start nodepool "autoscale-az" update in cluster "np-autoscale-cluster" resourcegroup="np-autoscaling-bh8wcncrztgb": PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/np-autoscaling-bh8wcncrztgb/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-autoscale-cluster/nodePools/autoscale-az
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"aro-hcp.experimental.nodepool.max-update-duration\": unrecognized experimental tag",
"target": "tags[aro-hcp.experimental.nodepool.max-update-duration]"
}
}
--------------------------------------------------------------------------------
...
occurredERROR CODE: DeadlineExceeded; detail message cluster creation did not complete before the deadline; [cosmosCluster] .api.url is empty; [servingCABundle] ServingCABundle not yet populated; [clusterServiceClusterStatus] cluster service has not been successfully created; [hypershiftHostedCluster] hosted cluster state not cached yet; provider Microsoft.RedHatOpenShift Show raw failurefail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:443]: failed to create HCP cluster np-skip-minor-pwn8nb at version 4.22.0-0.nightly-multi-2026-07-08-192436
Unexpected error:
<*fmt.wrapError | 0xc000692a40>:
failed to create HCP cluster np-skip-minor-pwn8nb: failed waiting for cluster="np-skip-minor-pwn8nb" in resourcegroup="rg-np-skip-minor-pwn8nb-fcqmqdxvjmqx" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/canadacentral/hcpOperationStatuses/9282a0a5-6015-4ac4-8987-2148df07fa63
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: DeadlineExceeded
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/canadacentral/hcpOperationStatuses/9282a0a5-6015-4ac4-8987-2148df07fa63",
"name": "9282a0a5-6015-4ac4-8987-2148df07fa63",
"status": "Failed",
"startTime": "2026-10-03T00:07:14.845401374Z",
"endTime": "2026-10-03T00:36:15.664777437Z",
"error": {
"code": "DeadlineExceeded",
"message": "cluster creation did not complete before the deadline; [cosmosCluster] .api.url is empty; [servingCABundle] ServingCABundle not yet populated; [clusterServiceClusterStatus] cluster service has not been successfully created; [hypershiftHostedCluster] hosted cluster state not cached yet"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:443]: failed to create HCP cluster np-skip-minor-pwn8nb at version 4.22.0-0.nightly-multi-2026-07-08-192436
Unexpected error:
<*fmt.wrapError | 0xc000692a40>:
failed to create HCP cluster np-skip-minor-pwn8nb: failed waiting for cluster="np-skip-minor-pwn8nb" in resourcegroup="rg-np-skip-minor-pwn8nb-fcqmqdxvjmqx" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/canadacentral/hcpOperationStatuses/9282a0a5-6015-4ac4-8987-2148df07fa63
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: DeadlineExceeded
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/canadacentral/hcpOperationStatuses/9282a0a5-6015-4ac4-8987-2148df07fa63",
"name": "9282a0a5-6015-4ac4-8987-2148df07fa63",
"status": "Failed",
"startTime": "2026-10-03T00:07:14.845401374Z",
"endTime": "2026-10-03T00:36:15.664777437Z",
"error": {
"code": "DeadlineExceeded",
"message": "cluster creation did not complete before the deadline; [cosmosCluster] .api.url is empty; [servingCABundle] ServingCABundle not yet populated; [clusterServiceClusterStatus] cluster service has not been successfully created; [hypershiftHostedCluster] hosted cluster state not cached yet"
}
}
--------------------------------------------------------------------------------
...
occurredVerifyAllAPIServicesAvailable failed: failed to list all APIServices: Unauthorized Show raw failurefail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:135]: failed to verify HCP cluster "cluster-nightly-4-21-8rfqwl" is viable
Unexpected error:
<*errors.joinError | 0xc00127afc0>:
VerifyBasicAccess failed: failed to list services: Unauthorized
VerifyAllAPIServicesAvailable failed: failed to list all APIServices: Unauthorized
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:135]: failed to verify HCP cluster "cluster-nightly-4-21-8rfqwl" is viable
Unexpected error:
<*errors.joinError | 0xc00127afc0>:
VerifyBasicAccess failed: failed to list services: Unauthorized
VerifyAllAPIServicesAvailable failed: failed to list all APIServices: Unauthorized
...
occurredERROR CODE: DeadlineExceeded; detail message cluster creation did not complete before the deadline; [cosmosCluster] .api.url is empty; [servingCABundle] ServingCABundle not yet populated; [clusterServiceClusterStatus] cluster service has not been successfully created; [hypershiftHostedCluster] hosted cluster state not cached yet; provider Microsoft.RedHatOpenShift Show raw failurefail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:133]: failed to create HCP cluster "cp-ystream-upgrade-5-0-fts22d" at install version 4.22.0-0.nightly-multi-2026-07-08-192436
Unexpected error:
<*fmt.wrapError | 0xc00111ac60>:
failed to create HCP cluster cp-ystream-upgrade-5-0-fts22d: failed waiting for cluster="cp-ystream-upgrade-5-0-fts22d" in resourcegroup="rg-cp-ystream-upgrade-5-0-mljdp8jfhscw" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/canadacentral/hcpOperationStatuses/51ec54c6-385b-4f23-8fbf-a1f041a3140c
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: DeadlineExceeded
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/canadacentral/hcpOperationStatuses/51ec54c6-385b-4f23-8fbf-a1f041a3140c",
"name": "51ec54c6-385b-4f23-8fbf-a1f041a3140c",
"status": "Failed",
"startTime": "2026-10-02T23:43:28.788659857Z",
"endTime": "2026-10-03T00:12:29.488557875Z",
"error": {
"code": "DeadlineExceeded",
"message": "cluster creation did not complete before the deadline; [cosmosCluster] .api.url is empty; [servingCABundle] ServingCABundle not yet populated; [clusterServiceClusterStatus] cluster service has not been successfully created; [hypershiftHostedCluster] hosted cluster state not cached yet"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:133]: failed to create HCP cluster "cp-ystream-upgrade-5-0-fts22d" at install version 4.22.0-0.nightly-multi-2026-07-08-192436
Unexpected error:
<*fmt.wrapError | 0xc00111ac60>:
failed to create HCP cluster cp-ystream-upgrade-5-0-fts22d: failed waiting for cluster="cp-ystream-upgrade-5-0-fts22d" in resourcegroup="rg-cp-ystream-upgrade-5-0-mljdp8jfhscw" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/canadacentral/hcpOperationStatuses/51ec54c6-385b-4f23-8fbf-a1f041a3140c
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: DeadlineExceeded
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/canadacentral/hcpOperationStatuses/51ec54c6-385b-4f23-8fbf-a1f041a3140c",
"name": "51ec54c6-385b-4f23-8fbf-a1f041a3140c",
"status": "Failed",
"startTime": "2026-10-02T23:43:28.788659857Z",
"endTime": "2026-10-03T00:12:29.488557875Z",
"error": {
"code": "DeadlineExceeded",
"message": "cluster creation did not complete before the deadline; [cosmosCluster] .api.url is empty; [servingCABundle] ServingCABundle not yet populated; [clusterServiceClusterStatus] cluster service has not been successfully created; [hypershiftHostedCluster] hosted cluster state not cached yet"
}
}
--------------------------------------------------------------------------------
...
occurredERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-update-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift Show raw failurefail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:193]: failed to update HCP cluster tags via PATCH
Unexpected error:
<*exported.ResponseError | 0xc0015c28d0>:
PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-tags-czq6qgmpwn7z/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-tags-cluster
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"aro-hcp.experimental.cluster.max-update-duration\": unrecognized experimental tag",
"target": "tags[aro-hcp.experimental.cluster.max-update-duration]"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:193]: failed to update HCP cluster tags via PATCH
Unexpected error:
<*exported.ResponseError | 0xc0015c28d0>:
PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-tags-czq6qgmpwn7z/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-tags-cluster
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"aro-hcp.experimental.cluster.max-update-duration\": unrecognized experimental tag",
"target": "tags[aro-hcp.experimental.cluster.max-update-duration]"
}
}
--------------------------------------------------------------------------------
...
occurredERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-update-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift Show raw failurefail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:167]: failed to trigger y-stream upgrade of cluster "cp-ystream-upgrade-5-1-d88bcx" to 5.1.0-0.nightly-multi-2026-09-30-181926
Unexpected error:
<*exported.ResponseError | 0xc0010032f0>:
PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cp-ystream-upgrade-5-1-bh6j7gtlcdf2/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cp-ystream-upgrade-5-1-d88bcx
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"aro-hcp.experimental.cluster.max-update-duration\": unrecognized experimental tag",
"target": "tags[aro-hcp.experimental.cluster.max-update-duration]"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:167]: failed to trigger y-stream upgrade of cluster "cp-ystream-upgrade-5-1-d88bcx" to 5.1.0-0.nightly-multi-2026-09-30-181926
Unexpected error:
<*exported.ResponseError | 0xc0010032f0>:
PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cp-ystream-upgrade-5-1-bh6j7gtlcdf2/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cp-ystream-upgrade-5-1-d88bcx
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"aro-hcp.experimental.cluster.max-update-duration\": unrecognized experimental tag",
"target": "tags[aro-hcp.experimental.cluster.max-update-duration]"
}
}
--------------------------------------------------------------------------------
...
occurredERROR CODE: DeadlineExceeded; detail message cluster creation did not complete before the deadline; [cosmosCluster] .api.url is empty; [servingCABundle] ServingCABundle not yet populated; [clusterServiceClusterStatus] cluster service has not been successfully created; [hypershiftHostedCluster] hosted cluster state not cached yet; provider Microsoft.RedHatOpenShift Show raw failurefail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:123]: HCP cluster rg-nightly-4-22-m2xdl7kgwht2/cluster-nightly-4-22-q779x6 should provision
Unexpected error:
<*fmt.wrapError | 0xc0009c8f00>:
failed to create HCP cluster cluster-nightly-4-22-q779x6: failed waiting for cluster="cluster-nightly-4-22-q779x6" in resourcegroup="rg-nightly-4-22-m2xdl7kgwht2" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/canadacentral/hcpOperationStatuses/ecf6db04-6bc9-42e0-9339-05a023c3c7c2
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: DeadlineExceeded
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/canadacentral/hcpOperationStatuses/ecf6db04-6bc9-42e0-9339-05a023c3c7c2",
"name": "ecf6db04-6bc9-42e0-9339-05a023c3c7c2",
"status": "Failed",
"startTime": "2026-10-02T23:39:38.855575535Z",
"endTime": "2026-10-03T00:08:39.476306937Z",
"error": {
"code": "DeadlineExceeded",
"message": "cluster creation did not complete before the deadline; [cosmosCluster] .api.url is empty; [servingCABundle] ServingCABundle not yet populated; [clusterServiceClusterStatus] cluster service has not been successfully created; [hypershiftHostedCluster] hosted cluster state not cached yet"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:123]: HCP cluster rg-nightly-4-22-m2xdl7kgwht2/cluster-nightly-4-22-q779x6 should provision
Unexpected error:
<*fmt.wrapError | 0xc0009c8f00>:
failed to create HCP cluster cluster-nightly-4-22-q779x6: failed waiting for cluster="cluster-nightly-4-22-q779x6" in resourcegroup="rg-nightly-4-22-m2xdl7kgwht2" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/canadacentral/hcpOperationStatuses/ecf6db04-6bc9-42e0-9339-05a023c3c7c2
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: DeadlineExceeded
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/canadacentral/hcpOperationStatuses/ecf6db04-6bc9-42e0-9339-05a023c3c7c2",
"name": "ecf6db04-6bc9-42e0-9339-05a023c3c7c2",
"status": "Failed",
"startTime": "2026-10-02T23:39:38.855575535Z",
"endTime": "2026-10-03T00:08:39.476306937Z",
"error": {
"code": "DeadlineExceeded",
"message": "cluster creation did not complete before the deadline; [cosmosCluster] .api.url is empty; [servingCABundle] ServingCABundle not yet populated; [clusterServiceClusterStatus] cluster service has not been successfully created; [hypershiftHostedCluster] hosted cluster state not cached yet"
}
}
--------------------------------------------------------------------------------
...
occurredERROR CODE: DeadlineExceeded; detail message cluster creation did not complete before the deadline; [cosmosCluster] .api.url is empty; [servingCABundle] ServingCABundle not yet populated; [clusterServiceClusterStatus] cluster service has not been successfully created; [hypershiftHostedCluster] hosted cluster state not cached yet; provider Microsoft.RedHatOpenShift Show raw failurefail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:722]: failed to create HCP cluster
Unexpected error:
<*fmt.wrapError | 0xc000d72ea0>:
failed to create HCP cluster np-dg-minor-x975lx: failed waiting for cluster="np-dg-minor-x975lx" in resourcegroup="rg-np-dg-minor-x975lx-snvpx8zstwzg" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/canadacentral/hcpOperationStatuses/1bf1cc23-284f-4b81-90c6-b740356d67dd
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: DeadlineExceeded
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/canadacentral/hcpOperationStatuses/1bf1cc23-284f-4b81-90c6-b740356d67dd",
"name": "1bf1cc23-284f-4b81-90c6-b740356d67dd",
"status": "Failed",
"startTime": "2026-10-03T00:09:01.246824643Z",
"endTime": "2026-10-03T00:38:05.670946046Z",
"error": {
"code": "DeadlineExceeded",
"message": "cluster creation did not complete before the deadline; [cosmosCluster] .api.url is empty; [servingCABundle] ServingCABundle not yet populated; [clusterServiceClusterStatus] cluster service has not been successfully created; [hypershiftHostedCluster] hosted cluster state not cached yet"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:722]: failed to create HCP cluster
Unexpected error:
<*fmt.wrapError | 0xc000d72ea0>:
failed to create HCP cluster np-dg-minor-x975lx: failed waiting for cluster="np-dg-minor-x975lx" in resourcegroup="rg-np-dg-minor-x975lx-snvpx8zstwzg" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/canadacentral/hcpOperationStatuses/1bf1cc23-284f-4b81-90c6-b740356d67dd
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: DeadlineExceeded
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/canadacentral/hcpOperationStatuses/1bf1cc23-284f-4b81-90c6-b740356d67dd",
"name": "1bf1cc23-284f-4b81-90c6-b740356d67dd",
"status": "Failed",
"startTime": "2026-10-03T00:09:01.246824643Z",
"endTime": "2026-10-03T00:38:05.670946046Z",
"error": {
"code": "DeadlineExceeded",
"message": "cluster creation did not complete before the deadline; [cosmosCluster] .api.url is empty; [servingCABundle] ServingCABundle not yet populated; [clusterServiceClusterStatus] cluster service has not been successfully created; [hypershiftHostedCluster] hosted cluster state not cached yet"
}
}
--------------------------------------------------------------------------------
...
occurredERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-update-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift Show raw failurefail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:167]: failed to trigger y-stream upgrade of cluster "cp-ystream-upgrade-4-22-qssg82" to 4.22.0-0.nightly-multi-2026-07-08-192436
Unexpected error:
<*exported.ResponseError | 0xc0014d0480>:
PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cp-ystream-upgrade-4-22-cz4jzkrzbs6n/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cp-ystream-upgrade-4-22-qssg82
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"aro-hcp.experimental.cluster.max-update-duration\": unrecognized experimental tag",
"target": "tags[aro-hcp.experimental.cluster.max-update-duration]"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:167]: failed to trigger y-stream upgrade of cluster "cp-ystream-upgrade-4-22-qssg82" to 4.22.0-0.nightly-multi-2026-07-08-192436
Unexpected error:
<*exported.ResponseError | 0xc0014d0480>:
PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-cp-ystream-upgrade-4-22-cz4jzkrzbs6n/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cp-ystream-upgrade-4-22-qssg82
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"aro-hcp.experimental.cluster.max-update-duration\": unrecognized experimental tag",
"target": "tags[aro-hcp.experimental.cluster.max-update-duration]"
}
}
--------------------------------------------------------------------------------
...
occurredERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-update-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift Show raw failurefail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:173]: failed to update cluster "autoscaling-hcp-cluster" autoscaling configuration
Unexpected error:
<*exported.ResponseError | 0xc0026fb2f0>:
PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/autoscaling-cluster-cr2tzrn9kjll/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/autoscaling-hcp-cluster
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"aro-hcp.experimental.cluster.max-update-duration\": unrecognized experimental tag",
"target": "tags[aro-hcp.experimental.cluster.max-update-duration]"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:173]: failed to update cluster "autoscaling-hcp-cluster" autoscaling configuration
Unexpected error:
<*exported.ResponseError | 0xc0026fb2f0>:
PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/autoscaling-cluster-cr2tzrn9kjll/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/autoscaling-hcp-cluster
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"aro-hcp.experimental.cluster.max-update-duration\": unrecognized experimental tag",
"target": "tags[aro-hcp.experimental.cluster.max-update-duration]"
}
}
--------------------------------------------------------------------------------
...
occurredERROR CODE: InvalidRequestContent; detail message Invalid value: "<ocp-nightly>": must be at least 4.20.8; provider Microsoft.RedHatOpenShift Show raw failurefail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:165]: failed to create node pool npupgrade-4-20 with version 4.20.0-0.nightly-multi-2026-10-01-125400
Unexpected error:
<*fmt.wrapError | 0xc000de8120>:
failed to create NodePool npupgrade-4-20: failed starting nodepool creation "npupgrade-4-20" for cluster "np-version-upgrade-cluster-4slk4t" in resourcegroup="rg-np-version-upgrade-4slk4t-r4jcn6z5vwbl": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-version-upgrade-4slk4t-r4jcn6z5vwbl/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-version-upgrade-cluster-4slk4t/nodePools/npupgrade-4-20
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"4.20.0-0.nightly-multi-2026-10-01-125400\": must be at least 4.20.8",
"target": "properties.version.id"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:165]: failed to create node pool npupgrade-4-20 with version 4.20.0-0.nightly-multi-2026-10-01-125400
Unexpected error:
<*fmt.wrapError | 0xc000de8120>:
failed to create NodePool npupgrade-4-20: failed starting nodepool creation "npupgrade-4-20" for cluster "np-version-upgrade-cluster-4slk4t" in resourcegroup="rg-np-version-upgrade-4slk4t-r4jcn6z5vwbl": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-version-upgrade-4slk4t-r4jcn6z5vwbl/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-version-upgrade-cluster-4slk4t/nodePools/npupgrade-4-20
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"4.20.0-0.nightly-multi-2026-10-01-125400\": must be at least 4.20.8",
"target": "properties.version.id"
}
}
--------------------------------------------------------------------------------
...
occurredERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.max-update-duration": unrecognized experimental tag; provider Microsoft.RedHatOpenShift Show raw failurefail [github.com/Azure/ARO-HCP/test/e2e/idms_lifecycle.go:175]: failed to update cluster with second ImageDigestMirror set
Unexpected error:
<*exported.ResponseError | 0xc001074300>:
PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/idms-vhhr845dxfj7/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/idms-e2e-hcp-cluster
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"aro-hcp.experimental.cluster.max-update-duration\": unrecognized experimental tag",
"target": "tags[aro-hcp.experimental.cluster.max-update-duration]"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/idms_lifecycle.go:175]: failed to update cluster with second ImageDigestMirror set
Unexpected error:
<*exported.ResponseError | 0xc001074300>:
PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/idms-vhhr845dxfj7/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/idms-e2e-hcp-cluster
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"aro-hcp.experimental.cluster.max-update-duration\": unrecognized experimental tag",
"target": "tags[aro-hcp.experimental.cluster.max-update-duration]"
}
}
--------------------------------------------------------------------------------
...
occurredERROR CODE: DeadlineExceeded; detail message cluster creation did not complete before the deadline; [cosmosCluster] .api.url is empty; [servingCABundle] ServingCABundle not yet populated; [clusterServiceClusterStatus] cluster service has not been successfully created; [hypershiftHostedCluster] hosted cluster state not cached yet; provider Microsoft.RedHatOpenShift Show raw failurefail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:133]: failed to create HCP cluster "cp-ystream-upgrade-4-23-wlr9js" at install version 4.22.0-0.nightly-multi-2026-07-08-192436
Unexpected error:
<*fmt.wrapError | 0xc000d5cb20>:
failed to create HCP cluster cp-ystream-upgrade-4-23-wlr9js: failed waiting for cluster="cp-ystream-upgrade-4-23-wlr9js" in resourcegroup="rg-cp-ystream-upgrade-4-23-hsrbsrxk7qbw" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/canadacentral/hcpOperationStatuses/c9a34556-9c6b-44f5-9047-c18c0b099bd1
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: DeadlineExceeded
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/canadacentral/hcpOperationStatuses/c9a34556-9c6b-44f5-9047-c18c0b099bd1",
"name": "c9a34556-9c6b-44f5-9047-c18c0b099bd1",
"status": "Failed",
"startTime": "2026-10-02T23:42:52.038313719Z",
"endTime": "2026-10-03T00:11:59.486058349Z",
"error": {
"code": "DeadlineExceeded",
"message": "cluster creation did not complete before the deadline; [cosmosCluster] .api.url is empty; [servingCABundle] ServingCABundle not yet populated; [clusterServiceClusterStatus] cluster service has not been successfully created; [hypershiftHostedCluster] hosted cluster state not cached yet"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:133]: failed to create HCP cluster "cp-ystream-upgrade-4-23-wlr9js" at install version 4.22.0-0.nightly-multi-2026-07-08-192436
Unexpected error:
<*fmt.wrapError | 0xc000d5cb20>:
failed to create HCP cluster cp-ystream-upgrade-4-23-wlr9js: failed waiting for cluster="cp-ystream-upgrade-4-23-wlr9js" in resourcegroup="rg-cp-ystream-upgrade-4-23-hsrbsrxk7qbw" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/canadacentral/hcpOperationStatuses/c9a34556-9c6b-44f5-9047-c18c0b099bd1
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: DeadlineExceeded
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/canadacentral/hcpOperationStatuses/c9a34556-9c6b-44f5-9047-c18c0b099bd1",
"name": "c9a34556-9c6b-44f5-9047-c18c0b099bd1",
"status": "Failed",
"startTime": "2026-10-02T23:42:52.038313719Z",
"endTime": "2026-10-03T00:11:59.486058349Z",
"error": {
"code": "DeadlineExceeded",
"message": "cluster creation did not complete before the deadline; [cosmosCluster] .api.url is empty; [servingCABundle] ServingCABundle not yet populated; [clusterServiceClusterStatus] cluster service has not been successfully created; [hypershiftHostedCluster] hosted cluster state not cached yet"
}
}
--------------------------------------------------------------------------------
...
occurred |
|
| 4h 14m 50s | branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel post-goodmerged PR |
centralindia | n/a | passed | n/a | 0 | n/a | |
| 4h 12m 57s | branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel post-goodmerged PR |
canadacentral | n/a | passed | n/a | 0 | n/a | |
| 2h 44m 18s | branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel post-goodmerged PR |
eastus2 | e2e | failed | n/a | 2 | e2e (2)ERROR CODE: DeadlineExceeded; detail message cluster creation did not complete before the deadline; [clusterServiceClusterStatus] cluster service is installing; [hypershiftHostedCluster] hosted cluster is not available: KubeconfigWaitingForCreate: Waiting for hosted control plane kubeconfig to be created; hosted cluster degraded: UnavailableReplicas: [capi-provider]; provider Microsoft.RedHatOpenShift Show raw failurefail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:208]: failed to create HCP cluster "agg-cluster" with aggregated settings
Unexpected error:
<*fmt.wrapError | 0xc0022ce920>:
failed waiting for cluster="agg-cluster" in resourcegroup="feature-aggregation-fgvr8lf6bszk" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2/hcpOperationStatuses/db510478-c50d-4f9a-86b6-3bb190ec8daa
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: DeadlineExceeded
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2/hcpOperationStatuses/db510478-c50d-4f9a-86b6-3bb190ec8daa",
"name": "db510478-c50d-4f9a-86b6-3bb190ec8daa",
"status": "Failed",
"startTime": "2026-10-02T17:45:05.919871003Z",
"endTime": "2026-10-02T18:14:10.848146933Z",
"error": {
"code": "DeadlineExceeded",
"message": "cluster creation did not complete before the deadline; [clusterServiceClusterStatus] cluster service is installing; [hypershiftHostedCluster] hosted cluster is not available: KubeconfigWaitingForCreate: Waiting for hosted control plane kubeconfig to be created; hosted cluster degraded: UnavailableReplicas: capi-provider deployment has 1 unavailable replicas"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:208]: failed to create HCP cluster "agg-cluster" with aggregated settings
Unexpected error:
<*fmt.wrapError | 0xc0022ce920>:
failed waiting for cluster="agg-cluster" in resourcegroup="feature-aggregation-fgvr8lf6bszk" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2/hcpOperationStatuses/db510478-c50d-4f9a-86b6-3bb190ec8daa
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: DeadlineExceeded
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2/hcpOperationStatuses/db510478-c50d-4f9a-86b6-3bb190ec8daa",
"name": "db510478-c50d-4f9a-86b6-3bb190ec8daa",
"status": "Failed",
"startTime": "2026-10-02T17:45:05.919871003Z",
"endTime": "2026-10-02T18:14:10.848146933Z",
"error": {
"code": "DeadlineExceeded",
"message": "cluster creation did not complete before the deadline; [clusterServiceClusterStatus] cluster service is installing; [hypershiftHostedCluster] hosted cluster is not available: KubeconfigWaitingForCreate: Waiting for hosted control plane kubeconfig to be created; hosted cluster degraded: UnavailableReplicas: capi-provider deployment has 1 unavailable replicas"
}
}
--------------------------------------------------------------------------------
...
occurredselector "<selector>" matched no usable VM size in eastus2: no unrestricted VM size satisfied the selector in the test location; this typically indicates a VM SKU restriction or quota issue with the test subscription/region, not a product defect Show raw failurefail [github.com/Azure/ARO-HCP/test/e2e/nodepool_ephemeral_osdisk.go:88]: failed to select a VM size with ephemeral OS disk support; this typically indicates a SKU restriction or quota issue in the test subscription/region
Unexpected error:
<*fmt.wrapError | 0xc001361080>:
selector "ephemeral-osdisk-worker" matched no usable VM size in eastus2: no unrestricted VM size satisfied the selector in the test location; this typically indicates a VM SKU restriction or quota issue with the test subscription/region, not a product defect
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_ephemeral_osdisk.go:88]: failed to select a VM size with ephemeral OS disk support; this typically indicates a SKU restriction or quota issue in the test subscription/region
Unexpected error:
<*fmt.wrapError | 0xc001361080>:
selector "ephemeral-osdisk-worker" matched no usable VM size in eastus2: no unrestricted VM size satisfied the selector in the test location; this typically indicates a VM SKU restriction or quota issue with the test subscription/region, not a product defect
...
occurred |
|
| 2h 25m 21s | branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel post-goodmerged PR |
westeurope | n/a | passed | n/a | 0 | n/a | |
| 2h 30m 8s | branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel post-goodmerged PR |
brazilsouth | e2e | failed | n/a | 2 | e2e (2)VerifyFIPSEnabled failed: failed to check FIPS mode on node <node>: failed to get pod logs for pod fips-check-<id> in namespace e2e-fips-test-<id> on node <node> after retries: an error on the server ("unknown") has prevented the request from succeeding (get Show raw failurefail [github.com/Azure/ARO-HCP/test/e2e/cluster_fips_mode.go:123]: failed to verify FIPS is enabled on cluster fips-enabled-cluster
Unexpected error:
<*errors.joinError | 0xc001253878>:
VerifyFIPSEnabled failed: failed to check FIPS mode on node c7z4o0b2f4w6f5y-np-1-7fd77-f8d28: failed to get pod logs for pod fips-check-z86wg in namespace e2e-fips-test-pdk46 on node c7z4o0b2f4w6f5y-np-1-7fd77-f8d28 after retries: an error on the server ("unknown") has prevented the request from succeeding (get pods fips-check-z86wg)
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_fips_mode.go:123]: failed to verify FIPS is enabled on cluster fips-enabled-cluster
Unexpected error:
<*errors.joinError | 0xc001253878>:
VerifyFIPSEnabled failed: failed to check FIPS mode on node c7z4o0b2f4w6f5y-np-1-7fd77-f8d28: failed to get pod logs for pod fips-check-z86wg in namespace e2e-fips-test-pdk46 on node c7z4o0b2f4w6f5y-np-1-7fd77-f8d28 after retries: an error on the server ("unknown") has prevented the request from succeeding (get pods fips-check-z86wg)
...
occurredERROR CODE: DeadlineExceeded; provider Microsoft.Compute Show raw failurefail [github.com/Azure/ARO-HCP/test/e2e/oidc_issuer_workload_identity.go:270]: failed to create node pool oidc-wi-np
Unexpected error:
<*fmt.wrapError | 0xc000c93500>:
failed to create NodePool oidc-wi-np: failed waiting for nodepool="oidc-wi-np" for cluster "oidc-wi-cluster" in resourcegroup="oidc-wi-tlcz46v8f5nd" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/9d16ce0e-bad0-4102-85e4-8ff744b9a2fd
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: DeadlineExceeded
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/9d16ce0e-bad0-4102-85e4-8ff744b9a2fd",
"name": "9d16ce0e-bad0-4102-85e4-8ff744b9a2fd",
"status": "Failed",
"startTime": "2026-10-02T19:19:55.216406379Z",
"endTime": "2026-10-02T19:39:00.624366265Z",
"error": {
"code": "DeadlineExceeded",
"message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 1, want 2; node pool condition AllNodesHealthy is False: 1 of 2 machines are not healthy\nMachine oidc-wi-cluster-oidc-wi-np-9wdrv-tkpqf: InspectionFailed: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/oidc-wi-tlcz46v8f5nd--managed/providers/Microsoft.Compute/virtualMachines/oidc-wi-cluster-oidc-wi-np-9wdrv-tkpqf to exist\n; node pool condition AllMachinesReady is False: 1 of 2 machines are not ready\nMachine oidc-wi-cluster-oidc-wi-np-9wdrv-tkpqf: ReadyUnknown: * NodeHealthy: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/oidc-wi-tlcz46v8f5nd--managed/providers/Microsoft.Compute/virtualMachines/oidc-wi-cluster-oidc-wi-np-9wdrv-tkpqf to exist\n"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/oidc_issuer_workload_identity.go:270]: failed to create node pool oidc-wi-np
Unexpected error:
<*fmt.wrapError | 0xc000c93500>:
failed to create NodePool oidc-wi-np: failed waiting for nodepool="oidc-wi-np" for cluster "oidc-wi-cluster" in resourcegroup="oidc-wi-tlcz46v8f5nd" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/9d16ce0e-bad0-4102-85e4-8ff744b9a2fd
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: DeadlineExceeded
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/9d16ce0e-bad0-4102-85e4-8ff744b9a2fd",
"name": "9d16ce0e-bad0-4102-85e4-8ff744b9a2fd",
"status": "Failed",
"startTime": "2026-10-02T19:19:55.216406379Z",
"endTime": "2026-10-02T19:39:00.624366265Z",
"error": {
"code": "DeadlineExceeded",
"message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 1, want 2; node pool condition AllNodesHealthy is False: 1 of 2 machines are not healthy\nMachine oidc-wi-cluster-oidc-wi-np-9wdrv-tkpqf: InspectionFailed: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/oidc-wi-tlcz46v8f5nd--managed/providers/Microsoft.Compute/virtualMachines/oidc-wi-cluster-oidc-wi-np-9wdrv-tkpqf to exist\n; node pool condition AllMachinesReady is False: 1 of 2 machines are not ready\nMachine oidc-wi-cluster-oidc-wi-np-9wdrv-tkpqf: ReadyUnknown: * NodeHealthy: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/oidc-wi-tlcz46v8f5nd--managed/providers/Microsoft.Compute/virtualMachines/oidc-wi-cluster-oidc-wi-np-9wdrv-tkpqf to exist\n"
}
}
--------------------------------------------------------------------------------
...
occurred |
|
| 2h 24m 58s | branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel post-goodmerged PR |
australiaeast | n/a | passed | n/a | 0 | n/a | |
| 2h 52m 31s | branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel post-goodmerged PR |
westeurope | e2e | failed | n/a | 2 | e2e (2)ERROR CODE: DeadlineExceeded; provider Microsoft.Compute Show raw failurefail [github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:156]: failed to create node pool np-1 for TLS endpoint cluster
Unexpected error:
<*fmt.wrapError | 0xc00109c260>:
failed to create NodePool np-1: failed waiting for nodepool="np-1" for cluster "tls-endpoint-hcp-cluster" in resourcegroup="tls-endpoint-cluster-vl5bmxfjpvnl" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/76a0d1d5-c260-480b-9486-2626418cd0aa
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: DeadlineExceeded
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/76a0d1d5-c260-480b-9486-2626418cd0aa",
"name": "76a0d1d5-c260-480b-9486-2626418cd0aa",
"status": "Failed",
"startTime": "2026-10-02T15:16:07.904782817Z",
"endTime": "2026-10-02T15:35:14.636915839Z",
"error": {
"code": "DeadlineExceeded",
"message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 1, want 2; node pool condition AllNodesHealthy is False: 1 of 2 machines are not healthy\nMachine p5k5p2r5e7o0k9d-np-1-pmf67-7nch9: InspectionFailed: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/tls-endpoint-cluster-vl5bmxfjpvnl--managed/providers/Microsoft.Compute/virtualMachines/p5k5p2r5e7o0k9d-np-1-pmf67-7nch9 to exist\n; node pool condition AllMachinesReady is False: 1 of 2 machines are not ready\nMachine p5k5p2r5e7o0k9d-np-1-pmf67-7nch9: ReadyUnknown: * NodeHealthy: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/tls-endpoint-cluster-vl5bmxfjpvnl--managed/providers/Microsoft.Compute/virtualMachines/p5k5p2r5e7o0k9d-np-1-pmf67-7nch9 to exist\n"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:156]: failed to create node pool np-1 for TLS endpoint cluster
Unexpected error:
<*fmt.wrapError | 0xc00109c260>:
failed to create NodePool np-1: failed waiting for nodepool="np-1" for cluster "tls-endpoint-hcp-cluster" in resourcegroup="tls-endpoint-cluster-vl5bmxfjpvnl" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/76a0d1d5-c260-480b-9486-2626418cd0aa
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: DeadlineExceeded
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/76a0d1d5-c260-480b-9486-2626418cd0aa",
"name": "76a0d1d5-c260-480b-9486-2626418cd0aa",
"status": "Failed",
"startTime": "2026-10-02T15:16:07.904782817Z",
"endTime": "2026-10-02T15:35:14.636915839Z",
"error": {
"code": "DeadlineExceeded",
"message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 1, want 2; node pool condition AllNodesHealthy is False: 1 of 2 machines are not healthy\nMachine p5k5p2r5e7o0k9d-np-1-pmf67-7nch9: InspectionFailed: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/tls-endpoint-cluster-vl5bmxfjpvnl--managed/providers/Microsoft.Compute/virtualMachines/p5k5p2r5e7o0k9d-np-1-pmf67-7nch9 to exist\n; node pool condition AllMachinesReady is False: 1 of 2 machines are not ready\nMachine p5k5p2r5e7o0k9d-np-1-pmf67-7nch9: ReadyUnknown: * NodeHealthy: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/tls-endpoint-cluster-vl5bmxfjpvnl--managed/providers/Microsoft.Compute/virtualMachines/p5k5p2r5e7o0k9d-np-1-pmf67-7nch9 to exist\n"
}
}
--------------------------------------------------------------------------------
...
occurredERROR CODE: DeadlineExceeded; provider Microsoft.Compute Show raw failurefail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-qx4psk" for cluster "cluster-candidate-5-1-qx4psk"
Unexpected error:
<*fmt.wrapError | 0xc0013d5b40>:
failed to create NodePool np-qx4psk: failed waiting for nodepool="np-qx4psk" for cluster "cluster-candidate-5-1-qx4psk" in resourcegroup="rg-candidate-5-1-kt5m7922qfx8" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/08a3ce96-4944-47f9-8702-e2ad9f78a6cf
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: DeadlineExceeded
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/08a3ce96-4944-47f9-8702-e2ad9f78a6cf",
"name": "08a3ce96-4944-47f9-8702-e2ad9f78a6cf",
"status": "Failed",
"startTime": "2026-10-02T15:30:38.36539796Z",
"endTime": "2026-10-02T15:49:40.880753059Z",
"error": {
"code": "DeadlineExceeded",
"message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 1, want 2; node pool condition AllNodesHealthy is False: 1 of 2 machines are not healthy\nMachine z1v0n8y8q7v5r0b-np-qx4psk-v8zlj-7bpw8: InspectionFailed: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-5-1-kt5m7922qfx8-candidate-qx4psk--managed/providers/Microsoft.Compute/virtualMachines/z1v0n8y8q7v5r0b-np-qx4psk-v8zlj-7bpw8 to exist\n; node pool condition AllMachinesReady is False: 1 of 2 machines are not ready\nMachine z1v0n8y8q7v5r0b-np-qx4psk-v8zlj-7bpw8: ReadyUnknown: * NodeHealthy: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-5-1-kt5m7922qfx8-candidate-qx4psk--managed/providers/Microsoft.Compute/virtualMachines/z1v0n8y8q7v5r0b-np-qx4psk-v8zlj-7bpw8 to exist\n"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:176]: failed to create node pool "np-qx4psk" for cluster "cluster-candidate-5-1-qx4psk"
Unexpected error:
<*fmt.wrapError | 0xc0013d5b40>:
failed to create NodePool np-qx4psk: failed waiting for nodepool="np-qx4psk" for cluster "cluster-candidate-5-1-qx4psk" in resourcegroup="rg-candidate-5-1-kt5m7922qfx8" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/08a3ce96-4944-47f9-8702-e2ad9f78a6cf
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: DeadlineExceeded
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/westeurope/hcpOperationStatuses/08a3ce96-4944-47f9-8702-e2ad9f78a6cf",
"name": "08a3ce96-4944-47f9-8702-e2ad9f78a6cf",
"status": "Failed",
"startTime": "2026-10-02T15:30:38.36539796Z",
"endTime": "2026-10-02T15:49:40.880753059Z",
"error": {
"code": "DeadlineExceeded",
"message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 1, want 2; node pool condition AllNodesHealthy is False: 1 of 2 machines are not healthy\nMachine z1v0n8y8q7v5r0b-np-qx4psk-v8zlj-7bpw8: InspectionFailed: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-5-1-kt5m7922qfx8-candidate-qx4psk--managed/providers/Microsoft.Compute/virtualMachines/z1v0n8y8q7v5r0b-np-qx4psk-v8zlj-7bpw8 to exist\n; node pool condition AllMachinesReady is False: 1 of 2 machines are not ready\nMachine z1v0n8y8q7v5r0b-np-qx4psk-v8zlj-7bpw8: ReadyUnknown: * NodeHealthy: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-5-1-kt5m7922qfx8-candidate-qx4psk--managed/providers/Microsoft.Compute/virtualMachines/z1v0n8y8q7v5r0b-np-qx4psk-v8zlj-7bpw8 to exist\n"
}
}
--------------------------------------------------------------------------------
...
occurred |
|
| 2h 47m 16s | branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel post-goodmerged PR |
brazilsouth | n/a | passed | n/a | 0 | n/a | |
| 2h 42m 6s | branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel post-goodmerged PR |
australiaeast | n/a | passed | n/a | 0 | n/a | |
| 3h 1m 42s | branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel post-goodmerged PR |
eastus2 | e2e | failed | n/a | 7 | e2e (7)ERROR CODE: DeadlineExceeded; detail message cluster creation did not complete before the deadline; [clusterServiceClusterStatus] cluster service is installing; [hypershiftHostedCluster] hosted cluster is not available: KASLoadBalancerNotReachable: APIServer external route not admitted; hosted cluster degraded: UnavailableReplicas: [router]; provider Microsoft.RedHatOpenShift Show raw failurefail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:123]: HCP cluster rg-candidate-4-20-494d7tjl9z6l/cluster-candidate-4-20-zdwtn5 should provision
Unexpected error:
<*fmt.wrapError | 0xc000c31000>:
failed to create HCP cluster cluster-candidate-4-20-zdwtn5: failed waiting for cluster="cluster-candidate-4-20-zdwtn5" in resourcegroup="rg-candidate-4-20-494d7tjl9z6l" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2/hcpOperationStatuses/cecb851f-dab3-443d-be30-9e210495bb30
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: DeadlineExceeded
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2/hcpOperationStatuses/cecb851f-dab3-443d-be30-9e210495bb30",
"name": "cecb851f-dab3-443d-be30-9e210495bb30",
"status": "Failed",
"startTime": "2026-10-02T14:47:02.598518687Z",
"endTime": "2026-10-02T15:16:11.481682438Z",
"error": {
"code": "DeadlineExceeded",
"message": "cluster creation did not complete before the deadline; [clusterServiceClusterStatus] cluster service is installing; [hypershiftHostedCluster] hosted cluster is not available: KASLoadBalancerNotReachable: APIServer external route not admitted; hosted cluster degraded: UnavailableReplicas: router deployment has 2 unavailable replicas"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:123]: HCP cluster rg-candidate-4-20-494d7tjl9z6l/cluster-candidate-4-20-zdwtn5 should provision
Unexpected error:
<*fmt.wrapError | 0xc000c31000>:
failed to create HCP cluster cluster-candidate-4-20-zdwtn5: failed waiting for cluster="cluster-candidate-4-20-zdwtn5" in resourcegroup="rg-candidate-4-20-494d7tjl9z6l" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2/hcpOperationStatuses/cecb851f-dab3-443d-be30-9e210495bb30
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: DeadlineExceeded
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2/hcpOperationStatuses/cecb851f-dab3-443d-be30-9e210495bb30",
"name": "cecb851f-dab3-443d-be30-9e210495bb30",
"status": "Failed",
"startTime": "2026-10-02T14:47:02.598518687Z",
"endTime": "2026-10-02T15:16:11.481682438Z",
"error": {
"code": "DeadlineExceeded",
"message": "cluster creation did not complete before the deadline; [clusterServiceClusterStatus] cluster service is installing; [hypershiftHostedCluster] hosted cluster is not available: KASLoadBalancerNotReachable: APIServer external route not admitted; hosted cluster degraded: UnavailableReplicas: router deployment has 2 unavailable replicas"
}
}
--------------------------------------------------------------------------------
...
occurredERROR CODE: DeadlineExceeded; detail message cluster creation did not complete before the deadline; [hypershiftHostedCluster] hosted cluster is not available: KASLoadBalancerNotReachable: APIServer external route not admitted; hosted cluster degraded: UnavailableReplicas: [router]; provider Microsoft.RedHatOpenShift Show raw failurefail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:208]: failed to create HCP cluster "agg-cluster" with aggregated settings
Unexpected error:
<*fmt.wrapError | 0xc002486980>:
failed waiting for cluster="agg-cluster" in resourcegroup="feature-aggregation-4z7j926fljlc" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2/hcpOperationStatuses/f9754b3c-7f81-4d15-8726-14659a0f0d2b
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: DeadlineExceeded
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2/hcpOperationStatuses/f9754b3c-7f81-4d15-8726-14659a0f0d2b",
"name": "f9754b3c-7f81-4d15-8726-14659a0f0d2b",
"status": "Failed",
"startTime": "2026-10-02T14:15:50.839633613Z",
"endTime": "2026-10-02T14:44:52.180223552Z",
"error": {
"code": "DeadlineExceeded",
"message": "cluster creation did not complete before the deadline; [hypershiftHostedCluster] hosted cluster is not available: KASLoadBalancerNotReachable: APIServer external route not admitted; hosted cluster degraded: UnavailableReplicas: router deployment has 1 unavailable replicas"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:208]: failed to create HCP cluster "agg-cluster" with aggregated settings
Unexpected error:
<*fmt.wrapError | 0xc002486980>:
failed waiting for cluster="agg-cluster" in resourcegroup="feature-aggregation-4z7j926fljlc" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2/hcpOperationStatuses/f9754b3c-7f81-4d15-8726-14659a0f0d2b
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: DeadlineExceeded
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2/hcpOperationStatuses/f9754b3c-7f81-4d15-8726-14659a0f0d2b",
"name": "f9754b3c-7f81-4d15-8726-14659a0f0d2b",
"status": "Failed",
"startTime": "2026-10-02T14:15:50.839633613Z",
"endTime": "2026-10-02T14:44:52.180223552Z",
"error": {
"code": "DeadlineExceeded",
"message": "cluster creation did not complete before the deadline; [hypershiftHostedCluster] hosted cluster is not available: KASLoadBalancerNotReachable: APIServer external route not admitted; hosted cluster degraded: UnavailableReplicas: router deployment has 1 unavailable replicas"
}
}
--------------------------------------------------------------------------------
...
occurredERROR CODE: DeadlineExceeded; detail message cluster creation did not complete before the deadline; [hypershiftHostedCluster] hosted cluster control plane version not yet completed: version <version> is Partial (want Completed), started <duration> ago; hosted cluster degraded: UnavailableReplicas: [router]; provider Microsoft.RedHatOpenShift Show raw failurefail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:123]: HCP cluster rg-candidate-4-22-48fc7bfssxlp/cluster-candidate-4-22-7klbxb should provision
Unexpected error:
<*fmt.wrapError | 0xc000d40740>:
failed to create HCP cluster cluster-candidate-4-22-7klbxb: failed waiting for cluster="cluster-candidate-4-22-7klbxb" in resourcegroup="rg-candidate-4-22-48fc7bfssxlp" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2/hcpOperationStatuses/3f2efc0e-134f-4975-8954-ac850a668709
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: DeadlineExceeded
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2/hcpOperationStatuses/3f2efc0e-134f-4975-8954-ac850a668709",
"name": "3f2efc0e-134f-4975-8954-ac850a668709",
"status": "Failed",
"startTime": "2026-10-02T14:48:41.860181794Z",
"endTime": "2026-10-02T15:17:41.529062952Z",
"error": {
"code": "DeadlineExceeded",
"message": "cluster creation did not complete before the deadline; [hypershiftHostedCluster] hosted cluster control plane version not yet completed: version 4.22.16 is Partial (want Completed), started 23m55s ago; hosted cluster degraded: UnavailableReplicas: router deployment has 1 unavailable replicas"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:123]: HCP cluster rg-candidate-4-22-48fc7bfssxlp/cluster-candidate-4-22-7klbxb should provision
Unexpected error:
<*fmt.wrapError | 0xc000d40740>:
failed to create HCP cluster cluster-candidate-4-22-7klbxb: failed waiting for cluster="cluster-candidate-4-22-7klbxb" in resourcegroup="rg-candidate-4-22-48fc7bfssxlp" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2/hcpOperationStatuses/3f2efc0e-134f-4975-8954-ac850a668709
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: DeadlineExceeded
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2/hcpOperationStatuses/3f2efc0e-134f-4975-8954-ac850a668709",
"name": "3f2efc0e-134f-4975-8954-ac850a668709",
"status": "Failed",
"startTime": "2026-10-02T14:48:41.860181794Z",
"endTime": "2026-10-02T15:17:41.529062952Z",
"error": {
"code": "DeadlineExceeded",
"message": "cluster creation did not complete before the deadline; [hypershiftHostedCluster] hosted cluster control plane version not yet completed: version 4.22.16 is Partial (want Completed), started 23m55s ago; hosted cluster degraded: UnavailableReplicas: router deployment has 1 unavailable replicas"
}
}
--------------------------------------------------------------------------------
...
occurredERROR CODE: DeadlineExceeded; detail message cluster creation did not complete before the deadline; [clusterServiceClusterStatus] cluster service is installing; [hypershiftHostedCluster] hosted cluster is not available: ComponentsNotAvailable; hosted cluster degraded: UnavailableReplicas: [router]; provider Microsoft.RedHatOpenShift Show raw failurefail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:99]: failed to create HCP cluster "private-kv-cluster" with private keyvault
Unexpected error:
<*fmt.wrapError | 0xc0010ae5a0>:
failed waiting for cluster="private-kv-cluster" in resourcegroup="private-keyvault-k4q5zpfgpr4k" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2/hcpOperationStatuses/3c73c7ae-1999-4cb4-b92d-cbffe4356bef
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: DeadlineExceeded
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2/hcpOperationStatuses/3c73c7ae-1999-4cb4-b92d-cbffe4356bef",
"name": "3c73c7ae-1999-4cb4-b92d-cbffe4356bef",
"status": "Failed",
"startTime": "2026-10-02T14:14:49.716994175Z",
"endTime": "2026-10-02T14:43:52.152352909Z",
"error": {
"code": "DeadlineExceeded",
"message": "cluster creation did not complete before the deadline; [clusterServiceClusterStatus] cluster service is installing; [hypershiftHostedCluster] hosted cluster is not available: ComponentsNotAvailable: Waiting for components to be available: router; hosted cluster degraded: UnavailableReplicas: router deployment has 2 unavailable replicas"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:99]: failed to create HCP cluster "private-kv-cluster" with private keyvault
Unexpected error:
<*fmt.wrapError | 0xc0010ae5a0>:
failed waiting for cluster="private-kv-cluster" in resourcegroup="private-keyvault-k4q5zpfgpr4k" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2/hcpOperationStatuses/3c73c7ae-1999-4cb4-b92d-cbffe4356bef
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: DeadlineExceeded
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2/hcpOperationStatuses/3c73c7ae-1999-4cb4-b92d-cbffe4356bef",
"name": "3c73c7ae-1999-4cb4-b92d-cbffe4356bef",
"status": "Failed",
"startTime": "2026-10-02T14:14:49.716994175Z",
"endTime": "2026-10-02T14:43:52.152352909Z",
"error": {
"code": "DeadlineExceeded",
"message": "cluster creation did not complete before the deadline; [clusterServiceClusterStatus] cluster service is installing; [hypershiftHostedCluster] hosted cluster is not available: ComponentsNotAvailable: Waiting for components to be available: router; hosted cluster degraded: UnavailableReplicas: router deployment has 2 unavailable replicas"
}
}
--------------------------------------------------------------------------------
...
occurredERROR CODE: DeadlineExceeded; detail message cluster creation did not complete before the deadline; [clusterServiceClusterStatus] cluster service is installing; [hypershiftHostedCluster] hosted cluster is not available: KASLoadBalancerNotReachable: APIServer external route not admitted; hosted cluster degraded: UnavailableReplicas: [router]; provider Microsoft.RedHatOpenShift Show raw failurefail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:123]: HCP cluster rg-candidate-4-21-rf66b2pjnzs5/cluster-candidate-4-21-zp7tkm should provision
Unexpected error:
<*fmt.wrapError | 0xc000a9f080>:
failed to create HCP cluster cluster-candidate-4-21-zp7tkm: failed waiting for cluster="cluster-candidate-4-21-zp7tkm" in resourcegroup="rg-candidate-4-21-rf66b2pjnzs5" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2/hcpOperationStatuses/f866fdfa-caa6-4c46-a375-3e94e0748ca1
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: DeadlineExceeded
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2/hcpOperationStatuses/f866fdfa-caa6-4c46-a375-3e94e0748ca1",
"name": "f866fdfa-caa6-4c46-a375-3e94e0748ca1",
"status": "Failed",
"startTime": "2026-10-02T14:48:13.433342194Z",
"endTime": "2026-10-02T15:17:21.452859562Z",
"error": {
"code": "DeadlineExceeded",
"message": "cluster creation did not complete before the deadline; [clusterServiceClusterStatus] cluster service is installing; [hypershiftHostedCluster] hosted cluster is not available: KASLoadBalancerNotReachable: APIServer external route not admitted; hosted cluster degraded: UnavailableReplicas: router deployment has 2 unavailable replicas"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:123]: HCP cluster rg-candidate-4-21-rf66b2pjnzs5/cluster-candidate-4-21-zp7tkm should provision
Unexpected error:
<*fmt.wrapError | 0xc000a9f080>:
failed to create HCP cluster cluster-candidate-4-21-zp7tkm: failed waiting for cluster="cluster-candidate-4-21-zp7tkm" in resourcegroup="rg-candidate-4-21-rf66b2pjnzs5" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2/hcpOperationStatuses/f866fdfa-caa6-4c46-a375-3e94e0748ca1
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: DeadlineExceeded
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2/hcpOperationStatuses/f866fdfa-caa6-4c46-a375-3e94e0748ca1",
"name": "f866fdfa-caa6-4c46-a375-3e94e0748ca1",
"status": "Failed",
"startTime": "2026-10-02T14:48:13.433342194Z",
"endTime": "2026-10-02T15:17:21.452859562Z",
"error": {
"code": "DeadlineExceeded",
"message": "cluster creation did not complete before the deadline; [clusterServiceClusterStatus] cluster service is installing; [hypershiftHostedCluster] hosted cluster is not available: KASLoadBalancerNotReachable: APIServer external route not admitted; hosted cluster degraded: UnavailableReplicas: router deployment has 2 unavailable replicas"
}
}
--------------------------------------------------------------------------------
...
occurredselector "<selector>" matched no usable VM size in eastus2: no unrestricted VM size satisfied the selector in the test location; this typically indicates a VM SKU restriction or quota issue with the test subscription/region, not a product defect Show raw failurefail [github.com/Azure/ARO-HCP/test/e2e/nodepool_ephemeral_osdisk.go:88]: failed to select a VM size with ephemeral OS disk support; this typically indicates a SKU restriction or quota issue in the test subscription/region
Unexpected error:
<*fmt.wrapError | 0xc002f205e0>:
selector "ephemeral-osdisk-worker" matched no usable VM size in eastus2: no unrestricted VM size satisfied the selector in the test location; this typically indicates a VM SKU restriction or quota issue with the test subscription/region, not a product defect
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_ephemeral_osdisk.go:88]: failed to select a VM size with ephemeral OS disk support; this typically indicates a SKU restriction or quota issue in the test subscription/region
Unexpected error:
<*fmt.wrapError | 0xc002f205e0>:
selector "ephemeral-osdisk-worker" matched no usable VM size in eastus2: no unrestricted VM size satisfied the selector in the test location; this typically indicates a VM SKU restriction or quota issue with the test subscription/region, not a product defect
...
occurredERROR CODE: DeadlineExceeded; provider Microsoft.Compute Show raw failurefail [github.com/Azure/ARO-HCP/test/e2e/cluster_delete_cx_rg.go:139]: failed to create node pools in parallel for cluster "cx-rg-hcp-cluster"
Unexpected error:
<*fmt.wrapError | 0xc00105d400>:
failed to create NodePool np-1: failed waiting for nodepool="np-1" for cluster "cx-rg-hcp-cluster" in resourcegroup="cx-rg-cluster-tqrndt5v6zjd" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2/hcpOperationStatuses/8281d8b1-ac73-4bd5-b7ff-5e1ee0313132
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: DeadlineExceeded
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2/hcpOperationStatuses/8281d8b1-ac73-4bd5-b7ff-5e1ee0313132",
"name": "8281d8b1-ac73-4bd5-b7ff-5e1ee0313132",
"status": "Failed",
"startTime": "2026-10-02T14:42:30.687332757Z",
"endTime": "2026-10-02T15:01:32.464297541Z",
"error": {
"code": "DeadlineExceeded",
"message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 1, want 2; node pool condition AllNodesHealthy is False: 1 of 2 machines are not healthy\nMachine x0e7s2g7c7i0z9g-np-1-cpzvz-fbtjm: InspectionFailed: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/cx-rg-cluster-tqrndt5v6zjd--managed/providers/Microsoft.Compute/virtualMachines/x0e7s2g7c7i0z9g-np-1-cpzvz-fbtjm to exist\n; node pool condition AllMachinesReady is False: 1 of 2 machines are not ready\nMachine x0e7s2g7c7i0z9g-np-1-cpzvz-fbtjm: ReadyUnknown: * NodeHealthy: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/cx-rg-cluster-tqrndt5v6zjd--managed/providers/Microsoft.Compute/virtualMachines/x0e7s2g7c7i0z9g-np-1-cpzvz-fbtjm to exist\n"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delete_cx_rg.go:139]: failed to create node pools in parallel for cluster "cx-rg-hcp-cluster"
Unexpected error:
<*fmt.wrapError | 0xc00105d400>:
failed to create NodePool np-1: failed waiting for nodepool="np-1" for cluster "cx-rg-hcp-cluster" in resourcegroup="cx-rg-cluster-tqrndt5v6zjd" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2/hcpOperationStatuses/8281d8b1-ac73-4bd5-b7ff-5e1ee0313132
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: DeadlineExceeded
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2/hcpOperationStatuses/8281d8b1-ac73-4bd5-b7ff-5e1ee0313132",
"name": "8281d8b1-ac73-4bd5-b7ff-5e1ee0313132",
"status": "Failed",
"startTime": "2026-10-02T14:42:30.687332757Z",
"endTime": "2026-10-02T15:01:32.464297541Z",
"error": {
"code": "DeadlineExceeded",
"message": "node pool creation did not complete before the deadline; [clusterServiceNodePoolStatus] cluster service node pool is installing; [hypershiftNodePool] hypershift NodePool status replicas is 1, want 2; node pool condition AllNodesHealthy is False: 1 of 2 machines are not healthy\nMachine x0e7s2g7c7i0z9g-np-1-cpzvz-fbtjm: InspectionFailed: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/cx-rg-cluster-tqrndt5v6zjd--managed/providers/Microsoft.Compute/virtualMachines/x0e7s2g7c7i0z9g-np-1-cpzvz-fbtjm to exist\n; node pool condition AllMachinesReady is False: 1 of 2 machines are not ready\nMachine x0e7s2g7c7i0z9g-np-1-cpzvz-fbtjm: ReadyUnknown: * NodeHealthy: Waiting for a Node with spec.providerID azure:///subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/cx-rg-cluster-tqrndt5v6zjd--managed/providers/Microsoft.Compute/virtualMachines/x0e7s2g7c7i0z9g-np-1-cpzvz-fbtjm to exist\n"
}
}
--------------------------------------------------------------------------------
...
occurred |
|
| 2h 55m 47s | branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel post-goodmerged PR |
eastus2 | e2e | failed | n/a | 1 | e2e (1)selector "<selector>" matched no usable VM size in eastus2: no unrestricted VM size satisfied the selector in the test location; this typically indicates a VM SKU restriction or quota issue with the test subscription/region, not a product defect Show raw failurefail [github.com/Azure/ARO-HCP/test/e2e/nodepool_ephemeral_osdisk.go:88]: failed to select a VM size with ephemeral OS disk support; this typically indicates a SKU restriction or quota issue in the test subscription/region
Unexpected error:
<*fmt.wrapError | 0xc0006eb260>:
selector "ephemeral-osdisk-worker" matched no usable VM size in eastus2: no unrestricted VM size satisfied the selector in the test location; this typically indicates a VM SKU restriction or quota issue with the test subscription/region, not a product defect
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_ephemeral_osdisk.go:88]: failed to select a VM size with ephemeral OS disk support; this typically indicates a SKU restriction or quota issue in the test subscription/region
Unexpected error:
<*fmt.wrapError | 0xc0006eb260>:
selector "ephemeral-osdisk-worker" matched no usable VM size in eastus2: no unrestricted VM size satisfied the selector in the test location; this typically indicates a VM SKU restriction or quota issue with the test subscription/region, not a product defect
...
occurred |
|
| 2h 19m 13s | periodic-ci-Azure-ARO-HCP-main-periodic-prod-e2e-parallel post-goodmerged PR |
australiaeast | e2e | failed | n/a | 5 | e2e (5)ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.z-stream-update-policy": unrecognized experimental tag; provider Microsoft.RedHatOpenShift Show raw failurefail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:157]: failed to enable immediate z-stream updates for cluster "cluster-zstream-4-22-fnp94g" on minor version 4.22
Unexpected error:
<*exported.ResponseError | 0xc0017b8120>:
PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-22-pqm748zt4v25/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-zstream-4-22-fnp94g
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"aro-hcp.experimental.cluster.z-stream-update-policy\": unrecognized experimental tag",
"target": "tags[aro-hcp.experimental.cluster.z-stream-update-policy]"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:157]: failed to enable immediate z-stream updates for cluster "cluster-zstream-4-22-fnp94g" on minor version 4.22
Unexpected error:
<*exported.ResponseError | 0xc0017b8120>:
PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-22-pqm748zt4v25/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-zstream-4-22-fnp94g
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"aro-hcp.experimental.cluster.z-stream-update-policy\": unrecognized experimental tag",
"target": "tags[aro-hcp.experimental.cluster.z-stream-update-policy]"
}
}
--------------------------------------------------------------------------------
...
occurredERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.z-stream-update-policy": unrecognized experimental tag; provider Microsoft.RedHatOpenShift Show raw failurefail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:157]: failed to enable immediate z-stream updates for cluster "cluster-zstream-4-21-phqn4q" on minor version 4.21
Unexpected error:
<*exported.ResponseError | 0xc0014ae240>:
PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-21-krv2vhb295c2/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-zstream-4-21-phqn4q
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"aro-hcp.experimental.cluster.z-stream-update-policy\": unrecognized experimental tag",
"target": "tags[aro-hcp.experimental.cluster.z-stream-update-policy]"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:157]: failed to enable immediate z-stream updates for cluster "cluster-zstream-4-21-phqn4q" on minor version 4.21
Unexpected error:
<*exported.ResponseError | 0xc0014ae240>:
PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-21-krv2vhb295c2/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-zstream-4-21-phqn4q
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"aro-hcp.experimental.cluster.z-stream-update-policy\": unrecognized experimental tag",
"target": "tags[aro-hcp.experimental.cluster.z-stream-update-policy]"
}
}
--------------------------------------------------------------------------------
...
occurredERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.z-stream-update-policy": unrecognized experimental tag; provider Microsoft.RedHatOpenShift Show raw failurefail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:157]: failed to enable immediate z-stream updates for cluster "cluster-zstream-5-0-zrd7b7" on minor version 5.0
Unexpected error:
<*exported.ResponseError | 0xc0010807b0>:
PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-5-0-pn6t6n4tjrff/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-zstream-5-0-zrd7b7
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"aro-hcp.experimental.cluster.z-stream-update-policy\": unrecognized experimental tag",
"target": "tags[aro-hcp.experimental.cluster.z-stream-update-policy]"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:157]: failed to enable immediate z-stream updates for cluster "cluster-zstream-5-0-zrd7b7" on minor version 5.0
Unexpected error:
<*exported.ResponseError | 0xc0010807b0>:
PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-5-0-pn6t6n4tjrff/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-zstream-5-0-zrd7b7
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"aro-hcp.experimental.cluster.z-stream-update-policy\": unrecognized experimental tag",
"target": "tags[aro-hcp.experimental.cluster.z-stream-update-policy]"
}
}
--------------------------------------------------------------------------------
...
occurredERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.z-stream-update-policy": unrecognized experimental tag; provider Microsoft.RedHatOpenShift Show raw failurefail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:157]: failed to enable immediate z-stream updates for cluster "cluster-zstream-4-20-9p29lj" on minor version 4.20
Unexpected error:
<*exported.ResponseError | 0xc00153a060>:
PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-20-l7kx94q698fp/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-zstream-4-20-9p29lj
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"aro-hcp.experimental.cluster.z-stream-update-policy\": unrecognized experimental tag",
"target": "tags[aro-hcp.experimental.cluster.z-stream-update-policy]"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:157]: failed to enable immediate z-stream updates for cluster "cluster-zstream-4-20-9p29lj" on minor version 4.20
Unexpected error:
<*exported.ResponseError | 0xc00153a060>:
PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-20-l7kx94q698fp/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-zstream-4-20-9p29lj
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"aro-hcp.experimental.cluster.z-stream-update-policy\": unrecognized experimental tag",
"target": "tags[aro-hcp.experimental.cluster.z-stream-update-policy]"
}
}
--------------------------------------------------------------------------------
...
occurredERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.z-stream-update-policy": unrecognized experimental tag; provider Microsoft.RedHatOpenShift Show raw failurefail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:157]: failed to enable immediate z-stream updates for cluster "cluster-zstream-4-23-xmqw4g" on minor version 4.23
Unexpected error:
<*exported.ResponseError | 0xc00155a450>:
PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-23-v87rt9mg748g/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-zstream-4-23-xmqw4g
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"aro-hcp.experimental.cluster.z-stream-update-policy\": unrecognized experimental tag",
"target": "tags[aro-hcp.experimental.cluster.z-stream-update-policy]"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:157]: failed to enable immediate z-stream updates for cluster "cluster-zstream-4-23-xmqw4g" on minor version 4.23
Unexpected error:
<*exported.ResponseError | 0xc00155a450>:
PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-zstream-upgrade-4-23-v87rt9mg748g/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-zstream-4-23-xmqw4g
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"aro-hcp.experimental.cluster.z-stream-update-policy\": unrecognized experimental tag",
"target": "tags[aro-hcp.experimental.cluster.z-stream-update-policy]"
}
}
--------------------------------------------------------------------------------
...
occurred |