No failure patterns match your search.
Environment: PROD
Failure patterns: 15 · Failures matched: 131 · Total runs: 37 · Runs affected: 19
| Failure patternThe canonical description of a recurring CI failure, extracted and normalized from raw logs. | Failed atThe stage of the job run where this failure occurred: 'provision' (environment setup, DEV only), 'e2e' (test suite execution), or 'other' (CI infrastructure issues that did not produce a failure pattern). | Number of distinct job runs where this failure pattern was detected. | Percentage of all job runs in this environment affected by this failure pattern during the selected window. | Classification of this failure pattern: Regression (likely caused by a specific PR), Flake (intermittent failure spread across days), Noise (low-quality or generic pattern), or Indeterminate. | TrendShows daily activity for this failure pattern in a trailing window anchored to the selected end date. The sparkline covers at least 7 days and at most 14 days, depending on the current window size. | Also inOther environments where the same failure pattern was also detected during the selected window. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
ERROR CODE: InternalServerError; detail message node pool creation did not complete before the deadline; provider Microsoft.RedHatOpenShiftfull failure pattern: ERROR CODE: InternalServerError; detail message node pool creation did not complete before the deadline; provider Microsoft.RedHatOpenShift Signal: IndeterminateSignal: Indeterminate — no prior history; active 5 day(s) — New failure pattern — no prior history | e2e | 9 | 24.32%9 of 37 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 5 day(s) — New failure pattern — no prior history | Show trend detailsSep 13: 0 · Sep 14: 0 · Sep 15: 0 · Sep 16: 1 · Sep 17: 5 · Sep 18: 5 · Sep 19: 1 · Sep 20: 3 | none | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (3)fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:403]: failed to verify cilium is running and nodes are Ready for cluster "agg-cluster"
Unexpected error:
<*errors.joinError | 0xc000ed0f48>:
failed to create NodePool agg-np-a: failed waiting for nodepool="agg-np-a" for cluster "agg-cluster" in resourcegroup="feature-aggregation-75p2xk77rtzr" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/65cff58d-536e-4d46-b35b-58f67af72b80
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/65cff58d-536e-4d46-b35b-58f67af72b80",
"name": "65cff58d-536e-4d46-b35b-58f67af72b80",
"status": "Failed",
"startTime": "2026-09-20T02:16:51.798119339Z",
"endTime": "2026-09-20T02:35:59.041922136Z",
"error": {
"code": "InternalServerError",
"message": "node pool creation did not complete before the deadline"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:403]: failed to verify cilium is running and nodes are Ready for cluster "agg-cluster"
Unexpected error:
<*errors.joinError | 0xc000ed0f48>:
failed to create NodePool agg-np-a: failed waiting for nodepool="agg-np-a" for cluster "agg-cluster" in resourcegroup="feature-aggregation-75p2xk77rtzr" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/65cff58d-536e-4d46-b35b-58f67af72b80
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/65cff58d-536e-4d46-b35b-58f67af72b80",
"name": "65cff58d-536e-4d46-b35b-58f67af72b80",
"status": "Failed",
"startTime": "2026-09-20T02:16:51.798119339Z",
"endTime": "2026-09-20T02:35:59.041922136Z",
"error": {
"code": "InternalServerError",
"message": "node pool creation did not complete before the deadline"
}
}
--------------------------------------------------------------------------------
...
occurredfail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:459]: failed to create nodepool nps-420-422 at version 4.20.38
Unexpected error:
<*fmt.wrapError | 0xc00118b8a0>:
failed to create NodePool nps-420-422: failed waiting for nodepool="nps-420-422" for cluster "np-skip-minor-7sn2sc" in resourcegroup="rg-np-skip-minor-7sn2sc-x6d7knwq47tj" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0555a946-aa1d-468d-a6ad-48364e5f992e
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0555a946-aa1d-468d-a6ad-48364e5f992e",
"name": "0555a946-aa1d-468d-a6ad-48364e5f992e",
"status": "Failed",
"startTime": "2026-09-20T03:23:26.91112939Z",
"endTime": "2026-09-20T03:42:27.966002583Z",
"error": {
"code": "InternalServerError",
"message": "node pool creation did not complete before the deadline"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:459]: failed to create nodepool nps-420-422 at version 4.20.38
Unexpected error:
<*fmt.wrapError | 0xc00118b8a0>:
failed to create NodePool nps-420-422: failed waiting for nodepool="nps-420-422" for cluster "np-skip-minor-7sn2sc" in resourcegroup="rg-np-skip-minor-7sn2sc-x6d7knwq47tj" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0555a946-aa1d-468d-a6ad-48364e5f992e
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/0555a946-aa1d-468d-a6ad-48364e5f992e",
"name": "0555a946-aa1d-468d-a6ad-48364e5f992e",
"status": "Failed",
"startTime": "2026-09-20T03:23:26.91112939Z",
"endTime": "2026-09-20T03:42:27.966002583Z",
"error": {
"code": "InternalServerError",
"message": "node pool creation did not complete before the deadline"
}
}
--------------------------------------------------------------------------------
...
occurredfail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:220]: failed to create no-AZ nodepool autoscale-noaz with autoscaling
Unexpected error:
<*fmt.wrapError | 0xc00011a240>:
failed to create NodePool autoscale-noaz: failed waiting for nodepool="autoscale-noaz" for cluster "np-autoscale-cluster" in resourcegroup="np-autoscaling-h72vxbzhd2w4" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b5017a36-2ffb-4b1f-9f5e-28e6a6204319
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b5017a36-2ffb-4b1f-9f5e-28e6a6204319",
"name": "b5017a36-2ffb-4b1f-9f5e-28e6a6204319",
"status": "Failed",
"startTime": "2026-09-20T03:10:04.541118917Z",
"endTime": "2026-09-20T03:29:07.643791064Z",
"error": {
"code": "InternalServerError",
"message": "node pool creation did not complete before the deadline"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:220]: failed to create no-AZ nodepool autoscale-noaz with autoscaling
Unexpected error:
<*fmt.wrapError | 0xc00011a240>:
failed to create NodePool autoscale-noaz: failed waiting for nodepool="autoscale-noaz" for cluster "np-autoscale-cluster" in resourcegroup="np-autoscaling-h72vxbzhd2w4" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b5017a36-2ffb-4b1f-9f5e-28e6a6204319
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/b5017a36-2ffb-4b1f-9f5e-28e6a6204319",
"name": "b5017a36-2ffb-4b1f-9f5e-28e6a6204319",
"status": "Failed",
"startTime": "2026-09-20T03:10:04.541118917Z",
"endTime": "2026-09-20T03:29:07.643791064Z",
"error": {
"code": "InternalServerError",
"message": "node pool creation did not complete before the deadline"
}
}
--------------------------------------------------------------------------------
...
occurredContributing tests (14)
Affected runs (9)
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
ERROR CODE: SubscriptionNotRegistered; provider Microsoft.Insightsfull failure pattern: ERROR CODE: SubscriptionNotRegistered; provider Microsoft.Insights Signal: IndeterminateSignal: Indeterminate — no prior history; active 2 day(s) — New failure pattern — no prior history | e2e | 6 | 16.22%6 of 37 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 2 day(s) — New failure pattern — no prior history | Show trend detailsSep 13: 0 · Sep 14: 0 · Sep 15: 0 · Sep 16: 0 · Sep 17: 1 · Sep 18: 5 · Sep 19: 0 · Sep 20: 0 | none | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (3)fail [github.com/Azure/ARO-HCP/test/e2e/shoebox_logs.go:319]: failed to create storage account diagnostic setting for cluster /subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/shoebox-cluster-d9mbtgrlrxw2/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/shoebox-hcp-cluster
Unexpected error:
<*exported.ResponseError | 0xc000a45890>:
PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/shoebox-cluster-d9mbtgrlrxw2/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/shoebox-hcp-cluster/providers/Microsoft.Insights/diagnosticSettings/shoebox-storage-diag
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: SubscriptionNotRegistered
--------------------------------------------------------------------------------
{
"code": "SubscriptionNotRegistered",
"message": "The subscription 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX' is not registered to use microsoft.insights."
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/shoebox_logs.go:319]: failed to create storage account diagnostic setting for cluster /subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/shoebox-cluster-d9mbtgrlrxw2/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/shoebox-hcp-cluster
Unexpected error:
<*exported.ResponseError | 0xc000a45890>:
PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/shoebox-cluster-d9mbtgrlrxw2/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/shoebox-hcp-cluster/providers/Microsoft.Insights/diagnosticSettings/shoebox-storage-diag
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: SubscriptionNotRegistered
--------------------------------------------------------------------------------
{
"code": "SubscriptionNotRegistered",
"message": "The subscription 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX' is not registered to use microsoft.insights."
}
--------------------------------------------------------------------------------
...
occurredfail [github.com/Azure/ARO-HCP/test/e2e/shoebox_logs.go:319]: failed to create storage account diagnostic setting for cluster /subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/shoebox-cluster-986dm5j578hd/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/shoebox-hcp-cluster
Unexpected error:
<*exported.ResponseError | 0xc00117a2a0>:
PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/shoebox-cluster-986dm5j578hd/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/shoebox-hcp-cluster/providers/Microsoft.Insights/diagnosticSettings/shoebox-storage-diag
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: SubscriptionNotRegistered
--------------------------------------------------------------------------------
{
"code": "SubscriptionNotRegistered",
"message": "The subscription 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX' is not registered to use microsoft.insights."
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/shoebox_logs.go:319]: failed to create storage account diagnostic setting for cluster /subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/shoebox-cluster-986dm5j578hd/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/shoebox-hcp-cluster
Unexpected error:
<*exported.ResponseError | 0xc00117a2a0>:
PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/shoebox-cluster-986dm5j578hd/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/shoebox-hcp-cluster/providers/Microsoft.Insights/diagnosticSettings/shoebox-storage-diag
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: SubscriptionNotRegistered
--------------------------------------------------------------------------------
{
"code": "SubscriptionNotRegistered",
"message": "The subscription 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX' is not registered to use microsoft.insights."
}
--------------------------------------------------------------------------------
...
occurredfail [github.com/Azure/ARO-HCP/test/e2e/shoebox_logs.go:319]: failed to create storage account diagnostic setting for cluster /subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/shoebox-cluster-92rgrk84fv8d/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/shoebox-hcp-cluster
Unexpected error:
<*exported.ResponseError | 0xc00088e570>:
PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/shoebox-cluster-92rgrk84fv8d/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/shoebox-hcp-cluster/providers/Microsoft.Insights/diagnosticSettings/shoebox-storage-diag
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: SubscriptionNotRegistered
--------------------------------------------------------------------------------
{
"code": "SubscriptionNotRegistered",
"message": "The subscription 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX' is not registered to use microsoft.insights."
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/shoebox_logs.go:319]: failed to create storage account diagnostic setting for cluster /subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/shoebox-cluster-92rgrk84fv8d/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/shoebox-hcp-cluster
Unexpected error:
<*exported.ResponseError | 0xc00088e570>:
PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/shoebox-cluster-92rgrk84fv8d/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/shoebox-hcp-cluster/providers/Microsoft.Insights/diagnosticSettings/shoebox-storage-diag
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: SubscriptionNotRegistered
--------------------------------------------------------------------------------
{
"code": "SubscriptionNotRegistered",
"message": "The subscription 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX' is not registered to use microsoft.insights."
}
--------------------------------------------------------------------------------
...
occurredContributing tests (3)
Affected runs (6)
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
ERROR CODE: InvalidRequestContent; detail message Invalid value: "4.21": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when c...full failure pattern: ERROR CODE: InvalidRequestContent; detail message Invalid value: "4.21": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is "nightly"; provider Microsoft.RedHatOpenShift Signal: IndeterminateSignal: Indeterminate — no prior history; active 5 day(s) — New failure pattern — no prior history | e2e | 5 | 13.51%5 of 37 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 5 day(s) — New failure pattern — no prior history | Show trend detailsSep 13: 0 · Sep 14: 0 · Sep 15: 0 · Sep 16: 5 · Sep 17: 4 · Sep 18: 5 · Sep 19: 5 · Sep 20: 5 | none | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (3)fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:274]: failed to begin cluster "agg-cluster" authorized CIDR update
Unexpected error:
<*exported.ResponseError | 0xc001896cf0>:
PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/feature-aggregation-79mxdqgdx9t6/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/agg-cluster
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"4.21\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"",
"target": "properties.version.id"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:274]: failed to begin cluster "agg-cluster" authorized CIDR update
Unexpected error:
<*exported.ResponseError | 0xc001896cf0>:
PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/feature-aggregation-79mxdqgdx9t6/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/agg-cluster
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"4.21\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"",
"target": "properties.version.id"
}
}
--------------------------------------------------------------------------------
...
occurredfail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:173]: failed to update cluster "autoscaling-hcp-cluster" autoscaling configuration
Unexpected error:
<*exported.ResponseError | 0xc00286bb60>:
PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/autoscaling-cluster-r4zrv9dgbv7h/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/autoscaling-hcp-cluster
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"4.21\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"",
"target": "properties.version.id"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:173]: failed to update cluster "autoscaling-hcp-cluster" autoscaling configuration
Unexpected error:
<*exported.ResponseError | 0xc00286bb60>:
PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/autoscaling-cluster-r4zrv9dgbv7h/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/autoscaling-hcp-cluster
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"4.21\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"",
"target": "properties.version.id"
}
}
--------------------------------------------------------------------------------
...
occurredfail [github.com/Azure/ARO-HCP/test/e2e/idms_lifecycle.go:175]: failed to update cluster with second ImageDigestMirror set
Unexpected error:
<*exported.ResponseError | 0xc0016fa690>:
PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/idms-gc2rhgk55bsk/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/idms-e2e-hcp-cluster
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"4.21\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"",
"target": "properties.version.id"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/idms_lifecycle.go:175]: failed to update cluster with second ImageDigestMirror set
Unexpected error:
<*exported.ResponseError | 0xc0016fa690>:
PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/idms-gc2rhgk55bsk/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/idms-e2e-hcp-cluster
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"4.21\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"",
"target": "properties.version.id"
}
}
--------------------------------------------------------------------------------
...
occurredContributing tests (5)
Affected runs (5)
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
ERROR CODE: InvalidRequestContent; detail message Invalid value: "4.22": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when c...full failure pattern: ERROR CODE: InvalidRequestContent; detail message Invalid value: "4.22": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is "nightly"; provider Microsoft.RedHatOpenShift Signal: IndeterminateSignal: Indeterminate — no prior history; active 5 day(s) — New failure pattern — no prior history | e2e | 5 | 13.51%5 of 37 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 5 day(s) — New failure pattern — no prior history | Show trend detailsSep 13: 0 · Sep 14: 0 · Sep 15: 0 · Sep 16: 1 · Sep 17: 1 · Sep 18: 1 · Sep 19: 1 · Sep 20: 1 | none | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (3)fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:90]: failed to create HCP cluster "private-kas" with private KAS
Unexpected error:
<*fmt.wrapError | 0xc0009e7440>:
failed to create HCP cluster private-kas: failed starting cluster creation "private-kas" in resourcegroup="private-kas-mtt578qwjxhd": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/private-kas-mtt578qwjxhd/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/private-kas
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"4.22\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"",
"target": "properties.version.id"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:90]: failed to create HCP cluster "private-kas" with private KAS
Unexpected error:
<*fmt.wrapError | 0xc0009e7440>:
failed to create HCP cluster private-kas: failed starting cluster creation "private-kas" in resourcegroup="private-kas-mtt578qwjxhd": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/private-kas-mtt578qwjxhd/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/private-kas
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"4.22\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"",
"target": "properties.version.id"
}
}
--------------------------------------------------------------------------------
...
occurredfail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:90]: failed to create HCP cluster "private-kas" with private KAS
Unexpected error:
<*fmt.wrapError | 0xc00106b280>:
failed to create HCP cluster private-kas: failed starting cluster creation "private-kas" in resourcegroup="private-kas-mbf8fbn585g6": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/private-kas-mbf8fbn585g6/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/private-kas
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"4.22\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"",
"target": "properties.version.id"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:90]: failed to create HCP cluster "private-kas" with private KAS
Unexpected error:
<*fmt.wrapError | 0xc00106b280>:
failed to create HCP cluster private-kas: failed starting cluster creation "private-kas" in resourcegroup="private-kas-mbf8fbn585g6": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/private-kas-mbf8fbn585g6/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/private-kas
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"4.22\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"",
"target": "properties.version.id"
}
}
--------------------------------------------------------------------------------
...
occurredfail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:90]: failed to create HCP cluster "private-kas" with private KAS
Unexpected error:
<*fmt.wrapError | 0xc00298e7c0>:
failed to create HCP cluster private-kas: failed starting cluster creation "private-kas" in resourcegroup="private-kas-9f9tbz46mct7": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/private-kas-9f9tbz46mct7/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/private-kas
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"4.22\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"",
"target": "properties.version.id"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kas.go:90]: failed to create HCP cluster "private-kas" with private KAS
Unexpected error:
<*fmt.wrapError | 0xc00298e7c0>:
failed to create HCP cluster private-kas: failed starting cluster creation "private-kas" in resourcegroup="private-kas-9f9tbz46mct7": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/private-kas-9f9tbz46mct7/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/private-kas
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"4.22\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"",
"target": "properties.version.id"
}
}
--------------------------------------------------------------------------------
...
occurredContributing tests (1)
Affected runs (5)
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
VerifyKubeAPIServerServerVersionUpgraded failed: kube-apiserver ServerVersion not updated (unchanged from pre-upgrade)full failure pattern: VerifyKubeAPIServerServerVersionUpgraded failed: kube-apiserver ServerVersion not updated (unchanged from pre-upgrade) Signal: IndeterminateSignal: Indeterminate — no prior history; active 5 day(s) — New failure pattern — no prior history | e2e | 5 | 13.51%5 of 37 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 5 day(s) — New failure pattern — no prior history | Show trend detailsSep 13: 0 · Sep 14: 0 · Sep 15: 0 · Sep 16: 1 · Sep 17: 1 · Sep 18: 1 · Sep 19: 1 · Sep 20: 1 | none | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (3)fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:176]: Timed out after 2700.002s.
Expected success, but got an error:
<*errors.joinError | 0xc0018004e0>:
VerifyKubeAPIServerServerVersionUpgraded failed: kube-apiserver ServerVersion not updated (unchanged from pre-upgrade)
VerifyHostedControlPlaneYStreamUpgrade(previousMinor=5.0.0-0.nightly-multi-2026-09-16-151028, targetMinor=5.1.0-0.nightly-multi-2026-09-15-135250) failed: clusterversion status.history has no version in target minor "5.1.0-0.nightly-multi-2026-09-15-135250"
...
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:176]: Timed out after 2700.002s.
Expected success, but got an error:
<*errors.joinError | 0xc0018004e0>:
VerifyKubeAPIServerServerVersionUpgraded failed: kube-apiserver ServerVersion not updated (unchanged from pre-upgrade)
VerifyHostedControlPlaneYStreamUpgrade(previousMinor=5.0.0-0.nightly-multi-2026-09-16-151028, targetMinor=5.1.0-0.nightly-multi-2026-09-15-135250) failed: clusterversion status.history has no version in target minor "5.1.0-0.nightly-multi-2026-09-15-135250"
...fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:176]: Timed out after 2700.002s.
Expected success, but got an error:
<*errors.joinError | 0xc000ac15d8>:
VerifyKubeAPIServerServerVersionUpgraded failed: kube-apiserver ServerVersion not updated (unchanged from pre-upgrade)
VerifyHostedControlPlaneYStreamUpgrade(previousMinor=5.0.0-0.nightly-multi-2026-09-16-151028, targetMinor=5.1.0-0.nightly-multi-2026-09-15-135250) failed: clusterversion status.history has no version in target minor "5.1.0-0.nightly-multi-2026-09-15-135250"
...
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:176]: Timed out after 2700.002s.
Expected success, but got an error:
<*errors.joinError | 0xc000ac15d8>:
VerifyKubeAPIServerServerVersionUpgraded failed: kube-apiserver ServerVersion not updated (unchanged from pre-upgrade)
VerifyHostedControlPlaneYStreamUpgrade(previousMinor=5.0.0-0.nightly-multi-2026-09-16-151028, targetMinor=5.1.0-0.nightly-multi-2026-09-15-135250) failed: clusterversion status.history has no version in target minor "5.1.0-0.nightly-multi-2026-09-15-135250"
...fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:176]: Timed out after 2700.002s.
Expected success, but got an error:
<*errors.joinError | 0xc00055c870>:
VerifyKubeAPIServerServerVersionUpgraded failed: kube-apiserver ServerVersion not updated (unchanged from pre-upgrade)
VerifyHostedControlPlaneYStreamUpgrade(previousMinor=5.0.0-0.nightly-multi-2026-09-16-151028, targetMinor=5.1.0-0.nightly-multi-2026-09-15-135250) failed: clusterversion status.history has no version in target minor "5.1.0-0.nightly-multi-2026-09-15-135250"
...
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:176]: Timed out after 2700.002s.
Expected success, but got an error:
<*errors.joinError | 0xc00055c870>:
VerifyKubeAPIServerServerVersionUpgraded failed: kube-apiserver ServerVersion not updated (unchanged from pre-upgrade)
VerifyHostedControlPlaneYStreamUpgrade(previousMinor=5.0.0-0.nightly-multi-2026-09-16-151028, targetMinor=5.1.0-0.nightly-multi-2026-09-15-135250) failed: clusterversion status.history has no version in target minor "5.1.0-0.nightly-multi-2026-09-15-135250"
...Contributing tests (1)
Affected runs (5)
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
ERROR CODE: InvalidRequestContent; detail message Invalid value: "<ocp-nightly>": must be at least 4.20.8; provider Microsoft.RedHatOpenShiftfull failure pattern: ERROR CODE: InvalidRequestContent; detail message Invalid value: "<ocp-nightly>": must be at least 4.20.8; provider Microsoft.RedHatOpenShift Signal: IndeterminateSignal: Indeterminate — no prior history; active 4 day(s) — New failure pattern — no prior history | e2e | 4 | 10.81%4 of 37 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 4 day(s) — New failure pattern — no prior history | Show trend detailsSep 13: 0 · Sep 14: 0 · Sep 15: 0 · Sep 16: 0 · Sep 17: 4 · Sep 18: 4 · Sep 19: 4 · Sep 20: 4 | none | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (3)fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:165]: failed to create node pool npupgrade-4-20 with version 4.20.0-0.nightly-multi-2026-09-16-210648
Unexpected error:
<*fmt.wrapError | 0xc000d12020>:
failed to create NodePool npupgrade-4-20: failed starting nodepool creation "npupgrade-4-20" for cluster "np-version-upgrade-cluster-7hkgvb" in resourcegroup="rg-np-version-upgrade-7hkgvb-db7f9x6gvf59": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-version-upgrade-7hkgvb-db7f9x6gvf59/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-version-upgrade-cluster-7hkgvb/nodePools/npupgrade-4-20
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"4.20.0-0.nightly-multi-2026-09-16-210648\": must be at least 4.20.8",
"target": "properties.version.id"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:165]: failed to create node pool npupgrade-4-20 with version 4.20.0-0.nightly-multi-2026-09-16-210648
Unexpected error:
<*fmt.wrapError | 0xc000d12020>:
failed to create NodePool npupgrade-4-20: failed starting nodepool creation "npupgrade-4-20" for cluster "np-version-upgrade-cluster-7hkgvb" in resourcegroup="rg-np-version-upgrade-7hkgvb-db7f9x6gvf59": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-version-upgrade-7hkgvb-db7f9x6gvf59/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-version-upgrade-cluster-7hkgvb/nodePools/npupgrade-4-20
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"4.20.0-0.nightly-multi-2026-09-16-210648\": must be at least 4.20.8",
"target": "properties.version.id"
}
}
--------------------------------------------------------------------------------
...
occurredfail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:175]: failed to create node pool "np-dr4c56" for cluster "cluster-nightly-4-20-dr4c56"
Unexpected error:
<*fmt.wrapError | 0xc000d101a0>:
failed to create NodePool np-dr4c56: failed starting nodepool creation "np-dr4c56" for cluster "cluster-nightly-4-20-dr4c56" in resourcegroup="rg-nightly-4-20-xfvxqd785djj": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-nightly-4-20-xfvxqd785djj/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-nightly-4-20-dr4c56/nodePools/np-dr4c56
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"4.20.0-0.nightly-multi-2026-09-16-210648\": must be at least 4.20.8",
"target": "properties.version.id"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:175]: failed to create node pool "np-dr4c56" for cluster "cluster-nightly-4-20-dr4c56"
Unexpected error:
<*fmt.wrapError | 0xc000d101a0>:
failed to create NodePool np-dr4c56: failed starting nodepool creation "np-dr4c56" for cluster "cluster-nightly-4-20-dr4c56" in resourcegroup="rg-nightly-4-20-xfvxqd785djj": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-nightly-4-20-xfvxqd785djj/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-nightly-4-20-dr4c56/nodePools/np-dr4c56
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"4.20.0-0.nightly-multi-2026-09-16-210648\": must be at least 4.20.8",
"target": "properties.version.id"
}
}
--------------------------------------------------------------------------------
...
occurredfail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:766]: failed to update nodepool npdg-422-420 to downgrade target 4.20.0-0.nightly-multi-2026-09-16-210648
Unexpected error:
<*fmt.wrapError | 0xc001070ec0>:
failed to start nodepool "npdg-422-420" update in cluster "np-dg-minor-zrpt5w" resourcegroup="rg-np-dg-minor-zrpt5w-4m5b8rktsw6x": PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-dg-minor-zrpt5w-4m5b8rktsw6x/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-dg-minor-zrpt5w/nodePools/npdg-422-420
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"4.20.0-0.nightly-multi-2026-09-16-210648\": must be at least 4.20.8",
"target": "properties.version.id"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:766]: failed to update nodepool npdg-422-420 to downgrade target 4.20.0-0.nightly-multi-2026-09-16-210648
Unexpected error:
<*fmt.wrapError | 0xc001070ec0>:
failed to start nodepool "npdg-422-420" update in cluster "np-dg-minor-zrpt5w" resourcegroup="rg-np-dg-minor-zrpt5w-4m5b8rktsw6x": PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-dg-minor-zrpt5w-4m5b8rktsw6x/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-dg-minor-zrpt5w/nodePools/npdg-422-420
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"4.20.0-0.nightly-multi-2026-09-16-210648\": must be at least 4.20.8",
"target": "properties.version.id"
}
}
--------------------------------------------------------------------------------
...
occurredContributing tests (4)
Affected runs (4)
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
ERROR CODE: InternalServerError; detail message [hypershiftHostedCluster] hosted cluster is not available: KubeconfigWaitingForCreate: Waiting for hosted control plane kubeconfig ...full failure pattern: ERROR CODE: InternalServerError; detail message [hypershiftHostedCluster] hosted cluster is not available: KubeconfigWaitingForCreate: Waiting for hosted control plane kubeconfig to be created; hosted cluster degraded: UnavailableReplicas: [capi-provider]; provider Microsoft.RedHatOpenShift Signal: IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | e2e | 2 | 5.41%2 of 37 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | Show trend detailsSep 13: 0 · Sep 14: 0 · Sep 15: 0 · Sep 16: 0 · Sep 17: 0 · Sep 18: 2 · Sep 19: 0 · Sep 20: 0 | none | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (2)fail [github.com/Azure/ARO-HCP/test/e2e/cluster_ingress_cert_letsencrypt.go:123]: failed to create HCP cluster
Unexpected error:
<*fmt.wrapError | 0xc0014ced60>:
failed to create HCP cluster ingress-cert-le: failed waiting for cluster="ingress-cert-le" in resourcegroup="ingress-cert-le-wvt6tvvldjdg" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2/hcpOperationStatuses/80eaa57a-6c21-4104-9379-5f7032bfe26e
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2/hcpOperationStatuses/80eaa57a-6c21-4104-9379-5f7032bfe26e",
"name": "80eaa57a-6c21-4104-9379-5f7032bfe26e",
"status": "Failed",
"startTime": "2026-09-18T18:06:15.268087105Z",
"endTime": "2026-09-18T18:35:15.027123676Z",
"error": {
"code": "InternalServerError",
"message": "[hypershiftHostedCluster] hosted cluster is not available: KubeconfigWaitingForCreate: Waiting for hosted control plane kubeconfig to be created; hosted cluster degraded: UnavailableReplicas: capi-provider deployment has 1 unavailable replicas"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_ingress_cert_letsencrypt.go:123]: failed to create HCP cluster
Unexpected error:
<*fmt.wrapError | 0xc0014ced60>:
failed to create HCP cluster ingress-cert-le: failed waiting for cluster="ingress-cert-le" in resourcegroup="ingress-cert-le-wvt6tvvldjdg" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2/hcpOperationStatuses/80eaa57a-6c21-4104-9379-5f7032bfe26e
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/eastus2/hcpOperationStatuses/80eaa57a-6c21-4104-9379-5f7032bfe26e",
"name": "80eaa57a-6c21-4104-9379-5f7032bfe26e",
"status": "Failed",
"startTime": "2026-09-18T18:06:15.268087105Z",
"endTime": "2026-09-18T18:35:15.027123676Z",
"error": {
"code": "InternalServerError",
"message": "[hypershiftHostedCluster] hosted cluster is not available: KubeconfigWaitingForCreate: Waiting for hosted control plane kubeconfig to be created; hosted cluster degraded: UnavailableReplicas: capi-provider deployment has 1 unavailable replicas"
}
}
--------------------------------------------------------------------------------
...
occurredfail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:108]: failed to create HCP cluster "autoscaling-hcp-cluster" with custom autoscaling
Unexpected error:
<*fmt.wrapError | 0xc000860220>:
failed to create HCP cluster autoscaling-hcp-cluster: failed waiting for cluster="autoscaling-hcp-cluster" in resourcegroup="autoscaling-cluster-rm9vjp5gmczk" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/df01aa2b-a5c5-4899-b645-5ce58f5f8cee
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/df01aa2b-a5c5-4899-b645-5ce58f5f8cee",
"name": "df01aa2b-a5c5-4899-b645-5ce58f5f8cee",
"status": "Failed",
"startTime": "2026-09-18T02:08:07.344249043Z",
"endTime": "2026-09-18T02:37:10.01628355Z",
"error": {
"code": "InternalServerError",
"message": "[hypershiftHostedCluster] hosted cluster is not available: KubeconfigWaitingForCreate: Waiting for hosted control plane kubeconfig to be created; hosted cluster degraded: UnavailableReplicas: capi-provider deployment has 1 unavailable replicas"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:108]: failed to create HCP cluster "autoscaling-hcp-cluster" with custom autoscaling
Unexpected error:
<*fmt.wrapError | 0xc000860220>:
failed to create HCP cluster autoscaling-hcp-cluster: failed waiting for cluster="autoscaling-hcp-cluster" in resourcegroup="autoscaling-cluster-rm9vjp5gmczk" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/df01aa2b-a5c5-4899-b645-5ce58f5f8cee
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/df01aa2b-a5c5-4899-b645-5ce58f5f8cee",
"name": "df01aa2b-a5c5-4899-b645-5ce58f5f8cee",
"status": "Failed",
"startTime": "2026-09-18T02:08:07.344249043Z",
"endTime": "2026-09-18T02:37:10.01628355Z",
"error": {
"code": "InternalServerError",
"message": "[hypershiftHostedCluster] hosted cluster is not available: KubeconfigWaitingForCreate: Waiting for hosted control plane kubeconfig to be created; hosted cluster degraded: UnavailableReplicas: capi-provider deployment has 1 unavailable replicas"
}
}
--------------------------------------------------------------------------------
...
occurredContributing tests (2)
Affected runs (2)
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
ERROR CODE: InternalServerError; detail message cluster placement did not complete before the deadline; provider Microsoft.RedHatOpenShiftfull failure pattern: ERROR CODE: InternalServerError; detail message cluster placement did not complete before the deadline; provider Microsoft.RedHatOpenShift Signal: IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | e2e | 1 | 2.70%1 of 37 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | Show trend detailsSep 13: 0 · Sep 14: 0 · Sep 15: 0 · Sep 16: 0 · Sep 17: 0 · Sep 18: 51 · Sep 19: 0 · Sep 20: 0 | none | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (3)fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:146]: failed to create HCP cluster np-version-upgrade-cluster-n6q68g with version 4.21.34
Unexpected error:
<*fmt.wrapError | 0xc000be8cc0>:
failed to create HCP cluster np-version-upgrade-cluster-n6q68g: failed waiting for cluster="np-version-upgrade-cluster-n6q68g" in resourcegroup="rg-np-version-upgrade-n6q68g-z5wvsgjdr7l9" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/47d83203-d5e7-4af0-adbd-1321b73b5d96
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/47d83203-d5e7-4af0-adbd-1321b73b5d96",
"name": "47d83203-d5e7-4af0-adbd-1321b73b5d96",
"status": "Failed",
"startTime": "2026-09-18T19:55:24.695867176Z",
"endTime": "2026-09-18T20:24:27.02816156Z",
"error": {
"code": "InternalServerError",
"message": "cluster placement did not complete before the deadline"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:146]: failed to create HCP cluster np-version-upgrade-cluster-n6q68g with version 4.21.34
Unexpected error:
<*fmt.wrapError | 0xc000be8cc0>:
failed to create HCP cluster np-version-upgrade-cluster-n6q68g: failed waiting for cluster="np-version-upgrade-cluster-n6q68g" in resourcegroup="rg-np-version-upgrade-n6q68g-z5wvsgjdr7l9" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/47d83203-d5e7-4af0-adbd-1321b73b5d96
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/47d83203-d5e7-4af0-adbd-1321b73b5d96",
"name": "47d83203-d5e7-4af0-adbd-1321b73b5d96",
"status": "Failed",
"startTime": "2026-09-18T19:55:24.695867176Z",
"endTime": "2026-09-18T20:24:27.02816156Z",
"error": {
"code": "InternalServerError",
"message": "cluster placement did not complete before the deadline"
}
}
--------------------------------------------------------------------------------
...
occurredfail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:102]: failed to create HCP cluster np-autoscale-cluster
Unexpected error:
<*fmt.wrapError | 0xc000de50e0>:
failed to create HCP cluster np-autoscale-cluster: failed waiting for cluster="np-autoscale-cluster" in resourcegroup="np-autoscaling-bddwvtnwz8x8" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/1de5df02-d5bc-4ea1-8f35-f119420c215b
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/1de5df02-d5bc-4ea1-8f35-f119420c215b",
"name": "1de5df02-d5bc-4ea1-8f35-f119420c215b",
"status": "Failed",
"startTime": "2026-09-18T19:54:45.082839017Z",
"endTime": "2026-09-18T20:23:47.024709747Z",
"error": {
"code": "InternalServerError",
"message": "cluster placement did not complete before the deadline"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:102]: failed to create HCP cluster np-autoscale-cluster
Unexpected error:
<*fmt.wrapError | 0xc000de50e0>:
failed to create HCP cluster np-autoscale-cluster: failed waiting for cluster="np-autoscale-cluster" in resourcegroup="np-autoscaling-bddwvtnwz8x8" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/1de5df02-d5bc-4ea1-8f35-f119420c215b
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/1de5df02-d5bc-4ea1-8f35-f119420c215b",
"name": "1de5df02-d5bc-4ea1-8f35-f119420c215b",
"status": "Failed",
"startTime": "2026-09-18T19:54:45.082839017Z",
"endTime": "2026-09-18T20:23:47.024709747Z",
"error": {
"code": "InternalServerError",
"message": "cluster placement did not complete before the deadline"
}
}
--------------------------------------------------------------------------------
...
occurredfail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:98]: failed to create HCP cluster "private-kv-cluster" with private keyvault
Unexpected error:
<*fmt.wrapError | 0xc0010085a0>:
failed waiting for cluster="private-kv-cluster" in resourcegroup="private-keyvault-266qhn49mfp5" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/724dbb1d-6cfa-4cff-8bf4-5e1e742772df
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/724dbb1d-6cfa-4cff-8bf4-5e1e742772df",
"name": "724dbb1d-6cfa-4cff-8bf4-5e1e742772df",
"status": "Failed",
"startTime": "2026-09-18T18:06:21.25552087Z",
"endTime": "2026-09-18T18:35:23.923631594Z",
"error": {
"code": "InternalServerError",
"message": "cluster placement did not complete before the deadline"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:98]: failed to create HCP cluster "private-kv-cluster" with private keyvault
Unexpected error:
<*fmt.wrapError | 0xc0010085a0>:
failed waiting for cluster="private-kv-cluster" in resourcegroup="private-keyvault-266qhn49mfp5" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/724dbb1d-6cfa-4cff-8bf4-5e1e742772df
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/724dbb1d-6cfa-4cff-8bf4-5e1e742772df",
"name": "724dbb1d-6cfa-4cff-8bf4-5e1e742772df",
"status": "Failed",
"startTime": "2026-09-18T18:06:21.25552087Z",
"endTime": "2026-09-18T18:35:23.923631594Z",
"error": {
"code": "InternalServerError",
"message": "cluster placement did not complete before the deadline"
}
}
--------------------------------------------------------------------------------
...
occurredContributing tests (51)
Affected runs (1)
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Customer should be able to test admin credentials before cluster ready, then full admin credential lifecycle: Cluster provisioning failedfull failure pattern: Customer should be able to test admin credentials before cluster ready, then full admin credential lifecycle: Cluster provisioning failed Signal: IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | e2e | 1 | 2.70%1 of 37 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | Show trend detailsSep 13: 0 · Sep 14: 0 · Sep 15: 0 · Sep 16: 0 · Sep 17: 0 · Sep 18: 1 · Sep 19: 0 · Sep 20: 0 | none | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (1)fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:158]: Cluster provisioning failed fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:158]: Cluster provisioning failed Contributing tests (1)
Affected runs (1)
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
ERROR CODE: InternalServerError; detail code OperationNotAllowed; detail message Operation 'retrieveBootDiagnosticsData' is not allowed on VM 'cilium-cluster-cilium-np-sgbzk-<id>'...full failure pattern: ERROR CODE: InternalServerError; detail code OperationNotAllowed; detail message Operation 'retrieveBootDiagnosticsData' is not allowed on VM 'cilium-cluster-cilium-np-sgbzk-<id>' since another operation is in progress.; provider Microsoft.Compute Signal: IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | e2e | 1 | 2.70%1 of 37 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | Show trend detailsSep 13: 0 · Sep 14: 0 · Sep 15: 0 · Sep 16: 0 · Sep 17: 0 · Sep 18: 0 · Sep 19: 1 · Sep 20: 0 | none | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (1)fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:204]: failed to verify nodes are Ready with Cilium CNI for cluster "cilium-cluster"
Unexpected error:
<*errors.joinError | 0xc00160b7d0>:
failed to create NodePool cilium-np: failed waiting for nodepool="cilium-np" for cluster "cilium-cluster" in resourcegroup="complex-cilium-kv-rzlnqd88ksbr" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/0a92b991-1627-4a97-9fea-a9edfc1c8c70
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/0a92b991-1627-4a97-9fea-a9edfc1c8c70",
"name": "0a92b991-1627-4a97-9fea-a9edfc1c8c70",
"status": "Failed",
"startTime": "2026-09-19T02:54:29.386299179Z",
"endTime": "2026-09-19T03:13:30.082275878Z",
"error": {
"code": "InternalServerError",
"message": "node pool creation did not complete before the deadline"
}
}
--------------------------------------------------------------------------------
encountered errors downloading console logs:
- VM "cilium-cluster-cilium-np-sgbzk-2tmm2": failed to retrieve boot diagnostics data for VM "cilium-cluster-cilium-np-sgbzk-2tmm2" after 3 attempts, last error: POST https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/complex-cilium-kv-rzlnqd88ksbr--managed/providers/Microsoft.Compute/virtualMachines/cilium-cluster-cilium-np-sgbzk-2tmm2/retrieveBootDiagnosticsData
--------------------------------------------------------------------------------
RESPONSE 409: 409 Conflict
ERROR CODE: OperationNotAllowed
--------------------------------------------------------------------------------
{
"error": {
"code": "OperationNotAllowed",
"message": "Operation 'retrieveBootDiagnosticsData' is not allowed on VM 'cilium-cluster-cilium-np-sgbzk-2tmm2' since another operation is in progress."
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:204]: failed to verify nodes are Ready with Cilium CNI for cluster "cilium-cluster"
Unexpected error:
<*errors.joinError | 0xc00160b7d0>:
failed to create NodePool cilium-np: failed waiting for nodepool="cilium-np" for cluster "cilium-cluster" in resourcegroup="complex-cilium-kv-rzlnqd88ksbr" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/0a92b991-1627-4a97-9fea-a9edfc1c8c70
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/0a92b991-1627-4a97-9fea-a9edfc1c8c70",
"name": "0a92b991-1627-4a97-9fea-a9edfc1c8c70",
"status": "Failed",
"startTime": "2026-09-19T02:54:29.386299179Z",
"endTime": "2026-09-19T03:13:30.082275878Z",
"error": {
"code": "InternalServerError",
"message": "node pool creation did not complete before the deadline"
}
}
--------------------------------------------------------------------------------
encountered errors downloading console logs:
- VM "cilium-cluster-cilium-np-sgbzk-2tmm2": failed to retrieve boot diagnostics data for VM "cilium-cluster-cilium-np-sgbzk-2tmm2" after 3 attempts, last error: POST https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/complex-cilium-kv-rzlnqd88ksbr--managed/providers/Microsoft.Compute/virtualMachines/cilium-cluster-cilium-np-sgbzk-2tmm2/retrieveBootDiagnosticsData
--------------------------------------------------------------------------------
RESPONSE 409: 409 Conflict
ERROR CODE: OperationNotAllowed
--------------------------------------------------------------------------------
{
"error": {
"code": "OperationNotAllowed",
"message": "Operation 'retrieveBootDiagnosticsData' is not allowed on VM 'cilium-cluster-cilium-np-sgbzk-2tmm2' since another operation is in progress."
}
}
--------------------------------------------------------------------------------
...
occurredContributing tests (1)
Affected runs (1)
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
cluster entered terminal Failed state after role assignment deploymentfull failure pattern: cluster entered terminal Failed state after role assignment deployment Signal: IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | e2e | 1 | 2.70%1 of 37 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | Show trend detailsSep 13: 0 · Sep 14: 0 · Sep 15: 0 · Sep 16: 0 · Sep 17: 0 · Sep 18: 1 · Sep 19: 0 · Sep 20: 0 | none | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (1)fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:227]: Timed out after 2580.001s.
cluster should eventually succeed after role assignments are created
The function passed to Eventually failed at /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:223 with:
cluster entered terminal Failed state after role assignment deployment
Expected
<armredhatopenshifthcp.ProvisioningState>: ...
not to equal
<armredhatopenshifthcp.ProvisioningState>: ...
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:227]: Timed out after 2580.001s.
cluster should eventually succeed after role assignments are created
The function passed to Eventually failed at /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:223 with:
cluster entered terminal Failed state after role assignment deployment
Expected
<armredhatopenshifthcp.ProvisioningState>: ...
not to equal
<armredhatopenshifthcp.ProvisioningState>: ...Contributing tests (1)
Affected runs (1)
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
cluster operators not available: console (no Available condition), dns (Available=False, Progressing=True, Degraded=True, Upgradeable=True), image-registry (Available=False, Progr...full failure pattern: cluster operators not available: console (no Available condition), dns (Available=False, Progressing=True, Degraded=True, Upgradeable=True), image-registry (Available=False, Progressing=True, Degraded=True), ingress (Available=False, Progressing=True Signal: IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | e2e | 1 | 2.70%1 of 37 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | Show trend detailsSep 13: 0 · Sep 14: 0 · Sep 15: 0 · Sep 16: 1 · Sep 17: 0 · Sep 18: 0 · Sep 19: 0 · Sep 20: 0 | none | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (1)fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:251]: failed to verify cluster operators are available after external auth config creation
Unexpected error:
<*errors.joinError | 0xc002a4e018>:
context deadline exceeded
cluster operators not available: console (no Available condition), dns (Available=False, Progressing=True, Degraded=True, Upgradeable=True), image-registry (Available=False, Progressing=True, Degraded=True), ingress (Available=False, Progressing=True, Degraded=True, Upgradeable=True, EvaluationConditionsDetected=False), monitoring (no Available condition), openshift-samples (no Available condition), storage (Degraded=False, Progressing=False, Available=False, Upgradeable=True, EvaluationConditionsDetected=Unknown)
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:251]: failed to verify cluster operators are available after external auth config creation
Unexpected error:
<*errors.joinError | 0xc002a4e018>:
context deadline exceeded
cluster operators not available: console (no Available condition), dns (Available=False, Progressing=True, Degraded=True, Upgradeable=True), image-registry (Available=False, Progressing=True, Degraded=True), ingress (Available=False, Progressing=True, Degraded=True, Upgradeable=True, EvaluationConditionsDetected=False), monitoring (no Available condition), openshift-samples (no Available condition), storage (Degraded=False, Progressing=False, Available=False, Upgradeable=True, EvaluationConditionsDetected=Unknown)
...
occurredContributing tests (1)
Affected runs (1)
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
failed to create nodepool <nodepool>: failed waiting for nodepool="<nodepool>" for cluster "<cluster>" in resourcegroup="<resource-group>" to finish creating: GET <url>full failure pattern: failed to create nodepool <nodepool>: failed waiting for nodepool="<nodepool>" for cluster "<cluster>" in resourcegroup="<resource-group>" to finish creating: GET <url> Signal: IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | e2e | 1 | 2.70%1 of 37 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | Show trend detailsSep 13: 0 · Sep 14: 0 · Sep 15: 0 · Sep 16: 0 · Sep 17: 0 · Sep 18: 0 · Sep 19: 1 · Sep 20: 0 | none | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (1)fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_delete.go:127]: failed to create node pools
Expected success, but got an error:
<*fmt.wrapError | 0xc00381c7e0>:
failed to create NodePool np-deletable: failed waiting for nodepool="np-deletable" for cluster "np-delete-hcp-cluster" in resourcegroup="nodepool-delete-8j5t9t6fn8sk" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/d87907d8-e7ff-4012-ba96-a28fad371f71
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/d87907d8-e7ff-4012-ba96-a28fad371f71",
"name": "d87907d8-e7ff-4012-ba96-a28fad371f71",
"status": "Failed",
"startTime": "2026-09-19T03:13:36.546108406Z",
"endTime": "2026-09-19T03:32:43.229867603Z",
"error": {
"code": "InternalServerError",
"message": "node pool creation did not complete before the deadline"
}
}
--------------------------------------------------------------------------------
...
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_delete.go:127]: failed to create node pools
Expected success, but got an error:
<*fmt.wrapError | 0xc00381c7e0>:
failed to create NodePool np-deletable: failed waiting for nodepool="np-deletable" for cluster "np-delete-hcp-cluster" in resourcegroup="nodepool-delete-8j5t9t6fn8sk" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/d87907d8-e7ff-4012-ba96-a28fad371f71
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/d87907d8-e7ff-4012-ba96-a28fad371f71",
"name": "d87907d8-e7ff-4012-ba96-a28fad371f71",
"status": "Failed",
"startTime": "2026-09-19T03:13:36.546108406Z",
"endTime": "2026-09-19T03:32:43.229867603Z",
"error": {
"code": "InternalServerError",
"message": "node pool creation did not complete before the deadline"
}
}
--------------------------------------------------------------------------------
...Contributing tests (1)
Affected runs (1)
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
failed waiting for nodepool="<nodepool>" for cluster "<cluster>" in resourcegroup="<resource-group>" to finish creating: context deadline exceededfull failure pattern: failed waiting for nodepool="<nodepool>" for cluster "<cluster>" in resourcegroup="<resource-group>" to finish creating: context deadline exceeded Signal: IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | e2e | 1 | 2.70%1 of 37 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | Show trend detailsSep 13: 0 · Sep 14: 0 · Sep 15: 0 · Sep 16: 0 · Sep 17: 0 · Sep 18: 1 · Sep 19: 0 · Sep 20: 0 | none | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (1)fail [github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:198]: failed to create node pool version 4.20.32 for back-level cluster
Unexpected error:
<*fmt.wrapError | 0xc001376700>:
failed waiting for nodepool="np-ver-4-20-32" for cluster "cluster-ver-4-20" in resourcegroup="rg-cluster-back-version-tzdswnvvvrrb" to finish creating: context deadline exceeded
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:198]: failed to create node pool version 4.20.32 for back-level cluster
Unexpected error:
<*fmt.wrapError | 0xc001376700>:
failed waiting for nodepool="np-ver-4-20-32" for cluster "cluster-ver-4-20" in resourcegroup="rg-cluster-back-version-tzdswnvvvrrb" to finish creating: context deadline exceeded
...
occurredContributing tests (1)
Affected runs (1)
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
timeout during CreateHCPClusterAndWait; context deadline exceededfull failure pattern: timeout during CreateHCPClusterAndWait; context deadline exceeded Signal: IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | e2e | 1 | 2.70%1 of 37 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | Show trend detailsSep 13: 0 · Sep 14: 0 · Sep 15: 0 · Sep 16: 0 · Sep 17: 0 · Sep 18: 1 · Sep 19: 0 · Sep 20: 0 | none | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (1)fail [github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:152]: failed to create HCP cluster version 4.20
Unexpected error:
<*fmt.wrapErrors | 0xc0010a86c0>:
failed waiting for cluster="cluster-ver-4-20" in resourcegroup="rg-cluster-back-version-ql4cwwsm948w" to finish creating, caused by: timeout '30.000000' minutes exceeded during CreateHCPClusterAndWait for cluster cluster-ver-4-20 in resource group rg-cluster-back-version-ql4cwwsm948w, error: context deadline exceeded
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:152]: failed to create HCP cluster version 4.20
Unexpected error:
<*fmt.wrapErrors | 0xc0010a86c0>:
failed waiting for cluster="cluster-ver-4-20" in resourcegroup="rg-cluster-back-version-ql4cwwsm948w" to finish creating, caused by: timeout '30.000000' minutes exceeded during CreateHCPClusterAndWait for cluster cluster-ver-4-20 in resource group rg-cluster-back-version-ql4cwwsm948w, error: context deadline exceeded
...
occurredContributing tests (1)
Affected runs (1)
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||