No failure patterns match your search.
Environment: STG
Failure patterns: 5 · Failures matched: 21 · Total runs: 3 · Runs affected: 3
| Failure patternThe canonical description of a recurring CI failure, extracted and normalized from raw logs. | Failed atThe stage of the job run where this failure occurred: 'provision' (environment setup, DEV only), 'e2e' (test suite execution), or 'other' (CI infrastructure issues that did not produce a failure pattern). | Number of distinct job runs where this failure pattern was detected. | Percentage of all job runs in this environment affected by this failure pattern during the selected window. | Classification of this failure pattern: Regression (likely caused by a specific PR), Flake (intermittent failure spread across days), Noise (low-quality or generic pattern), or Indeterminate. | TrendShows daily activity for this failure pattern in a trailing window anchored to the selected end date. The sparkline covers at least 7 days and at most 14 days, depending on the current window size. | Also inOther environments where the same failure pattern was also detected during the selected window. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
CreateNodePoolFromParam timed out after <minutes> minutes; context deadline exceededfull failure pattern: CreateNodePoolFromParam timed out after <minutes> minutes; context deadline exceeded Signal: IndeterminateSignal: Indeterminate — seen in 3 prior week(s); active 1 day(s) | e2e | 3 | 100.00%3 of 3 job runs affected | IndeterminateSignal: Indeterminate — seen in 3 prior week(s); active 1 day(s) | Show trend detailsJun 29: 0 · Jun 30: 0 · Jul 1: 0 · Jul 2: 0 · Jul 3: 0 · Jul 4: 0 · Jul 5: 11 | none | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (3)fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:159]: failed to create node pool "np-nv7xcm" for cluster "cluster-nightly-4-21-nv7xcm"
Unexpected error:
<*fmt.wrapErrors | 0xc002593050>:
failed to create NodePool np-nv7xcm, caused by: timeout '20.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-nv7xcm in resource group rg-nightly-4-21-sdkqw5, error: failed waiting for nodepool="np-nv7xcm" for cluster "cluster-nightly-4-21-nv7xcm" in resourcegroup="rg-nightly-4-21-sdkqw5" to finish creating: context deadline exceeded
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:159]: failed to create node pool "np-nv7xcm" for cluster "cluster-nightly-4-21-nv7xcm"
Unexpected error:
<*fmt.wrapErrors | 0xc002593050>:
failed to create NodePool np-nv7xcm, caused by: timeout '20.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-nv7xcm in resource group rg-nightly-4-21-sdkqw5, error: failed waiting for nodepool="np-nv7xcm" for cluster "cluster-nightly-4-21-nv7xcm" in resourcegroup="rg-nightly-4-21-sdkqw5" to finish creating: context deadline exceeded
...
occurredfail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:136]: failed to create node pool "np-1" for private keyvault cluster "private-kv-cluster"
Unexpected error:
<*fmt.wrapErrors | 0xc002944450>:
failed to create NodePool np-1, caused by: timeout '20.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group private-keyvault-cppqwc, error: failed waiting for nodepool="np-1" for cluster "private-kv-cluster" in resourcegroup="private-keyvault-cppqwc" to finish creating: context deadline exceeded
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:136]: failed to create node pool "np-1" for private keyvault cluster "private-kv-cluster"
Unexpected error:
<*fmt.wrapErrors | 0xc002944450>:
failed to create NodePool np-1, caused by: timeout '20.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group private-keyvault-cppqwc, error: failed waiting for nodepool="np-1" for cluster "private-kv-cluster" in resourcegroup="private-keyvault-cppqwc" to finish creating: context deadline exceeded
...
occurredfail [github.com/Azure/ARO-HCP/test/e2e/cluster_fips_mode.go:100]: failed to create node pool "np-1" for fips-enabled cluster "fips-enabled-cluster"
Unexpected error:
<*fmt.wrapErrors | 0xc001c9d650>:
failed to create NodePool np-1, caused by: timeout '20.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group fips-enabled-gvqsv2, error: failed waiting for nodepool="np-1" for cluster "fips-enabled-cluster" in resourcegroup="fips-enabled-gvqsv2" to finish creating: context deadline exceeded
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_fips_mode.go:100]: failed to create node pool "np-1" for fips-enabled cluster "fips-enabled-cluster"
Unexpected error:
<*fmt.wrapErrors | 0xc001c9d650>:
failed to create NodePool np-1, caused by: timeout '20.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group fips-enabled-gvqsv2, error: failed waiting for nodepool="np-1" for cluster "fips-enabled-cluster" in resourcegroup="fips-enabled-gvqsv2" to finish creating: context deadline exceeded
...
occurredContributing tests (11)
Affected runs (3)
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
timeout during CreateHCPClusterAndWait; context deadline exceededfull failure pattern: timeout during CreateHCPClusterAndWait; context deadline exceeded Signal: IndeterminateSignal: Indeterminate — seen in 4 prior week(s); active 1 day(s) | e2e | 2 | 66.67%2 of 3 job runs affected | IndeterminateSignal: Indeterminate — seen in 4 prior week(s); active 1 day(s) | Show trend detailsJun 29: 0 · Jun 30: 0 · Jul 1: 0 · Jul 2: 0 · Jul 3: 0 · Jul 4: 0 · Jul 5: 3 | none | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (3)fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:106]: HCP cluster rg-candidate-5-0-znf2h5/cluster-candidate-5-0-c62xlb should provision
Unexpected error:
<*fmt.wrapErrors | 0xc000b6edb0>:
failed to create HCP cluster cluster-candidate-5-0-c62xlb, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster cluster-candidate-5-0-c62xlb in resource group rg-candidate-5-0-znf2h5, error: failed waiting for cluster="cluster-candidate-5-0-c62xlb" in resourcegroup="rg-candidate-5-0-znf2h5" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster cluster-candidate-5-0-c62xlb in resource group rg-candidate-5-0-znf2h5, error: context deadline exceeded
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:106]: HCP cluster rg-candidate-5-0-znf2h5/cluster-candidate-5-0-c62xlb should provision
Unexpected error:
<*fmt.wrapErrors | 0xc000b6edb0>:
failed to create HCP cluster cluster-candidate-5-0-c62xlb, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster cluster-candidate-5-0-c62xlb in resource group rg-candidate-5-0-znf2h5, error: failed waiting for cluster="cluster-candidate-5-0-c62xlb" in resourcegroup="rg-candidate-5-0-znf2h5" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster cluster-candidate-5-0-c62xlb in resource group rg-candidate-5-0-znf2h5, error: context deadline exceeded
...
occurredfail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:106]: HCP cluster rg-candidate-4-22-bp5mz6/cluster-candidate-4-22-hkn2wx should provision
Unexpected error:
<*fmt.wrapErrors | 0xc00140a660>:
failed to create HCP cluster cluster-candidate-4-22-hkn2wx, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster cluster-candidate-4-22-hkn2wx in resource group rg-candidate-4-22-bp5mz6, error: failed waiting for cluster="cluster-candidate-4-22-hkn2wx" in resourcegroup="rg-candidate-4-22-bp5mz6" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster cluster-candidate-4-22-hkn2wx in resource group rg-candidate-4-22-bp5mz6, error: context deadline exceeded
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:106]: HCP cluster rg-candidate-4-22-bp5mz6/cluster-candidate-4-22-hkn2wx should provision
Unexpected error:
<*fmt.wrapErrors | 0xc00140a660>:
failed to create HCP cluster cluster-candidate-4-22-hkn2wx, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster cluster-candidate-4-22-hkn2wx in resource group rg-candidate-4-22-bp5mz6, error: failed waiting for cluster="cluster-candidate-4-22-hkn2wx" in resourcegroup="rg-candidate-4-22-bp5mz6" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster cluster-candidate-4-22-hkn2wx in resource group rg-candidate-4-22-bp5mz6, error: context deadline exceeded
...
occurredfail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:106]: HCP cluster rg-candidate-5-0-jvq82b/cluster-candidate-5-0-k6hscf should provision
Unexpected error:
<*fmt.wrapErrors | 0xc000fe82a0>:
failed to create HCP cluster cluster-candidate-5-0-k6hscf, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster cluster-candidate-5-0-k6hscf in resource group rg-candidate-5-0-jvq82b, error: failed waiting for cluster="cluster-candidate-5-0-k6hscf" in resourcegroup="rg-candidate-5-0-jvq82b" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster cluster-candidate-5-0-k6hscf in resource group rg-candidate-5-0-jvq82b, error: context deadline exceeded
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:106]: HCP cluster rg-candidate-5-0-jvq82b/cluster-candidate-5-0-k6hscf should provision
Unexpected error:
<*fmt.wrapErrors | 0xc000fe82a0>:
failed to create HCP cluster cluster-candidate-5-0-k6hscf, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster cluster-candidate-5-0-k6hscf in resource group rg-candidate-5-0-jvq82b, error: failed waiting for cluster="cluster-candidate-5-0-k6hscf" in resourcegroup="rg-candidate-5-0-jvq82b" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster cluster-candidate-5-0-k6hscf in resource group rg-candidate-5-0-jvq82b, error: context deadline exceeded
...
occurredContributing tests (3)
Affected runs (2)
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
failed to cleanup resource group: at least one HCP cluster <cluster> to delete: failed waiting for hcpcluster="<cluster>" in resourcegroup="<resource-group>" to finish deleting: c...full failure pattern: failed to cleanup resource group: at least one HCP cluster <cluster> to delete: failed waiting for hcpcluster="<cluster>" in resourcegroup="<resource-group>" to finish deleting: context canceled Signal: IndeterminateSignal: Indeterminate — seen in 4 prior week(s); active 1 day(s) | e2e | 1 | 33.33%1 of 3 job runs affected | IndeterminateSignal: Indeterminate — seen in 4 prior week(s); active 1 day(s) | Show trend detailsJun 29: 0 · Jun 30: 0 · Jul 1: 0 · Jul 2: 0 · Jul 3: 0 · Jul 4: 0 · Jul 5: 5 | none | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (3)fail [github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:283] A node timeout occurred and then the following failure was recorded in the timedout node before it exited:
Unexpected error:
<*errors.joinError | 0xc000ef8438>:
failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster="np-downgrade-lbdt2z" in resourcegroup="rg-np-downgrade-lbdt2z-lp9gxb" to finish deleting: context canceled
...
occurred
fail [:0]: A node timeout occurred
fail [github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:283] A node timeout occurred and then the following failure was recorded in the timedout node before it exited:
Unexpected error:
<*errors.joinError | 0xc000ef8438>:
failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster="np-downgrade-lbdt2z" in resourcegroup="rg-np-downgrade-lbdt2z-lp9gxb" to finish deleting: context canceled
...
occurred
fail [:0]: A node timeout occurredfail [github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:283] A node timeout occurred and then the following failure was recorded in the timedout node before it exited:
Unexpected error:
<*errors.joinError | 0xc001f2c1f8>:
failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster="np-autoscale-cluster" in resourcegroup="np-autoscaling-pwdmv7" to finish deleting: context canceled
...
occurred
fail [:0]: A node timeout occurred
fail [github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:283] A node timeout occurred and then the following failure was recorded in the timedout node before it exited:
Unexpected error:
<*errors.joinError | 0xc001f2c1f8>:
failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster="np-autoscale-cluster" in resourcegroup="np-autoscaling-pwdmv7" to finish deleting: context canceled
...
occurred
fail [:0]: A node timeout occurredfail [github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:283] A node timeout occurred and then the following failure was recorded in the timedout node before it exited:
Unexpected error:
<*errors.joinError | 0xc002716078>:
failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster="np-skip-minor-mrk2x7" in resourcegroup="rg-np-skip-minor-mrk2x7-4m7cqm" to finish deleting: context canceled
...
occurred
fail [:0]: A node timeout occurred
fail [github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:283] A node timeout occurred and then the following failure was recorded in the timedout node before it exited:
Unexpected error:
<*errors.joinError | 0xc002716078>:
failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster="np-skip-minor-mrk2x7" in resourcegroup="rg-np-skip-minor-mrk2x7-4m7cqm" to finish deleting: context canceled
...
occurred
fail [:0]: A node timeout occurredContributing tests (5)
Affected runs (1)
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.fips-enabled": unrecognized experimental tag; provider Microsoft.RedHatOpenShiftfull failure pattern: ERROR CODE: InvalidRequestContent; detail message Invalid value: "aro-hcp.experimental.cluster.fips-enabled": unrecognized experimental tag; provider Microsoft.RedHatOpenShift Signal: IndeterminateSignal: Indeterminate — no prior history; active 2 day(s) — New failure pattern — no prior history | e2e | 1 | 33.33%1 of 3 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 2 day(s) — New failure pattern — no prior history | Show trend detailsJun 29: 0 · Jun 30: 0 · Jul 1: 0 · Jul 2: 0 · Jul 3: 0 · Jul 4: 2 · Jul 5: 1 | none | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (1)fail [github.com/Azure/ARO-HCP/test/e2e/cluster_fips_mode.go:84]: failed to create HCP cluster "fips-enabled-cluster" with FIPS enabled
Unexpected error:
<*fmt.wrapError | 0xc000785b00>:
failed starting cluster creation "fips-enabled-cluster" in resourcegroup="fips-enabled-g9zkzm": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/fips-enabled-g9zkzm/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/fips-enabled-cluster
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"aro-hcp.experimental.cluster.fips-enabled\": unrecognized experimental tag",
"target": "tags[aro-hcp.experimental.cluster.fips-enabled]"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_fips_mode.go:84]: failed to create HCP cluster "fips-enabled-cluster" with FIPS enabled
Unexpected error:
<*fmt.wrapError | 0xc000785b00>:
failed starting cluster creation "fips-enabled-cluster" in resourcegroup="fips-enabled-g9zkzm": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/fips-enabled-g9zkzm/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/fips-enabled-cluster
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Invalid value: \"aro-hcp.experimental.cluster.fips-enabled\": unrecognized experimental tag",
"target": "tags[aro-hcp.experimental.cluster.fips-enabled]"
}
}
--------------------------------------------------------------------------------
...
occurredContributing tests (1)
Affected runs (1)
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
error should indicate FIPS value must be true or falsefull failure pattern: error should indicate FIPS value must be true or false Signal: IndeterminateSignal: Indeterminate — no prior history; active 2 day(s) — New failure pattern — no prior history | e2e | 1 | 33.33%1 of 3 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 2 day(s) — New failure pattern — no prior history | Show trend detailsJun 29: 0 · Jun 30: 0 · Jul 1: 0 · Jul 2: 0 · Jul 3: 0 · Jul 4: 2 · Jul 5: 1 | none | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (1)fail [github.com/Azure/ARO-HCP/test/e2e/cluster_fips_mode.go:203]: error should indicate FIPS value must be true or false
Expected
<string>: ...
to contain substring
<string>: ...
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_fips_mode.go:203]: error should indicate FIPS value must be true or false
Expected
<string>: ...
to contain substring
<string>: ...Contributing tests (1)
Affected runs (1)
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||