No failure patterns match your search.
Environment: STG
Failure patterns: 7 · Failures matched: 21 · Total runs: 12 · Runs affected: 7
| Failure patternThe canonical description of a recurring CI failure, extracted and normalized from raw logs. | Failed atThe stage of the job run where this failure occurred: 'provision' (environment setup, DEV only), 'e2e' (test suite execution), or 'other' (CI infrastructure issues that did not produce a failure pattern). | Number of distinct job runs where this failure pattern was detected. | Percentage of all job runs in this environment affected by this failure pattern during the selected window. | Classification of this failure pattern: Regression (likely caused by a specific PR), Flake (intermittent failure spread across days), Noise (low-quality or generic pattern), or Indeterminate. | TrendShows daily activity for this failure pattern in a trailing window anchored to the selected end date. The sparkline covers at least 7 days and at most 14 days, depending on the current window size. | Also inOther environments where the same failure pattern was also detected during the selected window. | |||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
failed to cleanup resource group: at least one HCP cluster <cluster> to delete: failed waiting for hcpcluster="<cluster>" in resourcegroup="<resource-group>" to finish deleting: c...full failure pattern: failed to cleanup resource group: at least one HCP cluster <cluster> to delete: failed waiting for hcpcluster="<cluster>" in resourcegroup="<resource-group>" to finish deleting: context canceled Signal: IndeterminateSignal: Indeterminate — seen in 1 prior week(s); active 3 day(s) | e2e | 3 | 25.00%3 of 12 job runs affected | IndeterminateSignal: Indeterminate — seen in 1 prior week(s); active 3 day(s) | Show trend detailsMay 17: 0 · May 18: 3 · May 19: 2 · May 20: 0 · May 21: 2 · May 22: 0 · May 23: 0 · May 24: 0 | none | |||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (3)fail [github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:262] A node timeout occurred and then the following failure was recorded in the timedout node before it exited:
Unexpected error:
<*errors.joinError | 0xc000d525e8>:
failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster="hcp-cluster-np-128" in resourcegroup="clusternp128-bjt9bn" to finish deleting: context canceled
...
occurred
fail [:0]: A node timeout occurred
fail [github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:262] A node timeout occurred and then the following failure was recorded in the timedout node before it exited:
Unexpected error:
<*errors.joinError | 0xc000d525e8>:
failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster="hcp-cluster-np-128" in resourcegroup="clusternp128-bjt9bn" to finish deleting: context canceled
...
occurred
fail [:0]: A node timeout occurredfail [github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:262] A node timeout occurred and then the following failure was recorded in the timedout node before it exited:
Unexpected error:
<*errors.joinError | 0xc0010f8138>:
failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster="cluster-candidate-4-20-xhwfzs" in resourcegroup="rg-candidate-4-20-xchgmf" to finish deleting: context canceled
...
occurred
fail [:0]: A node timeout occurred
fail [github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:262] A node timeout occurred and then the following failure was recorded in the timedout node before it exited:
Unexpected error:
<*errors.joinError | 0xc0010f8138>:
failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster="cluster-candidate-4-20-xhwfzs" in resourcegroup="rg-candidate-4-20-xchgmf" to finish deleting: context canceled
...
occurred
fail [:0]: A node timeout occurredfail [github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:262] A node timeout occurred and then the following failure was recorded in the timedout node before it exited:
Unexpected error:
<*errors.joinError | 0xc000bb4540>:
failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster="cluster-np-labels-taints" in resourcegroup="rg-np-labels-taints-jjhg79" to finish deleting: context canceled
...
occurred
fail [:0]: A node timeout occurred
fail [github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:262] A node timeout occurred and then the following failure was recorded in the timedout node before it exited:
Unexpected error:
<*errors.joinError | 0xc000bb4540>:
failed to cleanup resource group: at least one hcp cluster failed to delete: failed waiting for hcpCluster="cluster-np-labels-taints" in resourcegroup="rg-np-labels-taints-jjhg79" to finish deleting: context canceled
...
occurred
fail [:0]: A node timeout occurredContributing tests (7)
Affected runs (3)
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||
timeout during CreateHCPClusterAndWait; context deadline exceededfull failure pattern: timeout during CreateHCPClusterAndWait; context deadline exceeded Signal: IndeterminateSignal: Indeterminate — seen in 4 prior week(s); active 2 day(s) | e2e | 2 | 16.67%2 of 12 job runs affected | IndeterminateSignal: Indeterminate — seen in 4 prior week(s); active 2 day(s) | Show trend detailsMay 17: 0 · May 18: 0 · May 19: 0 · May 20: 0 · May 21: 0 · May 22: 8 · May 23: 1 · May 24: 0 | none | |||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (3)fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:158]: failed to create HCP cluster "cp-ystream-upgrade-4-22-24644n" with install version 4.21.16
Unexpected error:
<*fmt.wrapErrors | 0xc000aa96e0>:
failed to create HCP cluster cp-ystream-upgrade-4-22-24644n, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster cp-ystream-upgrade-4-22-24644n in resource group rg-cp-ystream-upgrade-4-22-xpdwqm, error: failed waiting for cluster="cp-ystream-upgrade-4-22-24644n" in resourcegroup="rg-cp-ystream-upgrade-4-22-xpdwqm" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster cp-ystream-upgrade-4-22-24644n in resource group rg-cp-ystream-upgrade-4-22-xpdwqm, error: context deadline exceeded
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_minor_upgrade.go:158]: failed to create HCP cluster "cp-ystream-upgrade-4-22-24644n" with install version 4.21.16
Unexpected error:
<*fmt.wrapErrors | 0xc000aa96e0>:
failed to create HCP cluster cp-ystream-upgrade-4-22-24644n, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster cp-ystream-upgrade-4-22-24644n in resource group rg-cp-ystream-upgrade-4-22-xpdwqm, error: failed waiting for cluster="cp-ystream-upgrade-4-22-24644n" in resourcegroup="rg-cp-ystream-upgrade-4-22-xpdwqm" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster cp-ystream-upgrade-4-22-24644n in resource group rg-cp-ystream-upgrade-4-22-xpdwqm, error: context deadline exceeded
...
occurredfail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:120]: Swift cluster rg-candidate-5-0-7jt2zr/cluster-candidate-5-0-898h7z should provision
Unexpected error:
<*fmt.wrapErrors | 0xc000a159e0>:
failed waiting for cluster="cluster-candidate-5-0-898h7z" in resourcegroup="rg-candidate-5-0-7jt2zr" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223AndWait for cluster cluster-candidate-5-0-898h7z in resource group rg-candidate-5-0-7jt2zr, error: context deadline exceeded
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:120]: Swift cluster rg-candidate-5-0-7jt2zr/cluster-candidate-5-0-898h7z should provision
Unexpected error:
<*fmt.wrapErrors | 0xc000a159e0>:
failed waiting for cluster="cluster-candidate-5-0-898h7z" in resourcegroup="rg-candidate-5-0-7jt2zr" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223AndWait for cluster cluster-candidate-5-0-898h7z in resource group rg-candidate-5-0-7jt2zr, error: context deadline exceeded
...
occurredfail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:137]: failed to create HCP cluster "cluster-zstream-4-22-9dtk7p" with version 4.22.0-rc.2 on candidate channel
Unexpected error:
<*fmt.wrapErrors | 0xc00114e9f0>:
failed to create HCP cluster cluster-zstream-4-22-9dtk7p, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster cluster-zstream-4-22-9dtk7p in resource group rg-zstream-upgrade-4-22-68pdm8, error: failed waiting for cluster="cluster-zstream-4-22-9dtk7p" in resourcegroup="rg-zstream-upgrade-4-22-68pdm8" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster cluster-zstream-4-22-9dtk7p in resource group rg-zstream-upgrade-4-22-68pdm8, error: context deadline exceeded
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/control_plane_automated_z_stream_upgrade.go:137]: failed to create HCP cluster "cluster-zstream-4-22-9dtk7p" with version 4.22.0-rc.2 on candidate channel
Unexpected error:
<*fmt.wrapErrors | 0xc00114e9f0>:
failed to create HCP cluster cluster-zstream-4-22-9dtk7p, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster cluster-zstream-4-22-9dtk7p in resource group rg-zstream-upgrade-4-22-68pdm8, error: failed waiting for cluster="cluster-zstream-4-22-9dtk7p" in resourcegroup="rg-zstream-upgrade-4-22-68pdm8" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster cluster-zstream-4-22-9dtk7p in resource group rg-zstream-upgrade-4-22-68pdm8, error: context deadline exceeded
...
occurredContributing tests (9)
Affected runs (2)
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||
context deadline exceededfull failure pattern: context deadline exceeded Signal: IndeterminateSignal: Indeterminate — seen in 3 prior week(s); active 1 day(s) | e2e | 1 | 8.33%1 of 12 job runs affected | IndeterminateSignal: Indeterminate — seen in 3 prior week(s); active 1 day(s) | Show trend detailsMay 17: 0 · May 18: 0 · May 19: 0 · May 20: 1 · May 21: 0 · May 22: 0 · May 23: 0 · May 24: 0 | none | |||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (1)fail [github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:132]: Unexpected error:
<context.deadlineExceededError>:
context deadline exceeded
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:132]: Unexpected error:
<context.deadlineExceededError>:
context deadline exceeded
...
occurredContributing tests (1)
Affected runs (1)
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||
ERROR CODE: InvalidRequestContent; detail message Duplicate ARO-HCP cluster <cluster>. An ARO-HCP cluster <cluster> name 'cluster-candidate-4-21-lwfbjc', Azure Subscription ID 'XX...full failure pattern: ERROR CODE: InvalidRequestContent; detail message Duplicate ARO-HCP cluster <cluster>. An ARO-HCP cluster <cluster> name 'cluster-candidate-4-21-lwfbjc', Azure Subscription ID 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX', and and Azure resource group <resource-group> 'rg-candidate-4-21-cgh878' already exists; provider Microsoft.RedHatOpenShift Signal: IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | e2e | 1 | 8.33%1 of 12 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | Show trend detailsMay 17: 0 · May 18: 0 · May 19: 0 · May 20: 1 · May 21: 0 · May 22: 0 · May 23: 0 · May 24: 0 | none | |||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (1)fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:109]: Unexpected error:
<*fmt.wrapError | 0xc00133ac60>:
failed to create HCP cluster cluster-candidate-4-21-lwfbjc: failed starting cluster creation "cluster-candidate-4-21-lwfbjc" in resourcegroup="rg-candidate-4-21-cgh878": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-4-21-cgh878/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-candidate-4-21-lwfbjc
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Duplicate ARO-HCP cluster name. An ARO-HCP cluster with name 'cluster-candidate-4-21-lwfbjc', Azure Subscription ID 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX', and and Azure Resource group name 'rg-candidate-4-21-cgh878' already exists"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:109]: Unexpected error:
<*fmt.wrapError | 0xc00133ac60>:
failed to create HCP cluster cluster-candidate-4-21-lwfbjc: failed starting cluster creation "cluster-candidate-4-21-lwfbjc" in resourcegroup="rg-candidate-4-21-cgh878": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-candidate-4-21-cgh878/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cluster-candidate-4-21-lwfbjc
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidRequestContent
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidRequestContent",
"message": "Duplicate ARO-HCP cluster name. An ARO-HCP cluster with name 'cluster-candidate-4-21-lwfbjc', Azure Subscription ID 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX', and and Azure Resource group name 'rg-candidate-4-21-cgh878' already exists"
}
}
--------------------------------------------------------------------------------
...
occurredContributing tests (1)
Affected runs (1)
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||
RevokeCredentialsAndWait timed out after <minutes> minutes; context deadline exceededfull failure pattern: RevokeCredentialsAndWait timed out after <minutes> minutes; context deadline exceeded Signal: IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | e2e | 1 | 8.33%1 of 12 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | Show trend detailsMay 17: 0 · May 18: 0 · May 19: 0 · May 20: 1 · May 21: 0 · May 22: 0 · May 23: 0 · May 24: 0 | none | |||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (1)fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:252]: Unexpected error:
<*fmt.wrapErrors | 0xc000bcc3c0>:
failed waiting for hcpCluster="admin-cred-lifecycle-n6spml" in resourcegroup="admin-credential-lifecycle-test-lk87zw" to finish revoking creds, caused by: timeout '15.000000' minutes exceeded during RevokeCredentialsAndWait for cluster admin-cred-lifecycle-n6spml in resource group admin-credential-lifecycle-test-lk87zw, error: context deadline exceeded
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/admin_credential_lifecycle.go:252]: Unexpected error:
<*fmt.wrapErrors | 0xc000bcc3c0>:
failed waiting for hcpCluster="admin-cred-lifecycle-n6spml" in resourcegroup="admin-credential-lifecycle-test-lk87zw" to finish revoking creds, caused by: timeout '15.000000' minutes exceeded during RevokeCredentialsAndWait for cluster admin-cred-lifecycle-n6spml in resource group admin-credential-lifecycle-test-lk87zw, error: context deadline exceeded
...
occurredContributing tests (1)
Affected runs (1)
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||
failed to wait for first cluster "<cluster>" to complete creationfull failure pattern: failed to wait for first cluster "<cluster>" to complete creation Signal: IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | e2e | 1 | 8.33%1 of 12 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | Show trend detailsMay 17: 0 · May 18: 0 · May 19: 0 · May 20: 0 · May 21: 0 · May 22: 1 · May 23: 0 · May 24: 0 | none | |||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (1)fail [github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:132]: failed to wait for first cluster "basic-hcp-cluster" to complete creation
Unexpected error:
<context.deadlineExceededError>:
context deadline exceeded
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/clusters_sharing_resgroup.go:132]: failed to wait for first cluster "basic-hcp-cluster" to complete creation
Unexpected error:
<context.deadlineExceededError>:
context deadline exceeded
...
occurredContributing tests (1)
Affected runs (1)
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||
validatingadmissionpolicies.admissionregistration.k8s.io "image-registry-allowlist-policy" is forbidden: User "system:serviceaccount:ci-op-n12p2cds:stage-e2e-parallel" cannot get ...full failure pattern: validatingadmissionpolicies.admissionregistration.k8s.io "image-registry-allowlist-policy" is forbidden: User "system:serviceaccount:ci-op-n12p2cds:stage-e2e-parallel" cannot get resource "validatingadmissionpolicies" in API group Signal: IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | e2e | 1 | 8.33%1 of 12 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | Show trend detailsMay 17: 0 · May 18: 0 · May 19: 0 · May 20: 0 · May 21: 0 · May 22: 1 · May 23: 0 · May 24: 0 | none | |||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (1)fail [github.com/Azure/ARO-HCP/test/e2e/image_registry_policy.go:135]: ValidatingAdmissionPolicy should exist
Unexpected error:
<*errors.StatusError | 0xc000e0e140>:
validatingadmissionpolicies.admissionregistration.k8s.io "image-registry-allowlist-policy" is forbidden: User "system:serviceaccount:ci-op-n12p2cds:stage-e2e-parallel" cannot get resource "validatingadmissionpolicies" in API group "admissionregistration.k8s.io" at the cluster scope
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/image_registry_policy.go:135]: ValidatingAdmissionPolicy should exist
Unexpected error:
<*errors.StatusError | 0xc000e0e140>:
validatingadmissionpolicies.admissionregistration.k8s.io "image-registry-allowlist-policy" is forbidden: User "system:serviceaccount:ci-op-n12p2cds:stage-e2e-parallel" cannot get resource "validatingadmissionpolicies" in API group "admissionregistration.k8s.io" at the cluster scope
...
occurredContributing tests (1)
Affected runs (1)
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||