No failure patterns match your search.
Environment: PROD
Failure patterns: 3 · Failures matched: 18 · Total runs: 2 · Runs affected: 2
| Failure patternThe canonical description of a recurring CI failure, extracted and normalized from raw logs. | Failed atThe stage of the job run where this failure occurred: 'provision' (environment setup, DEV only), 'e2e' (test suite execution), or 'other' (CI infrastructure issues that did not produce a failure pattern). | Number of distinct job runs where this failure pattern was detected. | Percentage of all job runs in this environment affected by this failure pattern during the selected window. | Classification of this failure pattern: Regression (likely caused by a specific PR), Flake (intermittent failure spread across days), Noise (low-quality or generic pattern), or Indeterminate. | TrendShows daily activity for this failure pattern in a trailing window anchored to the selected end date. The sparkline covers at least 7 days and at most 14 days, depending on the current window size. | Also inOther environments where the same failure pattern was also detected during the selected window. | |||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
CreateNodePoolFromParam timed out after <minutes> minutes; context deadline exceededfull failure pattern: CreateNodePoolFromParam timed out after <minutes> minutes; context deadline exceeded Signal: IndeterminateSignal: Indeterminate — seen in 3 prior week(s); active 1 day(s) | e2e | 2 | 100.00%2 of 2 job runs affected | IndeterminateSignal: Indeterminate — seen in 3 prior week(s); active 1 day(s) | Show trend detailsJun 8: 0 · Jun 9: 0 · Jun 10: 0 · Jun 11: 0 · Jun 12: 0 · Jun 13: 0 · Jun 14: 8 | none | |||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (3)fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:192]: failed to create node pool "np-w5pt4m" for cluster "cluster-nightly-4-21-w5pt4m"
Unexpected error:
<*fmt.wrapErrors | 0xc000976450>:
failed to create NodePool np-w5pt4m, caused by: timeout '20.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-w5pt4m in resource group rg-nightly-4-21-x7zgfd, error: failed waiting for nodepool="np-w5pt4m" for cluster "cluster-nightly-4-21-w5pt4m" in resourcegroup="rg-nightly-4-21-x7zgfd" to finish creating: context deadline exceeded
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:192]: failed to create node pool "np-w5pt4m" for cluster "cluster-nightly-4-21-w5pt4m"
Unexpected error:
<*fmt.wrapErrors | 0xc000976450>:
failed to create NodePool np-w5pt4m, caused by: timeout '20.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-w5pt4m in resource group rg-nightly-4-21-x7zgfd, error: failed waiting for nodepool="np-w5pt4m" for cluster "cluster-nightly-4-21-w5pt4m" in resourcegroup="rg-nightly-4-21-x7zgfd" to finish creating: context deadline exceeded
...
occurredfail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:192]: failed to create node pool "np-t2zvsb" for cluster "cluster-nightly-4-22-t2zvsb"
Unexpected error:
<*fmt.wrapErrors | 0xc000c15050>:
failed to create NodePool np-t2zvsb, caused by: timeout '20.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-t2zvsb in resource group rg-nightly-4-22-fsszt4, error: failed waiting for nodepool="np-t2zvsb" for cluster "cluster-nightly-4-22-t2zvsb" in resourcegroup="rg-nightly-4-22-fsszt4" to finish creating: context deadline exceeded
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:192]: failed to create node pool "np-t2zvsb" for cluster "cluster-nightly-4-22-t2zvsb"
Unexpected error:
<*fmt.wrapErrors | 0xc000c15050>:
failed to create NodePool np-t2zvsb, caused by: timeout '20.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-t2zvsb in resource group rg-nightly-4-22-fsszt4, error: failed waiting for nodepool="np-t2zvsb" for cluster "cluster-nightly-4-22-t2zvsb" in resourcegroup="rg-nightly-4-22-fsszt4" to finish creating: context deadline exceeded
...
occurredfail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:190]: failed to verify nodes are Ready with Cilium CNI for cluster "cilium-cluster"
Unexpected error:
<*errors.joinError | 0xc0013d4438>:
VerifyNodesReady failed: no nodes found in the cluster
VerifyCiliumOperational failed: not all pods in kube-system namespace are running: failed to list pods in kube-system namespace: client rate limiter Wait returned an error: rate: Wait(n=1) would exceed context deadline
failed to create NodePool cilium-np, caused by: timeout '20.000000' minutes exceeded during CreateNodePoolFromParam for node pool cilium-np in resource group complex-cilium-kv-5nhhnb, error: failed waiting for nodepool="cilium-np" for cluster "cilium-cluster" in resourcegroup="complex-cilium-kv-5nhhnb" to finish creating: context deadline exceeded
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:190]: failed to verify nodes are Ready with Cilium CNI for cluster "cilium-cluster"
Unexpected error:
<*errors.joinError | 0xc0013d4438>:
VerifyNodesReady failed: no nodes found in the cluster
VerifyCiliumOperational failed: not all pods in kube-system namespace are running: failed to list pods in kube-system namespace: client rate limiter Wait returned an error: rate: Wait(n=1) would exceed context deadline
failed to create NodePool cilium-np, caused by: timeout '20.000000' minutes exceeded during CreateNodePoolFromParam for node pool cilium-np in resource group complex-cilium-kv-5nhhnb, error: failed waiting for nodepool="cilium-np" for cluster "cilium-cluster" in resourcegroup="complex-cilium-kv-5nhhnb" to finish creating: context deadline exceeded
...
occurredContributing tests (8)
Affected runs (2)
| |||||||||||||||||||||||||||||||||||||||||||||||||||
timeout during CreateHCPClusterAndWait; context deadline exceededfull failure pattern: timeout during CreateHCPClusterAndWait; context deadline exceeded Signal: IndeterminateSignal: Indeterminate — seen in 4 prior week(s); active 1 day(s) | e2e | 2 | 100.00%2 of 2 job runs affected | IndeterminateSignal: Indeterminate — seen in 4 prior week(s); active 1 day(s) | Show trend detailsJun 8: 0 · Jun 9: 0 · Jun 10: 0 · Jun 11: 0 · Jun 12: 0 · Jun 13: 0 · Jun 14: 8 | none | |||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (3)fail [github.com/Azure/ARO-HCP/test/e2e/idms_lifecycle.go:112]: failed to create HCP cluster with ImageDigestMirrors
Unexpected error:
<*fmt.wrapErrors | 0xc001512570>:
failed to create HCP cluster idms-e2e-hcp-cluster, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster idms-e2e-hcp-cluster in resource group idms-tn428m, error: failed waiting for cluster="idms-e2e-hcp-cluster" in resourcegroup="idms-tn428m" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster idms-e2e-hcp-cluster in resource group idms-tn428m, error: context deadline exceeded
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/idms_lifecycle.go:112]: failed to create HCP cluster with ImageDigestMirrors
Unexpected error:
<*fmt.wrapErrors | 0xc001512570>:
failed to create HCP cluster idms-e2e-hcp-cluster, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster idms-e2e-hcp-cluster in resource group idms-tn428m, error: failed waiting for cluster="idms-e2e-hcp-cluster" in resourcegroup="idms-tn428m" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster idms-e2e-hcp-cluster in resource group idms-tn428m, error: context deadline exceeded
...
occurredfail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:126]: Swift cluster rg-nightly-4-23-v2vrn5/cluster-nightly-4-23-qz5lb5 should provision
Unexpected error:
<*fmt.wrapErrors | 0xc001756060>:
failed waiting for cluster="cluster-nightly-4-23-qz5lb5" in resourcegroup="rg-nightly-4-23-v2vrn5" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223AndWait for cluster cluster-nightly-4-23-qz5lb5 in resource group rg-nightly-4-23-v2vrn5, error: context deadline exceeded
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:126]: Swift cluster rg-nightly-4-23-v2vrn5/cluster-nightly-4-23-qz5lb5 should provision
Unexpected error:
<*fmt.wrapErrors | 0xc001756060>:
failed waiting for cluster="cluster-nightly-4-23-qz5lb5" in resourcegroup="rg-nightly-4-23-v2vrn5" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223AndWait for cluster cluster-nightly-4-23-qz5lb5 in resource group rg-nightly-4-23-v2vrn5, error: context deadline exceeded
...
occurredfail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:137]: HCP cluster rg-nightly-4-20-tvmzpn/cluster-nightly-4-20-rp6bj2 should provision
Unexpected error:
<*fmt.wrapErrors | 0xc000eb12c0>:
failed to create HCP cluster cluster-nightly-4-20-rp6bj2, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster cluster-nightly-4-20-rp6bj2 in resource group rg-nightly-4-20-tvmzpn, error: failed waiting for cluster="cluster-nightly-4-20-rp6bj2" in resourcegroup="rg-nightly-4-20-tvmzpn" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster cluster-nightly-4-20-rp6bj2 in resource group rg-nightly-4-20-tvmzpn, error: context deadline exceeded
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:137]: HCP cluster rg-nightly-4-20-tvmzpn/cluster-nightly-4-20-rp6bj2 should provision
Unexpected error:
<*fmt.wrapErrors | 0xc000eb12c0>:
failed to create HCP cluster cluster-nightly-4-20-rp6bj2, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster cluster-nightly-4-20-rp6bj2 in resource group rg-nightly-4-20-tvmzpn, error: failed waiting for cluster="cluster-nightly-4-20-rp6bj2" in resourcegroup="rg-nightly-4-20-tvmzpn" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPCluster20251223FromParam for cluster cluster-nightly-4-20-rp6bj2 in resource group rg-nightly-4-20-tvmzpn, error: context deadline exceeded
...
occurredContributing tests (8)
Affected runs (2)
| |||||||||||||||||||||||||||||||||||||||||||||||||||
ERROR CODE: InvalidTemplateDeployment; provider Microsoft.Computefull failure pattern: ERROR CODE: InvalidTemplateDeployment; provider Microsoft.Compute Signal: IndeterminateSignal: Indeterminate — no prior history; active 3 day(s) — New failure pattern — no prior history | e2e | 2 | 100.00%2 of 2 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 3 day(s) — New failure pattern — no prior history | Show trend detailsJun 8: 0 · Jun 9: 0 · Jun 10: 0 · Jun 11: 0 · Jun 12: 1 · Jun 13: 2 · Jun 14: 2 | none | |||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (2)fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:130]: failed to deploy test VM
Unexpected error:
<*fmt.wrapError | 0xc000d273a0>:
failed creating deployment "test-vm" in resourcegroup="e2e-cidr-connectivity-9xbp9m": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourcegroups/e2e-cidr-connectivity-9xbp9m/providers/Microsoft.Resources/deployments/test-vm
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidTemplateDeployment
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidTemplateDeployment",
"message": "The template deployment 'test-vm' is not valid according to the validation procedure. The following resource provider(s) - 'Microsoft.Compute/virtualMachines (2023-03-01)' reported preflight validation errors. Tracking id is '21503528-9017-4784-8156-8f2140dfcfce'. See inner errors for details.",
"details": [
{
"code": "QuotaExceeded",
"message": "Operation could not be completed as it results in exceeding approved standardDSv5Family Cores quota. Additional details - Deployment Model: Resource Manager, Location: uksouth, Current Limit: 0, Current Usage: 0, Additional Required: 2, (Minimum) New Limit Required: 2. Setup Alerts when Quota reaches threshold. Learn more at https://aka.ms/quotamonitoringalerting . Submit a request for Quota increase at https://aka.ms/ProdportalCRP/#blade/Microsoft_Azure_Capacity/UsageAndQuota.ReactView/Parameters/%7B%22subscriptionId%22:%22XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX%22,%22command%22:%22openQuotaApprovalBlade%22,%22quotas%22:[%7B%22location%22:%22uksouth%22,%22providerId%22:%22Microsoft.Compute%22,%22resourceName%22:%22standardDSv5Family%22,%22quotaRequest%22:%7B%22properties%22:%7B%22limit%22:2,%22unit%22:%22Count%22,%22name%22:%7B%22value%22:%22standardDSv5Family%22%7D%7D%7D%7D]%7D by specifying parameters listed in the ‘Details’ section for deployment to succeed. Please read more about quota limits at https://docs.microsoft.com/en-us/azure/azure-supportability/per-vm-quota-requests"
}
]
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:130]: failed to deploy test VM
Unexpected error:
<*fmt.wrapError | 0xc000d273a0>:
failed creating deployment "test-vm" in resourcegroup="e2e-cidr-connectivity-9xbp9m": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourcegroups/e2e-cidr-connectivity-9xbp9m/providers/Microsoft.Resources/deployments/test-vm
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidTemplateDeployment
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidTemplateDeployment",
"message": "The template deployment 'test-vm' is not valid according to the validation procedure. The following resource provider(s) - 'Microsoft.Compute/virtualMachines (2023-03-01)' reported preflight validation errors. Tracking id is '21503528-9017-4784-8156-8f2140dfcfce'. See inner errors for details.",
"details": [
{
"code": "QuotaExceeded",
"message": "Operation could not be completed as it results in exceeding approved standardDSv5Family Cores quota. Additional details - Deployment Model: Resource Manager, Location: uksouth, Current Limit: 0, Current Usage: 0, Additional Required: 2, (Minimum) New Limit Required: 2. Setup Alerts when Quota reaches threshold. Learn more at https://aka.ms/quotamonitoringalerting . Submit a request for Quota increase at https://aka.ms/ProdportalCRP/#blade/Microsoft_Azure_Capacity/UsageAndQuota.ReactView/Parameters/%7B%22subscriptionId%22:%22XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX%22,%22command%22:%22openQuotaApprovalBlade%22,%22quotas%22:[%7B%22location%22:%22uksouth%22,%22providerId%22:%22Microsoft.Compute%22,%22resourceName%22:%22standardDSv5Family%22,%22quotaRequest%22:%7B%22properties%22:%7B%22limit%22:2,%22unit%22:%22Count%22,%22name%22:%7B%22value%22:%22standardDSv5Family%22%7D%7D%7D%7D]%7D by specifying parameters listed in the ‘Details’ section for deployment to succeed. Please read more about quota limits at https://docs.microsoft.com/en-us/azure/azure-supportability/per-vm-quota-requests"
}
]
}
}
--------------------------------------------------------------------------------
...
occurredfail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:130]: failed to deploy test VM
Unexpected error:
<*fmt.wrapError | 0xc0000d5ae0>:
failed creating deployment "test-vm" in resourcegroup="e2e-cidr-connectivity-7ktm25": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourcegroups/e2e-cidr-connectivity-7ktm25/providers/Microsoft.Resources/deployments/test-vm
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidTemplateDeployment
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidTemplateDeployment",
"message": "The template deployment 'test-vm' is not valid according to the validation procedure. The following resource provider(s) - 'Microsoft.Compute/virtualMachines (2023-03-01)' reported preflight validation errors. Tracking id is 'df96e1a8-3336-4b6e-8a5f-4f6ea39c547e'. See inner errors for details.",
"details": [
{
"code": "QuotaExceeded",
"message": "Operation could not be completed as it results in exceeding approved standardDSv5Family Cores quota. Additional details - Deployment Model: Resource Manager, Location: uksouth, Current Limit: 0, Current Usage: 0, Additional Required: 2, (Minimum) New Limit Required: 2. Setup Alerts when Quota reaches threshold. Learn more at https://aka.ms/quotamonitoringalerting . Submit a request for Quota increase at https://aka.ms/ProdportalCRP/#blade/Microsoft_Azure_Capacity/UsageAndQuota.ReactView/Parameters/%7B%22subscriptionId%22:%22XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX%22,%22command%22:%22openQuotaApprovalBlade%22,%22quotas%22:[%7B%22location%22:%22uksouth%22,%22providerId%22:%22Microsoft.Compute%22,%22resourceName%22:%22standardDSv5Family%22,%22quotaRequest%22:%7B%22properties%22:%7B%22limit%22:2,%22unit%22:%22Count%22,%22name%22:%7B%22value%22:%22standardDSv5Family%22%7D%7D%7D%7D]%7D by specifying parameters listed in the ‘Details’ section for deployment to succeed. Please read more about quota limits at https://docs.microsoft.com/en-us/azure/azure-supportability/per-vm-quota-requests"
}
]
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:130]: failed to deploy test VM
Unexpected error:
<*fmt.wrapError | 0xc0000d5ae0>:
failed creating deployment "test-vm" in resourcegroup="e2e-cidr-connectivity-7ktm25": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourcegroups/e2e-cidr-connectivity-7ktm25/providers/Microsoft.Resources/deployments/test-vm
--------------------------------------------------------------------------------
RESPONSE 400: 400 Bad Request
ERROR CODE: InvalidTemplateDeployment
--------------------------------------------------------------------------------
{
"error": {
"code": "InvalidTemplateDeployment",
"message": "The template deployment 'test-vm' is not valid according to the validation procedure. The following resource provider(s) - 'Microsoft.Compute/virtualMachines (2023-03-01)' reported preflight validation errors. Tracking id is 'df96e1a8-3336-4b6e-8a5f-4f6ea39c547e'. See inner errors for details.",
"details": [
{
"code": "QuotaExceeded",
"message": "Operation could not be completed as it results in exceeding approved standardDSv5Family Cores quota. Additional details - Deployment Model: Resource Manager, Location: uksouth, Current Limit: 0, Current Usage: 0, Additional Required: 2, (Minimum) New Limit Required: 2. Setup Alerts when Quota reaches threshold. Learn more at https://aka.ms/quotamonitoringalerting . Submit a request for Quota increase at https://aka.ms/ProdportalCRP/#blade/Microsoft_Azure_Capacity/UsageAndQuota.ReactView/Parameters/%7B%22subscriptionId%22:%22XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX%22,%22command%22:%22openQuotaApprovalBlade%22,%22quotas%22:[%7B%22location%22:%22uksouth%22,%22providerId%22:%22Microsoft.Compute%22,%22resourceName%22:%22standardDSv5Family%22,%22quotaRequest%22:%7B%22properties%22:%7B%22limit%22:2,%22unit%22:%22Count%22,%22name%22:%7B%22value%22:%22standardDSv5Family%22%7D%7D%7D%7D]%7D by specifying parameters listed in the ‘Details’ section for deployment to succeed. Please read more about quota limits at https://docs.microsoft.com/en-us/azure/azure-supportability/per-vm-quota-requests"
}
]
}
}
--------------------------------------------------------------------------------
...
occurredContributing tests (2)
Affected runs (2)
| |||||||||||||||||||||||||||||||||||||||||||||||||||