No failure patterns match your search.
Environment: PROD
Failure patterns: 5 · Failures matched: 12 · Total runs: 1 · Runs affected: 1
| Failure patternThe canonical description of a recurring CI failure, extracted and normalized from raw logs. | Failed atThe stage of the job run where this failure occurred: 'provision' (environment setup, DEV only), 'e2e' (test suite execution), or 'other' (CI infrastructure issues that did not produce a failure pattern). | Number of distinct job runs where this failure pattern was detected. | Percentage of all job runs in this environment affected by this failure pattern during the selected window. | Classification of this failure pattern: Regression (likely caused by a specific PR), Flake (intermittent failure spread across days), Noise (low-quality or generic pattern), or Indeterminate. | TrendShows daily activity for this failure pattern in a trailing window anchored to the selected end date. The sparkline covers at least 7 days and at most 14 days, depending on the current window size. | Also inOther environments where the same failure pattern was also detected during the selected window. | ||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Deserializaion Error: no output from commandfull failure pattern: Deserializaion Error: no output from command Signal: NoiseSignal: Noise — source deserialization/no-output error — New failure pattern — no prior history | e2e | 1 | 100.00%1 of 1 job runs affected | NoiseSignal: Noise — source deserialization/no-output error — New failure pattern — no prior history | Show trend detailsFeb 21: 0 · Feb 22: 0 · Feb 23: 0 · Feb 24: 0 · Feb 25: 0 · Feb 26: 0 · Feb 27: 2 | none | ||||||||||||||||||||||||||||||
Full failure examples (2): 1h29m59.786s)
/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:39
In [It] (Node Runtime: 1h29m59.786s)
/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:39
At [By Step] creating the node pool (Step Runtime: 43m10.278s)
/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:103
Spec Goroutine
goroutine 112 [select]
github.com/Azure/azure-sdk-for-go/sdk/azcore/internal/shared.Delay({0x820b858, 0xc000dd0550}, 0x77f1e58?)
/go/pkg/mod/github.com/!azure/azure-sdk-for-go/sdk/azcore@v1.21.0/internal/shared/shared.go:36
github.com/Azure/azure-sdk-for-go/sdk/azcore/runtime.(*Poller[...]).PollUntilDone(0x824f540, {0x820b858, 0xc000dd0550}, 0x0)
/go/pkg/mod/github.com/!azure/azure-sdk-for-go/sdk/azcore@v1.21.0/runtime/poller.go:284
github.com/Azure/ARO-HCP/test/util/framework.CreateNodePoolAndWait({0x820b8c8, 0xc000705490}, 0xc0000976f8, {0xc0015469c0, 0x1b}, {0x782be91, 0x18}, {0x77d2568, 0x4}, {0xc0017f1a80, ...}, ...)
/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/hcp_helper.go:718
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).CreateNodePoolFromParam(0xc000deb8c0, {0x7f2a7c514b98, 0xc000a42540}, {0xc0015469c0, 0x1b}, {0x782be91, 0x18}, {{0x77d5c37, 0x6}, {0x782be91, ...}, ...}, ...)
/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/deployment_helper.go:384
> github.com/Azure/ARO-HCP/test/e2e.init.func13.2({0x7f2a7c514b98, 0xc000a42540})
/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:108
| nodePoolParams.NodePoolName = customerNodePoolName
|
> err = tc.CreateNodePoolFromParam(ctx,
| *resourceGroup.Name,
| customerClusterName,
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x82210c0?, 0xc000a42540?})
/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 109
/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888
------------------------------
[FAILED] in [It] - /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:114 @ 02/27/26 03:34:05.194
"ts"="2026-02-27 03:34:05.194982" "level"=0 "msg"="Commiting timing metadata."
"ts"="2026-02-27 03:34:05.195051" "level"=0 "msg"="Dumping deployment operations." "deployment"="customer-infra-tls-endpoint-hcp-cluster-t8phmj" "resourceGroup"="tls-endpoint-cluster-8p26lt"
"ts"="2026-02-27 03:34:05.722278" "level"=0 "msg"="Dumping deployment operations." "deployment"="mi-tls-endpoint-hcp-cluster-t8phmj" "resourceGroup"=""
"ts"="2026-02-27 03:34:07.361055" "level"=0 "msg"="Wrote timing metadata" "path"="/tmp/secret/timing-metadata-81f205190df882aeed9c971ba5a51da44c00eb526b484755681d39236b7f3906.yaml.gz"
"ts"="2026-02-27 03:34:07.361174" "level"=0 "msg"="Wrote timing metadata" "path"="/logs/artifacts/test-timing/timing-metadata-81f205190df882aeed9c971ba5a51da44c00eb526b484755681d39236b7f3906.yaml.gz"
"ts"="2026-02-27 03:34:07.361242" "level"=0 "msg"="collecting debug info"
"ts"="2026-02-27 03:34:07.362196" "level"=0 "msg"="collecting deployments" "resourceGroup"="aro-hcp-test-msi-containers-prod-65"
"ts"="2026-02-27 03:34:07.362211" "level"=0 "msg"="collecting deployments" "resourceGroup"="tls-endpoint-cluster-8p26lt"
"ts"="2026-02-27 03:34:07.531799" "level"=0 "msg"="collecting operations" "resourceGroup"="tls-endpoint-cluster-8p26lt" "deployment"="pooledNonMsiScopedAssignments-oo7pyvaeowk44"
"ts"="2026-02-27 03:34:07.673589" "level"=0 "msg"="collecting operations" "resourceGroup"="aro-hcp-test-msi-containers-prod-65" "deployment"="pooledMsiScopedAssignments"
"ts"="2026-02-27 03:34:07.799985" "level"=0 "msg"="collecting operations" "resourceGroup"="aro-hcp-test-msi-containers-prod-65" "deployment"="msi-deployment-65"
"ts"="2026-02-27 03:34:08.041514" "level"=0 "msg"="collecting operations" "resourceGroup"="tls-endpoint-cluster-8p26lt" "deployment"="customer-infra-tls-endpoint-hcp-cluster-t8phmj"
"ts"="2026-02-27 03:34:08.198233" "level"=0 "msg"="finished collecting debug info"
"ts"="2026-02-27 03:34:08.198349" "level"=0 "msg"="deleting created resources"
"ts"="2026-02-27 03:34:08.198504" "level"=0 "msg"="deleting all hcp clusters in resource group" "resourceGroup"="tls-endpoint-cluster-8p26lt"
Command Error: signal: killed
Deserializaion Error: no output from command
: 1h29m59.786s)
/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:39
In [It] (Node Runtime: 1h29m59.786s)
/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:39
At [By Step] creating the node pool (Step Runtime: 43m10.278s)
/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:103
Spec Goroutine
goroutine 112 [select]
github.com/Azure/azure-sdk-for-go/sdk/azcore/internal/shared.Delay({0x820b858, 0xc000dd0550}, 0x77f1e58?)
/go/pkg/mod/github.com/!azure/azure-sdk-for-go/sdk/azcore@v1.21.0/internal/shared/shared.go:36
github.com/Azure/azure-sdk-for-go/sdk/azcore/runtime.(*Poller[...]).PollUntilDone(0x824f540, {0x820b858, 0xc000dd0550}, 0x0)
/go/pkg/mod/github.com/!azure/azure-sdk-for-go/sdk/azcore@v1.21.0/runtime/poller.go:284
github.com/Azure/ARO-HCP/test/util/framework.CreateNodePoolAndWait({0x820b8c8, 0xc000705490}, 0xc0000976f8, {0xc0015469c0, 0x1b}, {0x782be91, 0x18}, {0x77d2568, 0x4}, {0xc0017f1a80, ...}, ...)
/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/hcp_helper.go:718
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).CreateNodePoolFromParam(0xc000deb8c0, {0x7f2a7c514b98, 0xc000a42540}, {0xc0015469c0, 0x1b}, {0x782be91, 0x18}, {{0x77d5c37, 0x6}, {0x782be91, ...}, ...}, ...)
/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/deployment_helper.go:384
> github.com/Azure/ARO-HCP/test/e2e.init.func13.2({0x7f2a7c514b98, 0xc000a42540})
/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:108
| nodePoolParams.NodePoolName = customerNodePoolName
|
> err = tc.CreateNodePoolFromParam(ctx,
| *resourceGroup.Name,
| customerClusterName,
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x82210c0?, 0xc000a42540?})
/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 109
/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888
------------------------------
[FAILED] in [It] - /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:114 @ 02/27/26 03:34:05.194
"ts"="2026-02-27 03:34:05.194982" "level"=0 "msg"="Commiting timing metadata."
"ts"="2026-02-27 03:34:05.195051" "level"=0 "msg"="Dumping deployment operations." "deployment"="customer-infra-tls-endpoint-hcp-cluster-t8phmj" "resourceGroup"="tls-endpoint-cluster-8p26lt"
"ts"="2026-02-27 03:34:05.722278" "level"=0 "msg"="Dumping deployment operations." "deployment"="mi-tls-endpoint-hcp-cluster-t8phmj" "resourceGroup"=""
"ts"="2026-02-27 03:34:07.361055" "level"=0 "msg"="Wrote timing metadata" "path"="/tmp/secret/timing-metadata-81f205190df882aeed9c971ba5a51da44c00eb526b484755681d39236b7f3906.yaml.gz"
"ts"="2026-02-27 03:34:07.361174" "level"=0 "msg"="Wrote timing metadata" "path"="/logs/artifacts/test-timing/timing-metadata-81f205190df882aeed9c971ba5a51da44c00eb526b484755681d39236b7f3906.yaml.gz"
"ts"="2026-02-27 03:34:07.361242" "level"=0 "msg"="collecting debug info"
"ts"="2026-02-27 03:34:07.362196" "level"=0 "msg"="collecting deployments" "resourceGroup"="aro-hcp-test-msi-containers-prod-65"
"ts"="2026-02-27 03:34:07.362211" "level"=0 "msg"="collecting deployments" "resourceGroup"="tls-endpoint-cluster-8p26lt"
"ts"="2026-02-27 03:34:07.531799" "level"=0 "msg"="collecting operations" "resourceGroup"="tls-endpoint-cluster-8p26lt" "deployment"="pooledNonMsiScopedAssignments-oo7pyvaeowk44"
"ts"="2026-02-27 03:34:07.673589" "level"=0 "msg"="collecting operations" "resourceGroup"="aro-hcp-test-msi-containers-prod-65" "deployment"="pooledMsiScopedAssignments"
"ts"="2026-02-27 03:34:07.799985" "level"=0 "msg"="collecting operations" "resourceGroup"="aro-hcp-test-msi-containers-prod-65" "deployment"="msi-deployment-65"
"ts"="2026-02-27 03:34:08.041514" "level"=0 "msg"="collecting operations" "resourceGroup"="tls-endpoint-cluster-8p26lt" "deployment"="customer-infra-tls-endpoint-hcp-cluster-t8phmj"
"ts"="2026-02-27 03:34:08.198233" "level"=0 "msg"="finished collecting debug info"
"ts"="2026-02-27 03:34:08.198349" "level"=0 "msg"="deleting created resources"
"ts"="2026-02-27 03:34:08.198504" "level"=0 "msg"="deleting all hcp clusters in resource group" "resourceGroup"="tls-endpoint-cluster-8p26lt"
Command Error: signal: killed
Deserializaion Error: no output from commandd a nodepool with autoscaling enabled (Spec Runtime: 1h29m59.956s)
/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:36
In [It] (Node Runtime: 1h29m59.956s)
/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:36
At [By Step] creating the autoscaling nodepool (Step Runtime: 43m12.88s)
/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:95
Spec Goroutine
goroutine 115 [select]
github.com/Azure/azure-sdk-for-go/sdk/azcore/internal/shared.Delay({0x820b858, 0xc000b222d0}, 0x77f1e58?)
/go/pkg/mod/github.com/!azure/azure-sdk-for-go/sdk/azcore@v1.21.0/internal/shared/shared.go:36
github.com/Azure/azure-sdk-for-go/sdk/azcore/runtime.(*Poller[...]).PollUntilDone(0x824f540, {0x820b858, 0xc000b222d0}, 0x0)
/go/pkg/mod/github.com/!azure/azure-sdk-for-go/sdk/azcore@v1.21.0/runtime/poller.go:284
github.com/Azure/ARO-HCP/test/util/framework.CreateNodePoolAndWait({0x820b8c8, 0xc000306b60}, 0xc000097788, {0xc000b3c0f0, 0x15}, {0x78182bc, 0x14}, {0x77f1e34, 0xc}, {0xc000809e30, ...}, ...)
/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/hcp_helper.go:718
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).CreateNodePoolFromParam(0xc00087de60, {0x7f084c347788, 0xc00084cd80}, {0xc000b3c0f0, 0x15}, {0x78182bc, 0x14}, {{0x77d5c37, 0x6}, {0x78182bc, ...}, ...}, ...)
/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/deployment_helper.go:384
> github.com/Azure/ARO-HCP/test/e2e.init.func23.2({0x7f084c347788, 0xc00084cd80})
/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:96
|
| By("creating the autoscaling nodepool")
> err = tc.CreateNodePoolFromParam(ctx,
| *resourceGroup.Name,
| customerClusterName,
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x82210c0?, 0xc00084cd80?})
/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 112
/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888
------------------------------
[FAILED] in [It] - /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:102 @ 02/27/26 03:34:06.87
"ts"="2026-02-27 03:34:06.870227" "level"=0 "msg"="Commiting timing metadata."
"ts"="2026-02-27 03:34:06.870286" "level"=0 "msg"="Dumping deployment operations." "deployment"="customer-infra-np-autoscale-cluster-vt8brb" "resourceGroup"="np-autoscaling-9pd7cm"
"ts"="2026-02-27 03:34:07.029064" "level"=0 "msg"="Dumping deployment operations." "deployment"="mi-np-autoscale-cluster-vt8brb" "resourceGroup"=""
"ts"="2026-02-27 03:34:08.813086" "level"=0 "msg"="Wrote timing metadata" "path"="/tmp/secret/timing-metadata-3ace844abea5a4d3a28ffc64801a1d716d83f775e2e8d3dceadd0cb72b88474b.yaml.gz"
"ts"="2026-02-27 03:34:08.813201" "level"=0 "msg"="Wrote timing metadata" "path"="/logs/artifacts/test-timing/timing-metadata-3ace844abea5a4d3a28ffc64801a1d716d83f775e2e8d3dceadd0cb72b88474b.yaml.gz"
"ts"="2026-02-27 03:34:08.813273" "level"=0 "msg"="collecting debug info"
"ts"="2026-02-27 03:34:08.814251" "level"=0 "msg"="collecting deployments" "resourceGroup"="aro-hcp-test-msi-containers-prod-10"
"ts"="2026-02-27 03:34:08.814264" "level"=0 "msg"="collecting deployments" "resourceGroup"="np-autoscaling-9pd7cm"
"ts"="2026-02-27 03:34:09.189947" "level"=0 "msg"="collecting operations" "resourceGroup"="np-autoscaling-9pd7cm" "deployment"="pooledNonMsiScopedAssignments-ssrwqrnhyyabs"
"ts"="2026-02-27 03:34:09.337198" "level"=0 "msg"="collecting operations" "resourceGroup"="aro-hcp-test-msi-containers-prod-10" "deployment"="pooledMsiScopedAssignments"
"ts"="2026-02-27 03:34:09.337744" "level"=0 "msg"="collecting operations" "resourceGroup"="np-autoscaling-9pd7cm" "deployment"="customer-infra-np-autoscale-cluster-vt8brb"
"ts"="2026-02-27 03:34:09.474232" "level"=0 "msg"="collecting operations" "resourceGroup"="aro-hcp-test-msi-containers-prod-10" "deployment"="msi-deployment-10"
"ts"="2026-02-27 03:34:09.613226" "level"=0 "msg"="finished collecting debug info"
"ts"="2026-02-27 03:34:09.613375" "level"=0 "msg"="deleting created resources"
"ts"="2026-02-27 03:34:09.613494" "level"=0 "msg"="deleting all hcp clusters in resource group" "resourceGroup"="np-autoscaling-9pd7cm"
Command Error: signal: killed
Deserializaion Error: no output from command
d a nodepool with autoscaling enabled (Spec Runtime: 1h29m59.956s)
/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:36
In [It] (Node Runtime: 1h29m59.956s)
/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:36
At [By Step] creating the autoscaling nodepool (Step Runtime: 43m12.88s)
/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:95
Spec Goroutine
goroutine 115 [select]
github.com/Azure/azure-sdk-for-go/sdk/azcore/internal/shared.Delay({0x820b858, 0xc000b222d0}, 0x77f1e58?)
/go/pkg/mod/github.com/!azure/azure-sdk-for-go/sdk/azcore@v1.21.0/internal/shared/shared.go:36
github.com/Azure/azure-sdk-for-go/sdk/azcore/runtime.(*Poller[...]).PollUntilDone(0x824f540, {0x820b858, 0xc000b222d0}, 0x0)
/go/pkg/mod/github.com/!azure/azure-sdk-for-go/sdk/azcore@v1.21.0/runtime/poller.go:284
github.com/Azure/ARO-HCP/test/util/framework.CreateNodePoolAndWait({0x820b8c8, 0xc000306b60}, 0xc000097788, {0xc000b3c0f0, 0x15}, {0x78182bc, 0x14}, {0x77f1e34, 0xc}, {0xc000809e30, ...}, ...)
/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/hcp_helper.go:718
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).CreateNodePoolFromParam(0xc00087de60, {0x7f084c347788, 0xc00084cd80}, {0xc000b3c0f0, 0x15}, {0x78182bc, 0x14}, {{0x77d5c37, 0x6}, {0x78182bc, ...}, ...}, ...)
/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/deployment_helper.go:384
> github.com/Azure/ARO-HCP/test/e2e.init.func23.2({0x7f084c347788, 0xc00084cd80})
/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:96
|
| By("creating the autoscaling nodepool")
> err = tc.CreateNodePoolFromParam(ctx,
| *resourceGroup.Name,
| customerClusterName,
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x82210c0?, 0xc00084cd80?})
/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 112
/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888
------------------------------
[FAILED] in [It] - /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/nodepool_autoscaling.go:102 @ 02/27/26 03:34:06.87
"ts"="2026-02-27 03:34:06.870227" "level"=0 "msg"="Commiting timing metadata."
"ts"="2026-02-27 03:34:06.870286" "level"=0 "msg"="Dumping deployment operations." "deployment"="customer-infra-np-autoscale-cluster-vt8brb" "resourceGroup"="np-autoscaling-9pd7cm"
"ts"="2026-02-27 03:34:07.029064" "level"=0 "msg"="Dumping deployment operations." "deployment"="mi-np-autoscale-cluster-vt8brb" "resourceGroup"=""
"ts"="2026-02-27 03:34:08.813086" "level"=0 "msg"="Wrote timing metadata" "path"="/tmp/secret/timing-metadata-3ace844abea5a4d3a28ffc64801a1d716d83f775e2e8d3dceadd0cb72b88474b.yaml.gz"
"ts"="2026-02-27 03:34:08.813201" "level"=0 "msg"="Wrote timing metadata" "path"="/logs/artifacts/test-timing/timing-metadata-3ace844abea5a4d3a28ffc64801a1d716d83f775e2e8d3dceadd0cb72b88474b.yaml.gz"
"ts"="2026-02-27 03:34:08.813273" "level"=0 "msg"="collecting debug info"
"ts"="2026-02-27 03:34:08.814251" "level"=0 "msg"="collecting deployments" "resourceGroup"="aro-hcp-test-msi-containers-prod-10"
"ts"="2026-02-27 03:34:08.814264" "level"=0 "msg"="collecting deployments" "resourceGroup"="np-autoscaling-9pd7cm"
"ts"="2026-02-27 03:34:09.189947" "level"=0 "msg"="collecting operations" "resourceGroup"="np-autoscaling-9pd7cm" "deployment"="pooledNonMsiScopedAssignments-ssrwqrnhyyabs"
"ts"="2026-02-27 03:34:09.337198" "level"=0 "msg"="collecting operations" "resourceGroup"="aro-hcp-test-msi-containers-prod-10" "deployment"="pooledMsiScopedAssignments"
"ts"="2026-02-27 03:34:09.337744" "level"=0 "msg"="collecting operations" "resourceGroup"="np-autoscaling-9pd7cm" "deployment"="customer-infra-np-autoscale-cluster-vt8brb"
"ts"="2026-02-27 03:34:09.474232" "level"=0 "msg"="collecting operations" "resourceGroup"="aro-hcp-test-msi-containers-prod-10" "deployment"="msi-deployment-10"
"ts"="2026-02-27 03:34:09.613226" "level"=0 "msg"="finished collecting debug info"
"ts"="2026-02-27 03:34:09.613375" "level"=0 "msg"="deleting created resources"
"ts"="2026-02-27 03:34:09.613494" "level"=0 "msg"="deleting all hcp clusters in resource group" "resourceGroup"="np-autoscaling-9pd7cm"
Command Error: signal: killed
Deserializaion Error: no output from commandContributing tests (2)
Affected runs (1)
| ||||||||||||||||||||||||||||||||||||
CreateNodePoolFromParam timed out after <minutes> minutes; context deadline exceededfull failure pattern: CreateNodePoolFromParam timed out after <minutes> minutes; context deadline exceeded Signal: IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | e2e | 1 | 100.00%1 of 1 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | Show trend detailsFeb 21: 0 · Feb 22: 0 · Feb 23: 0 · Feb 24: 0 · Feb 25: 0 · Feb 26: 0 · Feb 27: 5 | none | ||||||||||||||||||||||||||||||
Full failure examples (3)fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:124]: Unexpected error:
<*fmt.wrapErrors | 0xc0011d2e70>:
failed to create NodePool np-1, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group autoscaling-cluster-h5r2qd, error: failed waiting for nodepool="np-1" for cluster "autoscaling-hcp-cluster" in resourcegroup="autoscaling-cluster-h5r2qd" to finish creating: context deadline exceeded
{
msg: "failed to create NodePool np-1, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group autoscaling-cluster-h5r2qd, error: failed waiting for nodepool=\"np-1\" for cluster \"autoscaling-hcp-cluster\" in resourcegroup=\"autoscaling-cluster-h5r2qd\" to finish creating: context deadline exceeded",
errs: [
<*errors.errorString | 0xc00068ba00>{
s: "timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group autoscaling-cluster-h5r2qd",
},
<*fmt.wrapError | 0xc000b8ad80>{
msg: "failed waiting for nodepool=\"np-1\" for cluster \"autoscaling-hcp-cluster\" in resourcegroup=\"autoscaling-cluster-h5r2qd\" to finish creating: context deadline exceeded",
err: <context.deadlineExceededError>{},
},
],
}
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:124]: Unexpected error:
<*fmt.wrapErrors | 0xc0011d2e70>:
failed to create NodePool np-1, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group autoscaling-cluster-h5r2qd, error: failed waiting for nodepool="np-1" for cluster "autoscaling-hcp-cluster" in resourcegroup="autoscaling-cluster-h5r2qd" to finish creating: context deadline exceeded
{
msg: "failed to create NodePool np-1, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group autoscaling-cluster-h5r2qd, error: failed waiting for nodepool=\"np-1\" for cluster \"autoscaling-hcp-cluster\" in resourcegroup=\"autoscaling-cluster-h5r2qd\" to finish creating: context deadline exceeded",
errs: [
<*errors.errorString | 0xc00068ba00>{
s: "timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group autoscaling-cluster-h5r2qd",
},
<*fmt.wrapError | 0xc000b8ad80>{
msg: "failed waiting for nodepool=\"np-1\" for cluster \"autoscaling-hcp-cluster\" in resourcegroup=\"autoscaling-cluster-h5r2qd\" to finish creating: context deadline exceeded",
err: <context.deadlineExceededError>{},
},
],
}
occurredfail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_nodepool_osdisk.go:94]: Unexpected error:
<*fmt.wrapErrors | 0xc0014baa20>:
failed to create NodePool nodepool-128GiB, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool nodepool-128GiB in resource group clusternp128-xkk5mj, error: failed waiting for nodepool="nodepool-128GiB" for cluster "hcp-cluster-np-128" in resourcegroup="clusternp128-xkk5mj" to finish creating: context deadline exceeded
{
msg: "failed to create NodePool nodepool-128GiB, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool nodepool-128GiB in resource group clusternp128-xkk5mj, error: failed waiting for nodepool=\"nodepool-128GiB\" for cluster \"hcp-cluster-np-128\" in resourcegroup=\"clusternp128-xkk5mj\" to finish creating: context deadline exceeded",
errs: [
<*errors.errorString | 0xc0010e7c20>{
s: "timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool nodepool-128GiB in resource group clusternp128-xkk5mj",
},
<*fmt.wrapError | 0xc0013b44a0>{
msg: "failed waiting for nodepool=\"nodepool-128GiB\" for cluster \"hcp-cluster-np-128\" in resourcegroup=\"clusternp128-xkk5mj\" to finish creating: context deadline exceeded",
err: <context.deadlineExceededError>{},
},
],
}
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_nodepool_osdisk.go:94]: Unexpected error:
<*fmt.wrapErrors | 0xc0014baa20>:
failed to create NodePool nodepool-128GiB, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool nodepool-128GiB in resource group clusternp128-xkk5mj, error: failed waiting for nodepool="nodepool-128GiB" for cluster "hcp-cluster-np-128" in resourcegroup="clusternp128-xkk5mj" to finish creating: context deadline exceeded
{
msg: "failed to create NodePool nodepool-128GiB, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool nodepool-128GiB in resource group clusternp128-xkk5mj, error: failed waiting for nodepool=\"nodepool-128GiB\" for cluster \"hcp-cluster-np-128\" in resourcegroup=\"clusternp128-xkk5mj\" to finish creating: context deadline exceeded",
errs: [
<*errors.errorString | 0xc0010e7c20>{
s: "timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool nodepool-128GiB in resource group clusternp128-xkk5mj",
},
<*fmt.wrapError | 0xc0013b44a0>{
msg: "failed waiting for nodepool=\"nodepool-128GiB\" for cluster \"hcp-cluster-np-128\" in resourcegroup=\"clusternp128-xkk5mj\" to finish creating: context deadline exceeded",
err: <context.deadlineExceededError>{},
},
],
}
occurredfail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:242]: Unexpected error:
<*fmt.wrapErrors | 0xc00176eae0>:
failed to create NodePool np-1, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group e2e-cidr-connectivity-dcdzqf, error: failed waiting for nodepool="np-1" for cluster "cidr-connectivity-test" in resourcegroup="e2e-cidr-connectivity-dcdzqf" to finish creating: context deadline exceeded
{
msg: "failed to create NodePool np-1, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group e2e-cidr-connectivity-dcdzqf, error: failed waiting for nodepool=\"np-1\" for cluster \"cidr-connectivity-test\" in resourcegroup=\"e2e-cidr-connectivity-dcdzqf\" to finish creating: context deadline exceeded",
errs: [
<*errors.errorString | 0xc001099d80>{
s: "timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group e2e-cidr-connectivity-dcdzqf",
},
<*fmt.wrapError | 0xc000a7a9c0>{
msg: "failed waiting for nodepool=\"np-1\" for cluster \"cidr-connectivity-test\" in resourcegroup=\"e2e-cidr-connectivity-dcdzqf\" to finish creating: context deadline exceeded",
err: <context.deadlineExceededError>{},
},
],
}
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:242]: Unexpected error:
<*fmt.wrapErrors | 0xc00176eae0>:
failed to create NodePool np-1, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group e2e-cidr-connectivity-dcdzqf, error: failed waiting for nodepool="np-1" for cluster "cidr-connectivity-test" in resourcegroup="e2e-cidr-connectivity-dcdzqf" to finish creating: context deadline exceeded
{
msg: "failed to create NodePool np-1, caused by: timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group e2e-cidr-connectivity-dcdzqf, error: failed waiting for nodepool=\"np-1\" for cluster \"cidr-connectivity-test\" in resourcegroup=\"e2e-cidr-connectivity-dcdzqf\" to finish creating: context deadline exceeded",
errs: [
<*errors.errorString | 0xc001099d80>{
s: "timeout '45.000000' minutes exceeded during CreateNodePoolFromParam for node pool np-1 in resource group e2e-cidr-connectivity-dcdzqf",
},
<*fmt.wrapError | 0xc000a7a9c0>{
msg: "failed waiting for nodepool=\"np-1\" for cluster \"cidr-connectivity-test\" in resourcegroup=\"e2e-cidr-connectivity-dcdzqf\" to finish creating: context deadline exceeded",
err: <context.deadlineExceededError>{},
},
],
}
occurredContributing tests (5)
Affected runs (1)
| ||||||||||||||||||||||||||||||||||||
timeout during GetAdminRESTConfigForHCPCluster while waiting for hcpcluster credsfull failure pattern: timeout during GetAdminRESTConfigForHCPCluster while waiting for hcpcluster creds Signal: IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | e2e | 1 | 100.00%1 of 1 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | Show trend detailsFeb 21: 0 · Feb 22: 0 · Feb 23: 0 · Feb 24: 0 · Feb 25: 0 · Feb 26: 0 · Feb 27: 3 | none | ||||||||||||||||||||||||||||||
Full failure examples (3)fail [github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:156]: Unexpected error:
<*fmt.wrapErrors | 0xc00118aa20>:
failed waiting for hcpCluster="cluster-ver-4-19" in resourcegroup="rg-cluster-back-version-fqjfbz" to finish getting creds, caused by: timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster cluster-ver-4-19 in resource group rg-cluster-back-version-fqjfbz, error: context deadline exceeded
{
msg: "failed waiting for hcpCluster=\"cluster-ver-4-19\" in resourcegroup=\"rg-cluster-back-version-fqjfbz\" to finish getting creds, caused by: timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster cluster-ver-4-19 in resource group rg-cluster-back-version-fqjfbz, error: context deadline exceeded",
errs: [
<*errors.errorString | 0xc0008bb430>{
s: "timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster cluster-ver-4-19 in resource group rg-cluster-back-version-fqjfbz",
},
<context.deadlineExceededError>{},
],
}
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_version_backlevel.go:156]: Unexpected error:
<*fmt.wrapErrors | 0xc00118aa20>:
failed waiting for hcpCluster="cluster-ver-4-19" in resourcegroup="rg-cluster-back-version-fqjfbz" to finish getting creds, caused by: timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster cluster-ver-4-19 in resource group rg-cluster-back-version-fqjfbz, error: context deadline exceeded
{
msg: "failed waiting for hcpCluster=\"cluster-ver-4-19\" in resourcegroup=\"rg-cluster-back-version-fqjfbz\" to finish getting creds, caused by: timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster cluster-ver-4-19 in resource group rg-cluster-back-version-fqjfbz, error: context deadline exceeded",
errs: [
<*errors.errorString | 0xc0008bb430>{
s: "timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster cluster-ver-4-19 in resource group rg-cluster-back-version-fqjfbz",
},
<context.deadlineExceededError>{},
],
}
occurredfail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create.go:100]: Unexpected error:
<*fmt.wrapErrors | 0xc00119a8d0>:
failed waiting for hcpCluster="basic-hcp-cluster" in resourcegroup="basic-cluster-b22djs" to finish getting creds, caused by: timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster basic-hcp-cluster in resource group basic-cluster-b22djs, error: context deadline exceeded
{
msg: "failed waiting for hcpCluster=\"basic-hcp-cluster\" in resourcegroup=\"basic-cluster-b22djs\" to finish getting creds, caused by: timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster basic-hcp-cluster in resource group basic-cluster-b22djs, error: context deadline exceeded",
errs: [
<*errors.errorString | 0xc000646250>{
s: "timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster basic-hcp-cluster in resource group basic-cluster-b22djs",
},
<context.deadlineExceededError>{},
],
}
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create.go:100]: Unexpected error:
<*fmt.wrapErrors | 0xc00119a8d0>:
failed waiting for hcpCluster="basic-hcp-cluster" in resourcegroup="basic-cluster-b22djs" to finish getting creds, caused by: timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster basic-hcp-cluster in resource group basic-cluster-b22djs, error: context deadline exceeded
{
msg: "failed waiting for hcpCluster=\"basic-hcp-cluster\" in resourcegroup=\"basic-cluster-b22djs\" to finish getting creds, caused by: timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster basic-hcp-cluster in resource group basic-cluster-b22djs, error: context deadline exceeded",
errs: [
<*errors.errorString | 0xc000646250>{
s: "timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster basic-hcp-cluster in resource group basic-cluster-b22djs",
},
<context.deadlineExceededError>{},
],
}
occurredfail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:170]: Unexpected error:
<*fmt.wrapErrors | 0xc000e9b110>:
failed waiting for hcpCluster="patch-tags-cluster" in resourcegroup="patch-tags-rxn4xb" to finish getting creds, caused by: timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster patch-tags-cluster in resource group patch-tags-rxn4xb, error: context deadline exceeded
{
msg: "failed waiting for hcpCluster=\"patch-tags-cluster\" in resourcegroup=\"patch-tags-rxn4xb\" to finish getting creds, caused by: timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster patch-tags-cluster in resource group patch-tags-rxn4xb, error: context deadline exceeded",
errs: [
<*errors.errorString | 0xc000812100>{
s: "timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster patch-tags-cluster in resource group patch-tags-rxn4xb",
},
<context.deadlineExceededError>{},
],
}
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:170]: Unexpected error:
<*fmt.wrapErrors | 0xc000e9b110>:
failed waiting for hcpCluster="patch-tags-cluster" in resourcegroup="patch-tags-rxn4xb" to finish getting creds, caused by: timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster patch-tags-cluster in resource group patch-tags-rxn4xb, error: context deadline exceeded
{
msg: "failed waiting for hcpCluster=\"patch-tags-cluster\" in resourcegroup=\"patch-tags-rxn4xb\" to finish getting creds, caused by: timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster patch-tags-cluster in resource group patch-tags-rxn4xb, error: context deadline exceeded",
errs: [
<*errors.errorString | 0xc000812100>{
s: "timeout '10.000000' minutes exceeded during GetAdminRESTConfigForHCPCluster for cluster patch-tags-cluster in resource group patch-tags-rxn4xb",
},
<context.deadlineExceededError>{},
],
}
occurredContributing tests (3)
Affected runs (1)
| ||||||||||||||||||||||||||||||||||||
CreateOrUpdateExternalAuthAndWait timed out after <minutes> minutes; context deadline exceededfull failure pattern: CreateOrUpdateExternalAuthAndWait timed out after <minutes> minutes; context deadline exceeded Signal: IndeterminateSignal: Indeterminate — no prior history; active 2 day(s) — New failure pattern — no prior history | e2e | 1 | 100.00%1 of 1 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 2 day(s) — New failure pattern — no prior history | Show trend detailsFeb 21: 0 · Feb 22: 0 · Feb 23: 0 · Feb 24: 0 · Feb 25: 0 · Feb 26: 1 · Feb 27: 1 | none | ||||||||||||||||||||||||||||||
Full failure examples (1)fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_list_and_verify.go:118]: Unexpected error:
<*fmt.wrapErrors | 0xc0013c5b30>:
failed waiting for external auth "ea-list" in resourcegroup="ea-list-dwdpnj" for cluster="ea-list677v96" to finish, caused by: timeout '15.000000' minutes exceeded during CreateOrUpdateExternalAuthAndWait for external auth ea-list in cluster ea-list677v96 in resource group ea-list-dwdpnj, error: context deadline exceeded
{
msg: "failed waiting for external auth \"ea-list\" in resourcegroup=\"ea-list-dwdpnj\" for cluster=\"ea-list677v96\" to finish, caused by: timeout '15.000000' minutes exceeded during CreateOrUpdateExternalAuthAndWait for external auth ea-list in cluster ea-list677v96 in resource group ea-list-dwdpnj, error: context deadline exceeded",
errs: [
<*errors.errorString | 0xc000718220>{
s: "timeout '15.000000' minutes exceeded during CreateOrUpdateExternalAuthAndWait for external auth ea-list in cluster ea-list677v96 in resource group ea-list-dwdpnj",
},
<context.deadlineExceededError>{},
],
}
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_list_and_verify.go:118]: Unexpected error:
<*fmt.wrapErrors | 0xc0013c5b30>:
failed waiting for external auth "ea-list" in resourcegroup="ea-list-dwdpnj" for cluster="ea-list677v96" to finish, caused by: timeout '15.000000' minutes exceeded during CreateOrUpdateExternalAuthAndWait for external auth ea-list in cluster ea-list677v96 in resource group ea-list-dwdpnj, error: context deadline exceeded
{
msg: "failed waiting for external auth \"ea-list\" in resourcegroup=\"ea-list-dwdpnj\" for cluster=\"ea-list677v96\" to finish, caused by: timeout '15.000000' minutes exceeded during CreateOrUpdateExternalAuthAndWait for external auth ea-list in cluster ea-list677v96 in resource group ea-list-dwdpnj, error: context deadline exceeded",
errs: [
<*errors.errorString | 0xc000718220>{
s: "timeout '15.000000' minutes exceeded during CreateOrUpdateExternalAuthAndWait for external auth ea-list in cluster ea-list677v96 in resource group ea-list-dwdpnj",
},
<context.deadlineExceededError>{},
],
}
occurredContributing tests (1)
Affected runs (1)
| ||||||||||||||||||||||||||||||||||||
Interrupted by Userfull failure pattern: Interrupted by User Signal: IndeterminateSignal: Indeterminate — no prior history; active 2 day(s) — New failure pattern — no prior history | e2e | 1 | 100.00%1 of 1 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 2 day(s) — New failure pattern — no prior history | Show trend detailsFeb 21: 0 · Feb 22: 0 · Feb 23: 0 · Feb 24: 0 · Feb 25: 0 · Feb 26: 1 · Feb 27: 1 | none | ||||||||||||||||||||||||||||||
Full failure examples (1)fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:47]: Interrupted by User fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:47]: Interrupted by User Contributing tests (1)
Affected runs (1)
| ||||||||||||||||||||||||||||||||||||