No failure patterns match your search.
Environment: INT
Failure patterns: 3 · Failures matched: 67 · Total runs: 7 · Runs affected: 4
| Failure patternThe canonical description of a recurring CI failure, extracted and normalized from raw logs. | Failed atThe stage of the job run where this failure occurred: 'provision' (environment setup, DEV only), 'e2e' (test suite execution), or 'other' (CI infrastructure issues that did not produce a failure pattern). | Number of distinct job runs where this failure pattern was detected. | Percentage of all job runs in this environment affected by this failure pattern during the selected window. | Classification of this failure pattern: Regression (likely caused by a specific PR), Flake (intermittent failure spread across days), Noise (low-quality or generic pattern), or Indeterminate. | TrendShows daily activity for this failure pattern in a trailing window anchored to the selected end date. The sparkline covers at least 7 days and at most 14 days, depending on the current window size. | Also inOther environments where the same failure pattern was also detected during the selected window. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Test Panickedfull failure pattern: Test Panicked Signal: IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | e2e | 3 | 42.86%3 of 7 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | Show trend detailsFeb 26: 0 · Feb 27: 0 · Feb 28: 0 · Mar 1: 0 · Mar 2: 0 · Mar 3: 0 · Mar 4: 55 | none | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (3)github.com/Azure/ARO-HCP/test/util/framework.Must[...](...)
/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000f66b90?, {0x7fc7e6d7b600?, 0xc0006e6d80?})
/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).NewResourceGroup(0xc0004f3320, {0x7fc7e6d7b600, 0xc0006e6d80}, {0xc000f586f0, 0x17}, {0xc0000140b9, 0x7})
/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5
github.com/Azure/ARO-HCP/test/e2e.init.func20.1({0x7fc7e6d7b600, 0xc0006e6d80})
/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/gpu_nodepools_create_delete.go:74 +0x20a
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc0006e6d80?})
/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b
created by github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 110
/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50
fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information
fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked
github.com/Azure/ARO-HCP/test/util/framework.Must[...](...)
/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000f66b90?, {0x7fc7e6d7b600?, 0xc0006e6d80?})
/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).NewResourceGroup(0xc0004f3320, {0x7fc7e6d7b600, 0xc0006e6d80}, {0xc000f586f0, 0x17}, {0xc0000140b9, 0x7})
/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5
github.com/Azure/ARO-HCP/test/e2e.init.func20.1({0x7fc7e6d7b600, 0xc0006e6d80})
/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/gpu_nodepools_create_delete.go:74 +0x20a
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc0006e6d80?})
/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b
created by github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 110
/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50
fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information
fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panickedgithub.com/Azure/ARO-HCP/test/util/framework.Must[...](...)
/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000088910?, {0x7fdcdc31ce00?, 0xc000ac8210?})
/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).NewResourceGroup(0xc000860b40, {0x7fdcdc31ce00, 0xc000ac8210}, {0x783a7e5, 0x1b}, {0xc0000140b9, 0x7})
/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5
github.com/Azure/ARO-HCP/test/e2e.init.func11.1({0x7fdcdc31ce00, 0xc000ac8210})
/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_nsg_subnet_reuse.go:49 +0xe9
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000ac8210?})
/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b
created by github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 92
/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50
fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information
fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked
github.com/Azure/ARO-HCP/test/util/framework.Must[...](...)
/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000088910?, {0x7fdcdc31ce00?, 0xc000ac8210?})
/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).NewResourceGroup(0xc000860b40, {0x7fdcdc31ce00, 0xc000ac8210}, {0x783a7e5, 0x1b}, {0xc0000140b9, 0x7})
/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5
github.com/Azure/ARO-HCP/test/e2e.init.func11.1({0x7fdcdc31ce00, 0xc000ac8210})
/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_nsg_subnet_reuse.go:49 +0xe9
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc000ac8210?})
/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b
created by github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 92
/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50
fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information
fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panickedgithub.com/Azure/ARO-HCP/test/util/framework.Must[...](...)
/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000c7c6c0?, {0x7fd105635138?, 0xc00096cae0?})
/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).NewResourceGroup(0xc0009970e0, {0x7fd105635138, 0xc00096cae0}, {0x77f67a9, 0xd}, {0xc0000140b9, 0x7})
/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5
github.com/Azure/ARO-HCP/test/e2e.init.func17.2({0x7fd105635138, 0xc00096cae0})
/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create.go:59 +0xe9
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc00096cae0?})
/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b
created by github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 75
/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50
fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information
fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked
github.com/Azure/ARO-HCP/test/util/framework.Must[...](...)
/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).GetARMResourcesClientFactoryOrDie(0xc000c7c6c0?, {0x7fd105635138?, 0xc00096cae0?})
/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:603 +0x37
github.com/Azure/ARO-HCP/test/util/framework.(*perItOrDescribeTestContext).NewResourceGroup(0xc0009970e0, {0x7fd105635138, 0xc00096cae0}, {0x77f67a9, 0xd}, {0xc0000140b9, 0x7})
/opt/app-root/src/github.com/Azure/ARO-HCP/test/util/framework/per_test_framework.go:367 +0x2a5
github.com/Azure/ARO-HCP/test/e2e.init.func17.2({0x7fd105635138, 0xc00096cae0})
/opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create.go:59 +0xe9
github.com/onsi/ginkgo/v2/internal.extractBodyFunction.func2({0x8221420?, 0xc00096cae0?})
/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/node.go:465 +0x3e
github.com/onsi/ginkgo/v2/internal.(*Suite).runNode.func3()
/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:901 +0x7b
created by github.com/onsi/ginkgo/v2/internal.(*Suite).runNode in goroutine 75
/go/pkg/mod/github.com/openshift/onsi-ginkgo/v2@v2.6.1-0.20250416174521-4eb003743b54/internal/suite.go:888 +0xd50
fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test Panicked: failed building environment credential: AZURE_TOKEN_CREDENTIALS must be set when RequireAzureTokenCredentials is true. See https://aka.ms/azsdk/go/identity/docs#DefaultAzureCredential for more information
fail [github.com/Azure/ARO-HCP/test/util/framework/per_invocation_framework.go:402]: Test PanickedContributing tests (22)
Affected runs (3)
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Interrupted by Userfull failure pattern: Interrupted by User Signal: IndeterminateSignal: Indeterminate — seen in 1 prior week(s); active 4 day(s) | e2e | 3 | 42.86%3 of 7 job runs affected | IndeterminateSignal: Indeterminate — seen in 1 prior week(s); active 4 day(s) | Show trend detailsFeb 26: 0 · Feb 27: 0 · Feb 28: 21 · Mar 1: 7 · Mar 2: 5 · Mar 3: 0 · Mar 4: 11 | none | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (3)fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:120]: Interrupted by User fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:120]: Interrupted by User fail [github.com/Azure/ARO-HCP/test/e2e/cluster_pullsecret.go:46]: Interrupted by User fail [github.com/Azure/ARO-HCP/test/e2e/cluster_pullsecret.go:46]: Interrupted by User fail [github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:39]: Interrupted by User fail [github.com/Azure/ARO-HCP/test/e2e/cluster_tls_endpoints.go:39]: Interrupted by User Contributing tests (10)
Affected runs (3)
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
failed to create nodepool <nodepool>: operation result model did not match expected model for type *armredhatopenshifthcp.NodePool:full failure pattern: failed to create nodepool <nodepool>: operation result model did not match expected model for type *armredhatopenshifthcp.NodePool: Signal: IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | e2e | 1 | 14.29%1 of 7 job runs affected | IndeterminateSignal: Indeterminate — no prior history; active 1 day(s) — New failure pattern — no prior history | Show trend detailsFeb 26: 0 · Feb 27: 0 · Feb 28: 0 · Mar 1: 0 · Mar 2: 0 · Mar 3: 0 · Mar 4: 1 | none | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Full failure examples (1)fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:124]: Unexpected error:
<*fmt.wrapError | 0xc0014d2ca0>:
failed to create NodePool np-1: operation result model did not match expected model for type *armredhatopenshifthcp.NodePool:
&armredhatopenshifthcp.NodePool{
Location: &"uksouth",
Identity: nil,
Properties: &armredhatopenshifthcp.NodePoolProperties{
... // 6 identical fields
Taints: nil,
Version: &{ChannelGroup: &"stable", ID: &"4.20.8"},
- ProvisioningState: &"Succeeded",
+ ProvisioningState: &"Provisioning",
},
Tags: nil,
ID: &"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGrou"...,
... // 1 ignored and 2 identical fields
}
{
msg: "failed to create NodePool np-1: operation result model did not match expected model for type *armredhatopenshifthcp.NodePool:\n &armredhatopenshifthcp.NodePool{\n \tLocation: &\"uksouth\",\n \tIdentity: nil,\n \tProperties: &armredhatopenshifthcp.NodePoolProperties{\n \t\t... // 6 identical fields\n \t\tTaints: nil,\n \t\tVersion: &{ChannelGroup: &\"stable\", ID: &\"4.20.8\"},\n- \t\tProvisioningState: &\"Succeeded\",\n+ \t\tProvisioningState: &\"Provisioning\",\n \t},\n \tTags: nil,\n \tID: &\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGrou\"...,\n \t... // 1 ignored and 2 identical fields\n }\n",
err: <*errors.errorString | 0xc0008972a0>{
s: "operation result model did not match expected model for type *armredhatopenshifthcp.NodePool:\n &armredhatopenshifthcp.NodePool{\n \tLocation: &\"uksouth\",\n \tIdentity: nil,\n \tProperties: &armredhatopenshifthcp.NodePoolProperties{\n \t\t... // 6 identical fields\n \t\tTaints: nil,\n \t\tVersion: &{ChannelGroup: &\"stable\", ID: &\"4.20.8\"},\n- \t\tProvisioningState: &\"Succeeded\",\n+ \t\tProvisioningState: &\"Provisioning\",\n \t},\n \tTags: nil,\n \tID: &\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGrou\"...,\n \t... // 1 ignored and 2 identical fields\n }\n",
},
}
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:124]: Unexpected error:
<*fmt.wrapError | 0xc0014d2ca0>:
failed to create NodePool np-1: operation result model did not match expected model for type *armredhatopenshifthcp.NodePool:
&armredhatopenshifthcp.NodePool{
Location: &"uksouth",
Identity: nil,
Properties: &armredhatopenshifthcp.NodePoolProperties{
... // 6 identical fields
Taints: nil,
Version: &{ChannelGroup: &"stable", ID: &"4.20.8"},
- ProvisioningState: &"Succeeded",
+ ProvisioningState: &"Provisioning",
},
Tags: nil,
ID: &"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGrou"...,
... // 1 ignored and 2 identical fields
}
{
msg: "failed to create NodePool np-1: operation result model did not match expected model for type *armredhatopenshifthcp.NodePool:\n &armredhatopenshifthcp.NodePool{\n \tLocation: &\"uksouth\",\n \tIdentity: nil,\n \tProperties: &armredhatopenshifthcp.NodePoolProperties{\n \t\t... // 6 identical fields\n \t\tTaints: nil,\n \t\tVersion: &{ChannelGroup: &\"stable\", ID: &\"4.20.8\"},\n- \t\tProvisioningState: &\"Succeeded\",\n+ \t\tProvisioningState: &\"Provisioning\",\n \t},\n \tTags: nil,\n \tID: &\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGrou\"...,\n \t... // 1 ignored and 2 identical fields\n }\n",
err: <*errors.errorString | 0xc0008972a0>{
s: "operation result model did not match expected model for type *armredhatopenshifthcp.NodePool:\n &armredhatopenshifthcp.NodePool{\n \tLocation: &\"uksouth\",\n \tIdentity: nil,\n \tProperties: &armredhatopenshifthcp.NodePoolProperties{\n \t\t... // 6 identical fields\n \t\tTaints: nil,\n \t\tVersion: &{ChannelGroup: &\"stable\", ID: &\"4.20.8\"},\n- \t\tProvisioningState: &\"Succeeded\",\n+ \t\tProvisioningState: &\"Provisioning\",\n \t},\n \tTags: nil,\n \tID: &\"/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGrou\"...,\n \t... // 1 ignored and 2 identical fields\n }\n",
},
}
occurredContributing tests (1)
Affected runs (1)
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||