No runs match your search.
Environment: PROD
| Time (UTC) | Job | Failed at | Result | PR | Failed tests | Details |
|---|---|---|---|---|---|---|
| branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel post-goodmerged PR |
n/a | passed | n/a | 0 | n/a | |
| periodic-ci-Azure-ARO-HCP-main-periodic-prod-e2e-parallel-ocp-nightly post-goodmerged PR |
e2e | failed | n/a | 1 | e2e (1)tls: failed to verify certificate: x509: certificate signed by unknown authority Show raw failurefail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:186]: failed to verify simple web app runs on cluster "cluster-nightly-4-23-dpbrz9"
Unexpected error:
<*fmt.wrapError | 0xc000fbe3e0>:
[strict TLS verification] route was never reachable: Get "https://agnhost-e2e-sample-app-zszzt.apps.aro.x0o8h6x6l7k9u1f.e0w5.uksouth.aroapp-hcp.io": tls: failed to verify certificate: x509: certificate signed by unknown authority
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:186]: failed to verify simple web app runs on cluster "cluster-nightly-4-23-dpbrz9"
Unexpected error:
<*fmt.wrapError | 0xc000fbe3e0>:
[strict TLS verification] route was never reachable: Get "https://agnhost-e2e-sample-app-zszzt.apps.aro.x0o8h6x6l7k9u1f.e0w5.uksouth.aroapp-hcp.io": tls: failed to verify certificate: x509: certificate signed by unknown authority
...
occurred |
|
| branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel post-goodmerged PR |
n/a | passed | n/a | 0 | n/a | |
| branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel post-goodmerged PR |
e2e | failed | n/a | 1 | e2e (1)cluster entered terminal Failed state after role assignment deployment Show raw failurefail [github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:219]: Timed out after 2520.004s.
cluster should eventually succeed after role assignments are created
The function passed to Eventually failed at /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:215 with:
cluster entered terminal Failed state after role assignment deployment
Expected
<armredhatopenshifthcp.ProvisioningState>: ...
not to equal
<armredhatopenshifthcp.ProvisioningState>: ...
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:219]: Timed out after 2520.004s.
cluster should eventually succeed after role assignments are created
The function passed to Eventually failed at /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:215 with:
cluster entered terminal Failed state after role assignment deployment
Expected
<armredhatopenshifthcp.ProvisioningState>: ...
not to equal
<armredhatopenshifthcp.ProvisioningState>: ... |
|
| branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel post-goodmerged PR |
e2e | failed | n/a | 1 | e2e (1)ERROR CODE: InternalServerError; detail message [clusterServiceNodePoolStatus] <no_message>; provider Microsoft.RedHatOpenShift Show raw failurefail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:173]: failed to create node pool "np-qcvwjv" for cluster "cluster-candidate-4-20-qcvwjv"
Unexpected error:
<*fmt.wrapError | 0xc0026a6940>:
failed to create NodePool np-qcvwjv: failed waiting for nodepool="np-qcvwjv" for cluster "cluster-candidate-4-20-qcvwjv" in resourcegroup="rg-candidate-4-20-kwkc79fpjrx2" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/australiaeast/hcpOperationStatuses/a9e953c4-caff-45c7-a41d-8606a89637ea
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/australiaeast/hcpOperationStatuses/a9e953c4-caff-45c7-a41d-8606a89637ea",
"name": "a9e953c4-caff-45c7-a41d-8606a89637ea",
"status": "Failed",
"startTime": "2026-07-22T22:01:39.504754038Z",
"endTime": "2026-07-22T22:20:44.648330077Z",
"error": {
"code": "InternalServerError",
"message": "[clusterServiceNodePoolStatus] \u003cno_message\u003e"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:173]: failed to create node pool "np-qcvwjv" for cluster "cluster-candidate-4-20-qcvwjv"
Unexpected error:
<*fmt.wrapError | 0xc0026a6940>:
failed to create NodePool np-qcvwjv: failed waiting for nodepool="np-qcvwjv" for cluster "cluster-candidate-4-20-qcvwjv" in resourcegroup="rg-candidate-4-20-kwkc79fpjrx2" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/australiaeast/hcpOperationStatuses/a9e953c4-caff-45c7-a41d-8606a89637ea
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/australiaeast/hcpOperationStatuses/a9e953c4-caff-45c7-a41d-8606a89637ea",
"name": "a9e953c4-caff-45c7-a41d-8606a89637ea",
"status": "Failed",
"startTime": "2026-07-22T22:01:39.504754038Z",
"endTime": "2026-07-22T22:20:44.648330077Z",
"error": {
"code": "InternalServerError",
"message": "[clusterServiceNodePoolStatus] \u003cno_message\u003e"
}
}
--------------------------------------------------------------------------------
...
occurred |
|
| branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel post-goodmerged PR |
n/a | passed | n/a | 0 | n/a | |
| branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel post-goodmerged PR |
n/a | passed | n/a | 0 | n/a | |
| branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel post-goodmerged PR |
n/a | passed | n/a | 0 | n/a | |
| branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel post-goodmerged PR |
e2e | failed | n/a | 1 | e2e (1)ERROR CODE: InternalServerError; detail message [cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content; provider Microsoft.RedHatOpenShift Show raw failurefail [github.com/Azure/ARO-HCP/test/e2e/idms_lifecycle.go:112]: failed to create HCP cluster with ImageDigestMirrors
Unexpected error:
<*fmt.wrapError | 0xc0006e58a0>:
failed to create HCP cluster idms-e2e-hcp-cluster: failed waiting for cluster="idms-e2e-hcp-cluster" in resourcegroup="idms-rp6jsc" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/174515fd-ae84-46d9-b491-3d03c175e06c
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/174515fd-ae84-46d9-b491-3d03c175e06c",
"name": "174515fd-ae84-46d9-b491-3d03c175e06c",
"status": "Failed",
"startTime": "2026-07-22T12:40:43.885322535Z",
"endTime": "2026-07-22T12:59:47.104093689Z",
"error": {
"code": "InternalServerError",
"message": "[cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/idms_lifecycle.go:112]: failed to create HCP cluster with ImageDigestMirrors
Unexpected error:
<*fmt.wrapError | 0xc0006e58a0>:
failed to create HCP cluster idms-e2e-hcp-cluster: failed waiting for cluster="idms-e2e-hcp-cluster" in resourcegroup="idms-rp6jsc" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/174515fd-ae84-46d9-b491-3d03c175e06c
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/brazilsouth/hcpOperationStatuses/174515fd-ae84-46d9-b491-3d03c175e06c",
"name": "174515fd-ae84-46d9-b491-3d03c175e06c",
"status": "Failed",
"startTime": "2026-07-22T12:40:43.885322535Z",
"endTime": "2026-07-22T12:59:47.104093689Z",
"error": {
"code": "InternalServerError",
"message": "[cosmosCluster] .api.url is empty; [hypershiftHostedCluster] ReadDesire has no kube content"
}
}
--------------------------------------------------------------------------------
...
occurred |
|
| branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel post-goodmerged PR |
e2e | failed | n/a | 1 | e2e (1)cluster entered terminal Failed state after role assignment deployment Show raw failurefail [github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:219]: Timed out after 2520.001s.
cluster should eventually succeed after role assignments are created
The function passed to Eventually failed at /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:215 with:
cluster entered terminal Failed state after role assignment deployment
Expected
<armredhatopenshifthcp.ProvisioningState>: ...
not to equal
<armredhatopenshifthcp.ProvisioningState>: ...
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:219]: Timed out after 2520.001s.
cluster should eventually succeed after role assignments are created
The function passed to Eventually failed at /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:215 with:
cluster entered terminal Failed state after role assignment deployment
Expected
<armredhatopenshifthcp.ProvisioningState>: ...
not to equal
<armredhatopenshifthcp.ProvisioningState>: ... |
|
| branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel post-goodmerged PR |
n/a | passed | n/a | 0 | n/a | |
| branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel post-goodmerged PR |
e2e | failed | n/a | 1 | e2e (1)ERROR CODE: InternalServerError; detail code DenyAssignmentAuthorizationFailed; detail message Access denied by deny assignment for action "Microsoft.Compute/virtualMachines/retrieveBootDiagnosticsData/action".; provider Microsoft.Authorization Show raw failurefail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:190]: failed to verify nodes are Ready with Cilium CNI for cluster "cilium-cluster"
Unexpected error:
<*errors.joinError | 0xc00003c078>:
VerifyNodesReady failed: no nodes found in the cluster
VerifyCiliumOperational failed: not all pods in kube-system namespace are running: failed to list pods in kube-system namespace: client rate limiter Wait returned an error: context deadline exceeded
failed to create NodePool cilium-np: failed waiting for nodepool="cilium-np" for cluster "cilium-cluster" in resourcegroup="complex-cilium-kv-xxxldl" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/australiaeast/hcpOperationStatuses/9c4f6c2a-fe7f-4a6d-b9a4-e7fb6b5215d9
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/australiaeast/hcpOperationStatuses/9c4f6c2a-fe7f-4a6d-b9a4-e7fb6b5215d9",
"name": "9c4f6c2a-fe7f-4a6d-b9a4-e7fb6b5215d9",
"status": "Failed",
"startTime": "2026-07-22T08:31:33.476055825Z",
"endTime": "2026-07-22T08:50:37.393817181Z",
"error": {
"code": "InternalServerError",
"message": "[clusterServiceNodePoolStatus] \u003cno_message\u003e"
}
}
--------------------------------------------------------------------------------
encountered errors downloading console logs:
- VM "cilium-cluster-cilium-np-8skrs-nxzhn": failed to retrieve boot diagnostics data for VM "cilium-cluster-cilium-np-8skrs-nxzhn": POST https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/complex-cilium-kv-xxxldl--managed/providers/Microsoft.Compute/virtualMachines/cilium-cluster-cilium-np-8skrs-nxzhn/retrieveBootDiagnosticsData
--------------------------------------------------------------------------------
RESPONSE 403: 403 Forbidden
ERROR CODE: DenyAssignmentAuthorizationFailed
--------------------------------------------------------------------------------
{
"error": {
"code": "DenyAssignmentAuthorizationFailed",
"message": "The client 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX' with object id '0e96dbb7-17a2-4393-b2d3-d84524e39c32' has permission to perform action 'Microsoft.Compute/virtualMachines/retrieveBootDiagnosticsData/action' on scope '/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/complex-cilium-kv-xxxldl--managed/providers/Microsoft.Compute/virtualMachines/cilium-cluster-cilium-np-8skrs-nxzhn'; however, the access is denied because of the deny assignment with name '/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/complex-cilium-kv-xxxldl--managed/providers/Microsoft.Authorization/denyAssignments/4e1d5354-563b-53f0-a663-9624ce82fcc6' and Id '4e1d5354563b53f0a6639624ce82fcc6' at scope '/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/complex-cilium-kv-xxxldl--managed'."
}
}
--------------------------------------------------------------------------------
- VM "cilium-cluster-cilium-np-8skrs-wwbw4": failed to retrieve boot diagnostics data for VM "cilium-cluster-cilium-np-8skrs-wwbw4": POST https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/complex-cilium-kv-xxxldl--managed/providers/Microsoft.Compute/virtualMachines/cilium-cluster-cilium-np-8skrs-wwbw4/retrieveBootDiagnosticsData
--------------------------------------------------------------------------------
RESPONSE 403: 403 Forbidden
ERROR CODE: DenyAssignmentAuthorizationFailed
--------------------------------------------------------------------------------
{
"error": {
"code": "DenyAssignmentAuthorizationFailed",
"message": "The client 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX' with object id '0e96dbb7-17a2-4393-b2d3-d84524e39c32' has permission to perform action 'Microsoft.Compute/virtualMachines/retrieveBootDiagnosticsData/action' on scope '/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/complex-cilium-kv-xxxldl--managed/providers/Microsoft.Compute/virtualMachines/cilium-cluster-cilium-np-8skrs-wwbw4'; however, the access is denied because of the deny assignment with name '/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/complex-cilium-kv-xxxldl--managed/providers/Microsoft.Authorization/denyAssignments/4e1d5354-563b-53f0-a663-9624ce82fcc6' and Id '4e1d5354563b53f0a6639624ce82fcc6' at scope '/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/complex-cilium-kv-xxxldl--managed'."
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:190]: failed to verify nodes are Ready with Cilium CNI for cluster "cilium-cluster"
Unexpected error:
<*errors.joinError | 0xc00003c078>:
VerifyNodesReady failed: no nodes found in the cluster
VerifyCiliumOperational failed: not all pods in kube-system namespace are running: failed to list pods in kube-system namespace: client rate limiter Wait returned an error: context deadline exceeded
failed to create NodePool cilium-np: failed waiting for nodepool="cilium-np" for cluster "cilium-cluster" in resourcegroup="complex-cilium-kv-xxxldl" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/australiaeast/hcpOperationStatuses/9c4f6c2a-fe7f-4a6d-b9a4-e7fb6b5215d9
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/australiaeast/hcpOperationStatuses/9c4f6c2a-fe7f-4a6d-b9a4-e7fb6b5215d9",
"name": "9c4f6c2a-fe7f-4a6d-b9a4-e7fb6b5215d9",
"status": "Failed",
"startTime": "2026-07-22T08:31:33.476055825Z",
"endTime": "2026-07-22T08:50:37.393817181Z",
"error": {
"code": "InternalServerError",
"message": "[clusterServiceNodePoolStatus] \u003cno_message\u003e"
}
}
--------------------------------------------------------------------------------
encountered errors downloading console logs:
- VM "cilium-cluster-cilium-np-8skrs-nxzhn": failed to retrieve boot diagnostics data for VM "cilium-cluster-cilium-np-8skrs-nxzhn": POST https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/complex-cilium-kv-xxxldl--managed/providers/Microsoft.Compute/virtualMachines/cilium-cluster-cilium-np-8skrs-nxzhn/retrieveBootDiagnosticsData
--------------------------------------------------------------------------------
RESPONSE 403: 403 Forbidden
ERROR CODE: DenyAssignmentAuthorizationFailed
--------------------------------------------------------------------------------
{
"error": {
"code": "DenyAssignmentAuthorizationFailed",
"message": "The client 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX' with object id '0e96dbb7-17a2-4393-b2d3-d84524e39c32' has permission to perform action 'Microsoft.Compute/virtualMachines/retrieveBootDiagnosticsData/action' on scope '/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/complex-cilium-kv-xxxldl--managed/providers/Microsoft.Compute/virtualMachines/cilium-cluster-cilium-np-8skrs-nxzhn'; however, the access is denied because of the deny assignment with name '/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/complex-cilium-kv-xxxldl--managed/providers/Microsoft.Authorization/denyAssignments/4e1d5354-563b-53f0-a663-9624ce82fcc6' and Id '4e1d5354563b53f0a6639624ce82fcc6' at scope '/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/complex-cilium-kv-xxxldl--managed'."
}
}
--------------------------------------------------------------------------------
- VM "cilium-cluster-cilium-np-8skrs-wwbw4": failed to retrieve boot diagnostics data for VM "cilium-cluster-cilium-np-8skrs-wwbw4": POST https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/complex-cilium-kv-xxxldl--managed/providers/Microsoft.Compute/virtualMachines/cilium-cluster-cilium-np-8skrs-wwbw4/retrieveBootDiagnosticsData
--------------------------------------------------------------------------------
RESPONSE 403: 403 Forbidden
ERROR CODE: DenyAssignmentAuthorizationFailed
--------------------------------------------------------------------------------
{
"error": {
"code": "DenyAssignmentAuthorizationFailed",
"message": "The client 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX' with object id '0e96dbb7-17a2-4393-b2d3-d84524e39c32' has permission to perform action 'Microsoft.Compute/virtualMachines/retrieveBootDiagnosticsData/action' on scope '/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/complex-cilium-kv-xxxldl--managed/providers/Microsoft.Compute/virtualMachines/cilium-cluster-cilium-np-8skrs-wwbw4'; however, the access is denied because of the deny assignment with name '/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/complex-cilium-kv-xxxldl--managed/providers/Microsoft.Authorization/denyAssignments/4e1d5354-563b-53f0-a663-9624ce82fcc6' and Id '4e1d5354563b53f0a6639624ce82fcc6' at scope '/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/complex-cilium-kv-xxxldl--managed'."
}
}
--------------------------------------------------------------------------------
...
occurred |
|
| branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel post-goodmerged PR |
n/a | passed | n/a | 0 | n/a | |
| branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel post-goodmerged PR |
e2e | failed | n/a | 1 | e2e (1)ERROR CODE: InternalServerError; detail message [clusterServiceNodePoolStatus] <no_message>; provider Microsoft.RedHatOpenShift Show raw failurefail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:159]: failed to create node pool "np-79gssd" for cluster "cluster-candidate-4-21-79gssd"
Unexpected error:
<*fmt.wrapError | 0xc00061b560>:
failed to create NodePool np-79gssd: failed waiting for nodepool="np-79gssd" for cluster "cluster-candidate-4-21-79gssd" in resourcegroup="rg-candidate-4-21-rjrdqb" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/australiaeast/hcpOperationStatuses/985baa3e-b3d4-4c63-a507-079bb53207c7
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/australiaeast/hcpOperationStatuses/985baa3e-b3d4-4c63-a507-079bb53207c7",
"name": "985baa3e-b3d4-4c63-a507-079bb53207c7",
"status": "Failed",
"startTime": "2026-07-22T06:00:51.385178815Z",
"endTime": "2026-07-22T06:19:51.057036582Z",
"error": {
"code": "InternalServerError",
"message": "[clusterServiceNodePoolStatus] \u003cno_message\u003e"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:159]: failed to create node pool "np-79gssd" for cluster "cluster-candidate-4-21-79gssd"
Unexpected error:
<*fmt.wrapError | 0xc00061b560>:
failed to create NodePool np-79gssd: failed waiting for nodepool="np-79gssd" for cluster "cluster-candidate-4-21-79gssd" in resourcegroup="rg-candidate-4-21-rjrdqb" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/australiaeast/hcpOperationStatuses/985baa3e-b3d4-4c63-a507-079bb53207c7
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/australiaeast/hcpOperationStatuses/985baa3e-b3d4-4c63-a507-079bb53207c7",
"name": "985baa3e-b3d4-4c63-a507-079bb53207c7",
"status": "Failed",
"startTime": "2026-07-22T06:00:51.385178815Z",
"endTime": "2026-07-22T06:19:51.057036582Z",
"error": {
"code": "InternalServerError",
"message": "[clusterServiceNodePoolStatus] \u003cno_message\u003e"
}
}
--------------------------------------------------------------------------------
...
occurred |
|
| periodic-ci-Azure-ARO-HCP-main-periodic-prod-e2e-parallel post-goodmerged PR |
e2e | failed | n/a | 3 | e2e (3)cluster entered terminal Failed state after role assignment deployment Show raw failurefail [github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:219]: Timed out after 2520.002s.
cluster should eventually succeed after role assignments are created
The function passed to Eventually failed at /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:215 with:
cluster entered terminal Failed state after role assignment deployment
Expected
<armredhatopenshifthcp.ProvisioningState>: ...
not to equal
<armredhatopenshifthcp.ProvisioningState>: ...
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:219]: Timed out after 2520.002s.
cluster should eventually succeed after role assignments are created
The function passed to Eventually failed at /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:215 with:
cluster entered terminal Failed state after role assignment deployment
Expected
<armredhatopenshifthcp.ProvisioningState>: ...
not to equal
<armredhatopenshifthcp.ProvisioningState>: ...ERROR CODE: InternalServerError; detail message [hypershiftHostedCluster] <no_message>; [cosmosCluster] <no_message>; provider Microsoft.RedHatOpenShift Show raw failurefail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:120]: HCP cluster rg-candidate-4-20-qnfrfst59rxk/cluster-candidate-4-20-z9f7qm should provision
Unexpected error:
<*fmt.wrapError | 0xc000b10b60>:
failed to create HCP cluster cluster-candidate-4-20-z9f7qm: failed waiting for cluster="cluster-candidate-4-20-z9f7qm" in resourcegroup="rg-candidate-4-20-qnfrfst59rxk" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/952533d9-7ad3-4b91-b8b8-23d93e01328b
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/952533d9-7ad3-4b91-b8b8-23d93e01328b",
"name": "952533d9-7ad3-4b91-b8b8-23d93e01328b",
"status": "Failed",
"startTime": "2026-07-22T02:05:40.15224731Z",
"endTime": "2026-07-22T02:24:41.285079074Z",
"error": {
"code": "InternalServerError",
"message": "[hypershiftHostedCluster] \u003cno_message\u003e; [cosmosCluster] \u003cno_message\u003e"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:120]: HCP cluster rg-candidate-4-20-qnfrfst59rxk/cluster-candidate-4-20-z9f7qm should provision
Unexpected error:
<*fmt.wrapError | 0xc000b10b60>:
failed to create HCP cluster cluster-candidate-4-20-z9f7qm: failed waiting for cluster="cluster-candidate-4-20-z9f7qm" in resourcegroup="rg-candidate-4-20-qnfrfst59rxk" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/952533d9-7ad3-4b91-b8b8-23d93e01328b
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/952533d9-7ad3-4b91-b8b8-23d93e01328b",
"name": "952533d9-7ad3-4b91-b8b8-23d93e01328b",
"status": "Failed",
"startTime": "2026-07-22T02:05:40.15224731Z",
"endTime": "2026-07-22T02:24:41.285079074Z",
"error": {
"code": "InternalServerError",
"message": "[hypershiftHostedCluster] \u003cno_message\u003e; [cosmosCluster] \u003cno_message\u003e"
}
}
--------------------------------------------------------------------------------
...
occurredERROR CODE: InternalServerError; detail message [hypershiftHostedCluster] <no_message>; [cosmosCluster] <no_message>; provider Microsoft.RedHatOpenShift Show raw failurefail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:97]: failed to create HCP cluster "private-kv-cluster" with private keyvault
Unexpected error:
<*fmt.wrapError | 0xc0005871c0>:
failed waiting for cluster="private-kv-cluster" in resourcegroup="private-keyvault-sj5lp4s8hdhx" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/913f1b53-79a1-4c56-90f3-f94f835b8c75
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/913f1b53-79a1-4c56-90f3-f94f835b8c75",
"name": "913f1b53-79a1-4c56-90f3-f94f835b8c75",
"status": "Failed",
"startTime": "2026-07-22T02:04:09.655224875Z",
"endTime": "2026-07-22T02:23:21.611762653Z",
"error": {
"code": "InternalServerError",
"message": "[hypershiftHostedCluster] \u003cno_message\u003e; [cosmosCluster] \u003cno_message\u003e"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_private_kv.go:97]: failed to create HCP cluster "private-kv-cluster" with private keyvault
Unexpected error:
<*fmt.wrapError | 0xc0005871c0>:
failed waiting for cluster="private-kv-cluster" in resourcegroup="private-keyvault-sj5lp4s8hdhx" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/913f1b53-79a1-4c56-90f3-f94f835b8c75
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/913f1b53-79a1-4c56-90f3-f94f835b8c75",
"name": "913f1b53-79a1-4c56-90f3-f94f835b8c75",
"status": "Failed",
"startTime": "2026-07-22T02:04:09.655224875Z",
"endTime": "2026-07-22T02:23:21.611762653Z",
"error": {
"code": "InternalServerError",
"message": "[hypershiftHostedCluster] \u003cno_message\u003e; [cosmosCluster] \u003cno_message\u003e"
}
}
--------------------------------------------------------------------------------
...
occurred |
|
| branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel post-goodmerged PR |
e2e | failed | n/a | 1 | e2e (1)cluster entered terminal Failed state after role assignment deployment Show raw failurefail [github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:219]: Timed out after 2520.002s.
cluster should eventually succeed after role assignments are created
The function passed to Eventually failed at /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:215 with:
cluster entered terminal Failed state after role assignment deployment
Expected
<armredhatopenshifthcp.ProvisioningState>: ...
not to equal
<armredhatopenshifthcp.ProvisioningState>: ...
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:219]: Timed out after 2520.002s.
cluster should eventually succeed after role assignments are created
The function passed to Eventually failed at /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/cluster_delayed_role_assignments.go:215 with:
cluster entered terminal Failed state after role assignment deployment
Expected
<armredhatopenshifthcp.ProvisioningState>: ...
not to equal
<armredhatopenshifthcp.ProvisioningState>: ... |
|
| branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel post-goodmerged PR |
e2e | failed | n/a | 1 | e2e (1)ERROR CODE: InternalServerError; detail message [clusterServiceNodePoolStatus] <no_message>; provider Microsoft.RedHatOpenShift Show raw failurefail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:159]: failed to create node pool "np-v8pbwf" for cluster "cluster-candidate-4-20-v8pbwf"
Unexpected error:
<*fmt.wrapError | 0xc000fb8780>:
failed to create NodePool np-v8pbwf: failed waiting for nodepool="np-v8pbwf" for cluster "cluster-candidate-4-20-v8pbwf" in resourcegroup="rg-candidate-4-20-tr95ln" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/australiaeast/hcpOperationStatuses/4dcd2df8-598c-4ddd-ae99-b9917b436164
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/australiaeast/hcpOperationStatuses/4dcd2df8-598c-4ddd-ae99-b9917b436164",
"name": "4dcd2df8-598c-4ddd-ae99-b9917b436164",
"status": "Failed",
"startTime": "2026-07-22T01:51:53.026515054Z",
"endTime": "2026-07-22T02:11:00.801292673Z",
"error": {
"code": "InternalServerError",
"message": "[clusterServiceNodePoolStatus] \u003cno_message\u003e"
}
}
--------------------------------------------------------------------------------
...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:159]: failed to create node pool "np-v8pbwf" for cluster "cluster-candidate-4-20-v8pbwf"
Unexpected error:
<*fmt.wrapError | 0xc000fb8780>:
failed to create NodePool np-v8pbwf: failed waiting for nodepool="np-v8pbwf" for cluster "cluster-candidate-4-20-v8pbwf" in resourcegroup="rg-candidate-4-20-tr95ln" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/australiaeast/hcpOperationStatuses/4dcd2df8-598c-4ddd-ae99-b9917b436164
--------------------------------------------------------------------------------
RESPONSE 200: 200 OK
ERROR CODE: InternalServerError
--------------------------------------------------------------------------------
{
"id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/australiaeast/hcpOperationStatuses/4dcd2df8-598c-4ddd-ae99-b9917b436164",
"name": "4dcd2df8-598c-4ddd-ae99-b9917b436164",
"status": "Failed",
"startTime": "2026-07-22T01:51:53.026515054Z",
"endTime": "2026-07-22T02:11:00.801292673Z",
"error": {
"code": "InternalServerError",
"message": "[clusterServiceNodePoolStatus] \u003cno_message\u003e"
}
}
--------------------------------------------------------------------------------
...
occurred |