<
Single Day: Jul 20
Pick a day
>
View JSON API
↗ Open Failure patterns for this day

Environment: INT

Time (UTC)JobFailed atResultPRFailed testsDetails
branch-ci-Azure-ARO-HCP-main-e2e-integration-e2e-parallel
post-goodmerged PR
No additional run metadata captured
e2e failed n/a 1
e2e (1)
UpdateNodePoolAndWait timed out after <minutes> minutes; context deadline exceeded
20:27:52 UTC · failed at e2e · test Customer should upgrade and update a nodepool from 4.20.z to 4.21.zLatest · suite integration/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:210]: failed to upgrade node pool npupgrade-4-20 to version 4.21.25
Unexpected error:
    <*fmt.wrapErrors | 0xc0027f6060>: 
    failed waiting for nodepool="npupgrade-4-20" in cluster="np-version-upgrade-cluster-9cldvc" resourcegroup="rg-np-version-upgrade-9cldvc-x78hlk" to finish updating, caused by: timeout '45.000000' minutes exceeded during UpdateNodePoolAndWait for nodepool npupgrade-4-20 in cluster np-version-upgrade-cluster-9cldvc in resource group rg-np-version-upgrade-9cldvc-x78hlk, error: context deadline exceeded
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:210]: failed to upgrade node pool npupgrade-4-20 to version 4.21.25
Unexpected error:
    <*fmt.wrapErrors | 0xc0027f6060>: 
    failed waiting for nodepool="npupgrade-4-20" in cluster="np-version-upgrade-cluster-9cldvc" resourcegroup="rg-np-version-upgrade-9cldvc-x78hlk" to finish updating, caused by: timeout '45.000000' minutes exceeded during UpdateNodePoolAndWait for nodepool npupgrade-4-20 in cluster np-version-upgrade-cluster-9cldvc in resource group rg-np-version-upgrade-9cldvc-x78hlk, error: context deadline exceeded
    ...
occurred
branch-ci-Azure-ARO-HCP-main-e2e-integration-e2e-parallel
post-goodmerged PR
No additional run metadata captured
e2e failed n/a 1
e2e (1)
ERROR CODE: DeploymentFailed; detail code VaultAlreadyExists; detail message the vault name '<vault-name>' is already in use. Vault names are globally unique so it is possible that the name is already taken. If you are sure that the vault name was not taken then it is possible that a vault with the same name was recently deleted but not purged after being placed in a recoverable state. If the vault is in a recoverable state then the vault will need to be purged before reusing the name. For more information about VaultAlreadyExists, soft delete and purging a vault follow this link <url>
15:52:41 UTC · failed at e2e · test Customer should be able to create an HCP cluster then delete it by deleting the customer resource group · suite integration/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delete_cx_rg.go:79]: failed to create customer resources for cluster "cx-rg-hcp-cluster"
Unexpected error:
    <*fmt.wrapError | 0xc000c2a6a0>: 
    failed to create customer-infra: failed waiting for deployment "customer-infra-cx-rg-hcp-cluster-r59cl2" in resourcegroup="cx-rg-cluster-7z2n9v" to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourcegroups/cx-rg-cluster-7z2n9v/providers/Microsoft.Resources/deployments/customer-infra-cx-rg-hcp-cluster-r59cl2/operationStatuses/08584170421377078879
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: DeploymentFailed
    --------------------------------------------------------------------------------
    {
      "status": "Failed",
      "error": {
        "code": "DeploymentFailed",
        "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.",
        "details": [
          {
            "code": "Conflict",
            "message": "{\r\n  \"error\": {\r\n    \"code\": \"VaultAlreadyExists\",\r\n    \"message\": \"The vault name 'cust-kv-hqzib2x2rf23w' is already in use. Vault names are globally unique so it is possible that the name is already taken. If you are sure that the vault name was not taken then it is possible that a vault with the same name was recently deleted but not purged after being placed in a recoverable state. If the vault is in a recoverable state then the vault will need to be purged before reusing the name. For more information about VaultAlreadyExists, soft delete and purging a vault follow this link https://go.microsoft.com/fwlink/?linkid=2147740.\"\r\n  }\r\n}"
          }
        ]
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_delete_cx_rg.go:79]: failed to create customer resources for cluster "cx-rg-hcp-cluster"
Unexpected error:
    <*fmt.wrapError | 0xc000c2a6a0>: 
    failed to create customer-infra: failed waiting for deployment "customer-infra-cx-rg-hcp-cluster-r59cl2" in resourcegroup="cx-rg-cluster-7z2n9v" to finish: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourcegroups/cx-rg-cluster-7z2n9v/providers/Microsoft.Resources/deployments/customer-infra-cx-rg-hcp-cluster-r59cl2/operationStatuses/08584170421377078879
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: DeploymentFailed
    --------------------------------------------------------------------------------
    {
      "status": "Failed",
      "error": {
        "code": "DeploymentFailed",
        "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/arm-deployment-operations for usage details.",
        "details": [
          {
            "code": "Conflict",
            "message": "{\r\n  \"error\": {\r\n    \"code\": \"VaultAlreadyExists\",\r\n    \"message\": \"The vault name 'cust-kv-hqzib2x2rf23w' is already in use. Vault names are globally unique so it is possible that the name is already taken. If you are sure that the vault name was not taken then it is possible that a vault with the same name was recently deleted but not purged after being placed in a recoverable state. If the vault is in a recoverable state then the vault will need to be purged before reusing the name. For more information about VaultAlreadyExists, soft delete and purging a vault follow this link https://go.microsoft.com/fwlink/?linkid=2147740.\"\r\n  }\r\n}"
          }
        ]
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
branch-ci-Azure-ARO-HCP-main-e2e-integration-e2e-parallel
post-goodmerged PR
No additional run metadata captured
e2e failed n/a 1
e2e (1)
ERROR CODE: InternalServerError; detail message [clusterServiceNodePoolStatus] <no_message>; provider Microsoft.RedHatOpenShift
11:39:54 UTC · failed at e2e · test Customer should be able to create a no-CNI private cluster with a private key vault, a nodepool and install cilium CNI successfully · suite integration/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:190]: failed to verify nodes are Ready with Cilium CNI for cluster "cilium-cluster"
Unexpected error:
    <*errors.joinError | 0xc001b16d68>: 
    failed to create NodePool cilium-np: failed waiting for nodepool="cilium-np" for cluster "cilium-cluster" in resourcegroup="complex-cilium-kv-bznnwq" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/5f6c6dfd-730b-4056-b31f-c855a36148c6
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/5f6c6dfd-730b-4056-b31f-c855a36148c6",
      "name": "5f6c6dfd-730b-4056-b31f-c855a36148c6",
      "status": "Failed",
      "startTime": "2026-07-20T12:10:01.730836206Z",
      "endTime": "2026-07-20T12:29:12.036855578Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceNodePoolStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:190]: failed to verify nodes are Ready with Cilium CNI for cluster "cilium-cluster"
Unexpected error:
    <*errors.joinError | 0xc001b16d68>: 
    failed to create NodePool cilium-np: failed waiting for nodepool="cilium-np" for cluster "cilium-cluster" in resourcegroup="complex-cilium-kv-bznnwq" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/5f6c6dfd-730b-4056-b31f-c855a36148c6
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/5f6c6dfd-730b-4056-b31f-c855a36148c6",
      "name": "5f6c6dfd-730b-4056-b31f-c855a36148c6",
      "status": "Failed",
      "startTime": "2026-07-20T12:10:01.730836206Z",
      "endTime": "2026-07-20T12:29:12.036855578Z",
      "error": {
        "code": "InternalServerError",
        "message": "[clusterServiceNodePoolStatus] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
branch-ci-Azure-ARO-HCP-main-e2e-integration-e2e-parallel
post-goodmerged PR
No additional run metadata captured
e2e failed n/a 1
e2e (1)
ERROR CODE: InternalServerError; detail message [hypershiftHostedCluster] <no_message>; [cosmosCluster] <no_message>; provider Microsoft.RedHatOpenShift
05:46:34 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP candidate channel for 4.21 · suite integration/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:106]: HCP cluster rg-candidate-4-21-pthb4x/cluster-candidate-4-21-tb29zs should provision
Unexpected error:
    <*fmt.wrapError | 0xc000d2b5e0>: 
    failed to create HCP cluster cluster-candidate-4-21-tb29zs: failed waiting for cluster="cluster-candidate-4-21-tb29zs" in resourcegroup="rg-candidate-4-21-pthb4x" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/964e61e7-c52a-4011-b322-68cdf175e817
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/964e61e7-c52a-4011-b322-68cdf175e817",
      "name": "964e61e7-c52a-4011-b322-68cdf175e817",
      "status": "Failed",
      "startTime": "2026-07-20T06:11:14.467304676Z",
      "endTime": "2026-07-20T06:30:23.960698535Z",
      "error": {
        "code": "InternalServerError",
        "message": "[hypershiftHostedCluster] \u003cno_message\u003e; [cosmosCluster] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:106]: HCP cluster rg-candidate-4-21-pthb4x/cluster-candidate-4-21-tb29zs should provision
Unexpected error:
    <*fmt.wrapError | 0xc000d2b5e0>: 
    failed to create HCP cluster cluster-candidate-4-21-tb29zs: failed waiting for cluster="cluster-candidate-4-21-tb29zs" in resourcegroup="rg-candidate-4-21-pthb4x" to finish creating: GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/964e61e7-c52a-4011-b322-68cdf175e817
    --------------------------------------------------------------------------------
    RESPONSE 200: 200 OK
    ERROR CODE: InternalServerError
    --------------------------------------------------------------------------------
    {
      "id": "/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.RedHatOpenShift/locations/uksouth/hcpOperationStatuses/964e61e7-c52a-4011-b322-68cdf175e817",
      "name": "964e61e7-c52a-4011-b322-68cdf175e817",
      "status": "Failed",
      "startTime": "2026-07-20T06:11:14.467304676Z",
      "endTime": "2026-07-20T06:30:23.960698535Z",
      "error": {
        "code": "InternalServerError",
        "message": "[hypershiftHostedCluster] \u003cno_message\u003e; [cosmosCluster] \u003cno_message\u003e"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred