CIHealth
<
Single Day: Jun 7▾
Pick a day
>
View JSON API
↗ Open Failure patterns for this day

Environment: PROD

Time (UTC)JobFailed atResultPRFailed testsDetails
periodic-ci-Azure-ARO-HCP-main-periodic-prod-e2e-parallel-ocp-nightly
post-goodmerged PR
No additional run metadata captured
 e2e failed n/a 3
e2e (3)
timeout during CreateHCPClusterAndWait; context deadline exceeded
23:00:01 UTC · failed at e2e · test Customer should be able to lifecycle and confirm external auth on a cluster · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_list_and_verify.go:83]: failed to create HCP cluster for external auth list test
Unexpected error:
    <*fmt.wrapErrors | 0xc0014819e0>: 
    failed to create HCP cluster ea-listg7dqqq, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster ea-listg7dqqq in resource group ea-list-89zt2g, error: failed waiting for cluster="ea-listg7dqqq" in resourcegroup="ea-list-89zt2g" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster ea-listg7dqqq in resource group ea-list-89zt2g, error: context deadline exceeded
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_list_and_verify.go:83]: failed to create HCP cluster for external auth list test
Unexpected error:
    <*fmt.wrapErrors | 0xc0014819e0>: 
    failed to create HCP cluster ea-listg7dqqq, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster ea-listg7dqqq in resource group ea-list-89zt2g, error: failed waiting for cluster="ea-listg7dqqq" in resourcegroup="ea-list-89zt2g" to finish creating, caused by: timeout '20.000000' minutes exceeded during CreateHCPClusterFromParam for cluster ea-listg7dqqq in resource group ea-list-89zt2g, error: context deadline exceeded
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Version 'openshift-v<version>' doesn't exist; provider Microsoft.RedHatOpenShift
23:00:01 UTC · failed at e2e · test Customer should upgrade a nodepool skipping one minor version (+2) from 4.20.z to 4.22.zLatest · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:443]: failed to create HCP cluster np-skip-minor-nxr6v8 at version 4.22.0
Unexpected error:
    <*fmt.wrapError | 0xc001012a40>: 
    failed to create HCP cluster np-skip-minor-nxr6v8: failed starting cluster creation "np-skip-minor-nxr6v8" in resourcegroup="rg-np-skip-minor-nxr6v8-m9wqh9": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-skip-minor-nxr6v8-m9wqh9/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-skip-minor-nxr6v8
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Version 'openshift-v4.22.0-nightly' doesn't exist"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:443]: failed to create HCP cluster np-skip-minor-nxr6v8 at version 4.22.0
Unexpected error:
    <*fmt.wrapError | 0xc001012a40>: 
    failed to create HCP cluster np-skip-minor-nxr6v8: failed starting cluster creation "np-skip-minor-nxr6v8" in resourcegroup="rg-np-skip-minor-nxr6v8-m9wqh9": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-skip-minor-nxr6v8-m9wqh9/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-skip-minor-nxr6v8
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Version 'openshift-v4.22.0-nightly' doesn't exist"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: InvalidRequestContent; detail message Version 'openshift-v<version>' doesn't exist; provider Microsoft.RedHatOpenShift
23:00:01 UTC · failed at e2e · test Customer should downgrade a nodepool to a lower minor version from 4.21.zLatest to 4.19.zLatest · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:704]: failed to create HCP cluster
Unexpected error:
    <*fmt.wrapError | 0xc00121b560>: 
    failed to create HCP cluster np-dg-minor-9mfg6z: failed starting cluster creation "np-dg-minor-9mfg6z" in resourcegroup="rg-np-dg-minor-9mfg6z-hhrzlx": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-dg-minor-9mfg6z-hhrzlx/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-dg-minor-9mfg6z
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Version 'openshift-v4.21.0-nightly' doesn't exist"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:704]: failed to create HCP cluster
Unexpected error:
    <*fmt.wrapError | 0xc00121b560>: 
    failed to create HCP cluster np-dg-minor-9mfg6z: failed starting cluster creation "np-dg-minor-9mfg6z" in resourcegroup="rg-np-dg-minor-9mfg6z-hhrzlx": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-dg-minor-9mfg6z-hhrzlx/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-dg-minor-9mfg6z
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Version 'openshift-v4.21.0-nightly' doesn't exist"
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
periodic-ci-Azure-ARO-HCP-main-periodic-prod-e2e-parallel
post-goodmerged PR
No additional run metadata captured
 e2e failed n/a 3
e2e (3)
tls: failed to verify certificate: x509: certificate signed by unknown authority
02:00:17 UTC · failed at e2e · test Customer should be able to create a no-CNI private cluster with a private key vault, a nodepool and install cilium CNI successfully · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:194]: failed to run simple web app and connectivity check with cilium CNI
Unexpected error:
    <*errors.joinError | 0xc00168a1e0>: 
    VerifySimpleWebApp failed: route was never reachable: Get "https://agnhost-e2e-serving-app-mvf6h.apps.aro.cilium-cluster.rxfu.uksouth.aroapp-hcp.io": tls: failed to verify certificate: x509: certificate signed by unknown authority
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_complex_cilium_kv.go:194]: failed to run simple web app and connectivity check with cilium CNI
Unexpected error:
    <*errors.joinError | 0xc00168a1e0>: 
    VerifySimpleWebApp failed: route was never reachable: Get "https://agnhost-e2e-serving-app-mvf6h.apps.aro.cilium-cluster.rxfu.uksouth.aroapp-hcp.io": tls: failed to verify certificate: x509: certificate signed by unknown authority
    ...
occurred
ERROR CODE: MultipleErrorsOccurred; detail code InvalidRequestContent; detail message Invalid value: "4.19.33": cannot downgrade from version 4.21.18 to 4.19.33; provider Microsoft.RedHatOpenShift
02:00:17 UTC · failed at e2e · test Customer should downgrade a nodepool to a lower minor version from 4.21.zLatest to 4.19.zLatest · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:750]: failed to update nodepool npdg-421-419 to downgrade target 4.19.33
Unexpected error:
    <*fmt.wrapError | 0xc000a426a0>: 
    failed to start nodepool "npdg-421-419" update in cluster "np-dg-minor-rh478b" resourcegroup="rg-np-dg-minor-rh478b-w9gctr": PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-dg-minor-rh478b-w9gctr/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-dg-minor-rh478b/nodePools/npdg-421-419
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: MultipleErrorsOccurred
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "MultipleErrorsOccurred",
        "message": "2 errors occurred",
        "details": [
          {
            "code": "InvalidRequestContent",
            "message": "Invalid value: \"4.19.33\": invalid node pool version 4.19.33: cannot downgrade from current version 4.21.18",
            "target": "properties.version.id"
          },
          {
            "code": "InvalidRequestContent",
            "message": "Invalid value: \"4.19.33\": cannot downgrade from version 4.21.18 to 4.19.33",
            "target": "properties.version.id"
          }
        ]
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:750]: failed to update nodepool npdg-421-419 to downgrade target 4.19.33
Unexpected error:
    <*fmt.wrapError | 0xc000a426a0>: 
    failed to start nodepool "npdg-421-419" update in cluster "np-dg-minor-rh478b" resourcegroup="rg-np-dg-minor-rh478b-w9gctr": PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-dg-minor-rh478b-w9gctr/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-dg-minor-rh478b/nodePools/npdg-421-419
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: MultipleErrorsOccurred
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "MultipleErrorsOccurred",
        "message": "2 errors occurred",
        "details": [
          {
            "code": "InvalidRequestContent",
            "message": "Invalid value: \"4.19.33\": invalid node pool version 4.19.33: cannot downgrade from current version 4.21.18",
            "target": "properties.version.id"
          },
          {
            "code": "InvalidRequestContent",
            "message": "Invalid value: \"4.19.33\": cannot downgrade from version 4.21.18 to 4.19.33",
            "target": "properties.version.id"
          }
        ]
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
ERROR CODE: MultipleErrorsOccurred; detail code InvalidRequestContent; detail message Invalid value: "4.21.17": cannot downgrade from version 4.21.18 to 4.21.17; provider Microsoft.RedHatOpenShift
02:00:17 UTC · failed at e2e · test Customer should downgrade a nodepool version z-stream downgrade from 4.21.zLatest to 4.21.zPrevious · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:619]: failed to update nodepool npdg-4-21 to downgrade target 4.21.17
Unexpected error:
    <*fmt.wrapError | 0xc000f121c0>: 
    failed to start nodepool "npdg-4-21" update in cluster "np-downgrade-2xj4hp" resourcegroup="rg-np-downgrade-2xj4hp-dpkmmj": PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-downgrade-2xj4hp-dpkmmj/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-downgrade-2xj4hp/nodePools/npdg-4-21
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: MultipleErrorsOccurred
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "MultipleErrorsOccurred",
        "message": "2 errors occurred",
        "details": [
          {
            "code": "InvalidRequestContent",
            "message": "Invalid value: \"4.21.17\": invalid node pool version 4.21.17: cannot downgrade from current version 4.21.18",
            "target": "properties.version.id"
          },
          {
            "code": "InvalidRequestContent",
            "message": "Invalid value: \"4.21.17\": cannot downgrade from version 4.21.18 to 4.21.17",
            "target": "properties.version.id"
          }
        ]
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_version_upgrade.go:619]: failed to update nodepool npdg-4-21 to downgrade target 4.21.17
Unexpected error:
    <*fmt.wrapError | 0xc000f121c0>: 
    failed to start nodepool "npdg-4-21" update in cluster "np-downgrade-2xj4hp" resourcegroup="rg-np-downgrade-2xj4hp-dpkmmj": PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/rg-np-downgrade-2xj4hp-dpkmmj/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/np-downgrade-2xj4hp/nodePools/npdg-4-21
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: MultipleErrorsOccurred
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "MultipleErrorsOccurred",
        "message": "2 errors occurred",
        "details": [
          {
            "code": "InvalidRequestContent",
            "message": "Invalid value: \"4.21.17\": invalid node pool version 4.21.17: cannot downgrade from current version 4.21.18",
            "target": "properties.version.id"
          },
          {
            "code": "InvalidRequestContent",
            "message": "Invalid value: \"4.21.17\": cannot downgrade from version 4.21.18 to 4.21.17",
            "target": "properties.version.id"
          }
        ]
      }
    }
    --------------------------------------------------------------------------------
    
    ...
occurred