CIHealth
<
Single Day: Oct 19▾
Pick a day
>
View JSON API
↗ Open Failure patterns for this day

Environment: PROD

Time (UTC)RuntimeJobRegionFailed atResultPRFailed testsDetails
1h 37m 5s periodic-ci-Azure-ARO-HCP-main-periodic-prod-e2e-parallel-ocp-nightly
post-goodmerged PR
No additional run metadata captured
 canadacentral e2e failed n/a 7
e2e (7)
ERROR CODE: InvalidRequestContent; detail message Invalid value: "4.21": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is "nightly"; provider Microsoft.RedHatOpenShift
23:00:08 UTC · failed at e2e · test Update HCPOpenShiftCluster Positive creates a cluster and updates tags with a PATCH request · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:198]: failed to update HCP cluster tags via PATCH
Unexpected error:
    <*exported.ResponseError | 0x25e73f10e780>: 
    PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-tags-zb5hfhnv2h42/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-tags-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"4.21\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"",
        "target": "properties.version.id"
      }
    }
    --------------------------------------------------------------------------------
    
    <Azure error internals omitted>
        Azure request IDs:
          PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-tags-zb5hfhnv2h42/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-tags-cluster
            correlation ID: c685429f-56f0-498b-b1b4-107179a89c93
            client request ID: unavailable
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_update.go:198]: failed to update HCP cluster tags via PATCH
Unexpected error:
    <*exported.ResponseError | 0x25e73f10e780>: 
    PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-tags-zb5hfhnv2h42/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-tags-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"4.21\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"",
        "target": "properties.version.id"
      }
    }
    --------------------------------------------------------------------------------
    
    <Azure error internals omitted>
        Azure request IDs:
          PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/patch-tags-zb5hfhnv2h42/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/patch-tags-cluster
            correlation ID: c685429f-56f0-498b-b1b4-107179a89c93
            client request ID: unavailable
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "4.21": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is "nightly"; provider Microsoft.RedHatOpenShift
23:00:08 UTC · failed at e2e · test Customer should be able to create a cluster and node pools with aggregated advanced features · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:275]: failed to begin cluster "agg-cluster" authorized CIDR update
Unexpected error:
    <*exported.ResponseError | 0x123635df0060>: 
    PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/feature-aggregation-k256l8ffc826/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/agg-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"4.21\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"",
        "target": "properties.version.id"
      }
    }
    --------------------------------------------------------------------------------
    
    <Azure error internals omitted>
        Azure request IDs:
          PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/feature-aggregation-k256l8ffc826/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/agg-cluster
            correlation ID: a49c09da-23cf-4bbe-85aa-61274c4b30d8
            client request ID: unavailable
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_create_feature_aggregation.go:275]: failed to begin cluster "agg-cluster" authorized CIDR update
Unexpected error:
    <*exported.ResponseError | 0x123635df0060>: 
    PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/feature-aggregation-k256l8ffc826/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/agg-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"4.21\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"",
        "target": "properties.version.id"
      }
    }
    --------------------------------------------------------------------------------
    
    <Azure error internals omitted>
        Azure request IDs:
          PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/feature-aggregation-k256l8ffc826/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/agg-cluster
            correlation ID: a49c09da-23cf-4bbe-85aa-61274c4b30d8
            client request ID: unavailable
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "4.21": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is "nightly"; provider Microsoft.RedHatOpenShift
23:00:08 UTC · failed at e2e · test Authorized CIDRs Connectivity should allow API access only from authorized VM IP · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:408]: failed to begin updating cluster "cidr-connectivity-test" authorized CIDRs
Unexpected error:
    <*exported.ResponseError | 0x2d38089f60c0>: 
    PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-sn4b9rdbmj8b/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cidr-connectivity-test
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"4.21\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"",
        "target": "properties.version.id"
      }
    }
    --------------------------------------------------------------------------------
    
    <Azure error internals omitted>
        Azure request IDs:
          PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-sn4b9rdbmj8b/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cidr-connectivity-test
            correlation ID: b31910fb-03a6-4862-89a6-260d7d2b4114
            client request ID: unavailable
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_authorized_cidrs_connectivity.go:408]: failed to begin updating cluster "cidr-connectivity-test" authorized CIDRs
Unexpected error:
    <*exported.ResponseError | 0x2d38089f60c0>: 
    PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-sn4b9rdbmj8b/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cidr-connectivity-test
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"4.21\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"",
        "target": "properties.version.id"
      }
    }
    --------------------------------------------------------------------------------
    
    <Azure error internals omitted>
        Azure request IDs:
          PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/e2e-cidr-connectivity-sn4b9rdbmj8b/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/cidr-connectivity-test
            correlation ID: b31910fb-03a6-4862-89a6-260d7d2b4114
            client request ID: unavailable
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "4.21": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is "nightly"; provider Microsoft.RedHatOpenShift
23:00:08 UTC · failed at e2e · test Customer should be able to create an HCP cluster and manage ImageDigestMirrors · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/idms_lifecycle.go:175]: failed to update cluster with second ImageDigestMirror set
Unexpected error:
    <*exported.ResponseError | 0x29cb3e4c46f0>: 
    PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/idms-bwmstl7szq95/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/idms-e2e-hcp-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"4.21\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"",
        "target": "properties.version.id"
      }
    }
    --------------------------------------------------------------------------------
    
    <Azure error internals omitted>
        Azure request IDs:
          PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/idms-bwmstl7szq95/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/idms-e2e-hcp-cluster
            correlation ID: fabaa794-9569-470e-a0c1-e00848586022
            client request ID: unavailable
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/idms_lifecycle.go:175]: failed to update cluster with second ImageDigestMirror set
Unexpected error:
    <*exported.ResponseError | 0x29cb3e4c46f0>: 
    PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/idms-bwmstl7szq95/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/idms-e2e-hcp-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"4.21\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"",
        "target": "properties.version.id"
      }
    }
    --------------------------------------------------------------------------------
    
    <Azure error internals omitted>
        Azure request IDs:
          PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/idms-bwmstl7szq95/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/idms-e2e-hcp-cluster
            correlation ID: fabaa794-9569-470e-a0c1-e00848586022
            client request ID: unavailable
occurred
VerifyBasicAccess failed: failed to list services: Unauthorized
23:00:08 UTC · failed at e2e · test ARO-HCP should be able to perform a control plane and node pool install with OCP nightly channel for 4.20 · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:136]: failed to verify HCP cluster "cluster-nightly-4-20-5bscpj" is viable
Unexpected error:
    <*errors.joinError | 0x1973f7046c48>: 
    VerifyAllAPIServicesAvailable failed: failed to list all APIServices: Unauthorized
    VerifyBasicAccess failed: failed to list services: Unauthorized
    {
        errs: [
            <*fmt.wrapError | 0x1973f6b0da20>{
                msg: "VerifyAllAPIServicesAvailable failed: failed to list all APIServices: Unauthorized",
                err: <*fmt.wrapError | 0x1973f6b0d9e0>{
                    msg: "failed to list all APIServices: Unauthorized",
                    err: <*errors.StatusError | 0x1973f6b18e60>{
                        ErrStatus: {
                            TypeMeta: {Kind: "", APIVersion: ""},
                            ListMeta: {
                                SelfLink: "",
                                ResourceVersion: "",
                                Continue: "",
                                RemainingItemCount: nil,
                                ShardInfo: nil,
                            },
                            Status: "Failure",
                            Message: "Unauthorized",
                            Reason: "Unauthorized",
                            Details: nil,
                            Code: 401,
                        },
                    },
                },
            },
            <*fmt.wrapError | 0x1973f71a3620>{
                msg: "VerifyBasicAccess failed: failed to list services: Unauthorized",
                err: <*fmt.wrapError | 0x1973f71a3600>{
                    msg: "failed to list services: Unauthorized",
                    err: <*errors.StatusError | 0x1973f7199c20>{
                        ErrStatus: {
                            TypeMeta: {Kind: "", APIVersion: ""},
                            ListMeta: {
                                SelfLink: "",
                                ResourceVersion: "",
                                Continue: "",
                                RemainingItemCount: nil,
                                ShardInfo: nil,
                            },
                            Status: "Failure",
                            Message: "Unauthorized",
                            Reason: "Unauthorized",
                            Details: nil,
                            Code: 401,
                        },
                    },
                },
            },
        ],
    }
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/complete_cluster_create_multiversion.go:136]: failed to verify HCP cluster "cluster-nightly-4-20-5bscpj" is viable
Unexpected error:
    <*errors.joinError | 0x1973f7046c48>: 
    VerifyAllAPIServicesAvailable failed: failed to list all APIServices: Unauthorized
    VerifyBasicAccess failed: failed to list services: Unauthorized
    {
        errs: [
            <*fmt.wrapError | 0x1973f6b0da20>{
                msg: "VerifyAllAPIServicesAvailable failed: failed to list all APIServices: Unauthorized",
                err: <*fmt.wrapError | 0x1973f6b0d9e0>{
                    msg: "failed to list all APIServices: Unauthorized",
                    err: <*errors.StatusError | 0x1973f6b18e60>{
                        ErrStatus: {
                            TypeMeta: {Kind: "", APIVersion: ""},
                            ListMeta: {
                                SelfLink: "",
                                ResourceVersion: "",
                                Continue: "",
                                RemainingItemCount: nil,
                                ShardInfo: nil,
                            },
                            Status: "Failure",
                            Message: "Unauthorized",
                            Reason: "Unauthorized",
                            Details: nil,
                            Code: 401,
                        },
                    },
                },
            },
            <*fmt.wrapError | 0x1973f71a3620>{
                msg: "VerifyBasicAccess failed: failed to list services: Unauthorized",
                err: <*fmt.wrapError | 0x1973f71a3600>{
                    msg: "failed to list services: Unauthorized",
                    err: <*errors.StatusError | 0x1973f7199c20>{
                        ErrStatus: {
                            TypeMeta: {Kind: "", APIVersion: ""},
                            ListMeta: {
                                SelfLink: "",
                                ResourceVersion: "",
                                Continue: "",
                                RemainingItemCount: nil,
                                ShardInfo: nil,
                            },
                            Status: "Failure",
                            Message: "Unauthorized",
                            Reason: "Unauthorized",
                            Details: nil,
                            Code: 401,
                        },
                    },
                },
            },
        ],
    }
occurred
ERROR CODE: InvalidRequestContent; detail message Invalid value: "4.21": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is "nightly"; provider Microsoft.RedHatOpenShift
23:00:08 UTC · failed at e2e · test Customer should be able to create an HCP cluster with custom autoscaling and update the autoscaling configuration · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:173]: failed to update cluster "autoscaling-hcp-cluster" autoscaling configuration
Unexpected error:
    <*exported.ResponseError | 0x3a1019d5f0>: 
    PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/autoscaling-cluster-gmn6p5j7ss7r/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/autoscaling-hcp-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"4.21\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"",
        "target": "properties.version.id"
      }
    }
    --------------------------------------------------------------------------------
    
    <Azure error internals omitted>
        Azure request IDs:
          PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/autoscaling-cluster-gmn6p5j7ss7r/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/autoscaling-hcp-cluster
            correlation ID: 653e93b6-5f35-4ec0-bb81-4f838d4d1e7b
            client request ID: unavailable
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_autoscaling.go:173]: failed to update cluster "autoscaling-hcp-cluster" autoscaling configuration
Unexpected error:
    <*exported.ResponseError | 0x3a1019d5f0>: 
    PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/autoscaling-cluster-gmn6p5j7ss7r/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/autoscaling-hcp-cluster
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: InvalidRequestContent
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "InvalidRequestContent",
        "message": "Invalid value: \"4.21\": must be specified as MAJOR.MINOR.PATCH (optionally with a pre-release, e.g. a nightly build suffix) when channelGroup is \"nightly\"",
        "target": "properties.version.id"
      }
    }
    --------------------------------------------------------------------------------
    
    <Azure error internals omitted>
        Azure request IDs:
          PATCH https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/autoscaling-cluster-gmn6p5j7ss7r/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/autoscaling-hcp-cluster
            correlation ID: 653e93b6-5f35-4ec0-bb81-4f838d4d1e7b
            client request ID: unavailable
occurred
VerifyBasicAccess failed: failed to list services: Unauthorized
23:00:08 UTC · failed at e2e · test Customer should be able to use workload identity via the cluster OIDC issuer URL · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/oidc_issuer_workload_identity.go:254]: cluster oidc-wi-cluster is not viable
Unexpected error:
    <*errors.joinError | 0xb2e45ac2030>: 
    VerifyAllAPIServicesAvailable failed: failed to list all APIServices: Unauthorized
    VerifyBasicAccess failed: failed to list services: Unauthorized
    {
        errs: [
            <*fmt.wrapError | 0xb2e4548fec0>{
                msg: "VerifyAllAPIServicesAvailable failed: failed to list all APIServices: Unauthorized",
                err: <*fmt.wrapError | 0xb2e4548fe80>{
                    msg: "failed to list all APIServices: Unauthorized",
                    err: <*errors.StatusError | 0xb2e456ea460>{
                        ErrStatus: {
                            TypeMeta: {Kind: "", APIVersion: ""},
                            ListMeta: {
                                SelfLink: "",
                                ResourceVersion: "",
                                Continue: "",
                                RemainingItemCount: nil,
                                ShardInfo: nil,
                            },
                            Status: "Failure",
                            Message: "Unauthorized",
                            Reason: "Unauthorized",
                            Details: nil,
                            Code: 401,
                        },
                    },
                },
            },
            <*fmt.wrapError | 0xb2e44d0e800>{
                msg: "VerifyBasicAccess failed: failed to list services: Unauthorized",
                err: <*fmt.wrapError | 0xb2e44d0e600>{
                    msg: "failed to list services: Unauthorized",
                    err: <*errors.StatusError | 0xb2e455a0500>{
                        ErrStatus: {
                            TypeMeta: {Kind: "", APIVersion: ""},
                            ListMeta: {
                                SelfLink: "",
                                ResourceVersion: "",
                                Continue: "",
                                RemainingItemCount: nil,
                                ShardInfo: nil,
                            },
                            Status: "Failure",
                            Message: "Unauthorized",
                            Reason: "Unauthorized",
                            Details: nil,
                            Code: 401,
                        },
                    },
                },
            },
        ],
    }
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/oidc_issuer_workload_identity.go:254]: cluster oidc-wi-cluster is not viable
Unexpected error:
    <*errors.joinError | 0xb2e45ac2030>: 
    VerifyAllAPIServicesAvailable failed: failed to list all APIServices: Unauthorized
    VerifyBasicAccess failed: failed to list services: Unauthorized
    {
        errs: [
            <*fmt.wrapError | 0xb2e4548fec0>{
                msg: "VerifyAllAPIServicesAvailable failed: failed to list all APIServices: Unauthorized",
                err: <*fmt.wrapError | 0xb2e4548fe80>{
                    msg: "failed to list all APIServices: Unauthorized",
                    err: <*errors.StatusError | 0xb2e456ea460>{
                        ErrStatus: {
                            TypeMeta: {Kind: "", APIVersion: ""},
                            ListMeta: {
                                SelfLink: "",
                                ResourceVersion: "",
                                Continue: "",
                                RemainingItemCount: nil,
                                ShardInfo: nil,
                            },
                            Status: "Failure",
                            Message: "Unauthorized",
                            Reason: "Unauthorized",
                            Details: nil,
                            Code: 401,
                        },
                    },
                },
            },
            <*fmt.wrapError | 0xb2e44d0e800>{
                msg: "VerifyBasicAccess failed: failed to list services: Unauthorized",
                err: <*fmt.wrapError | 0xb2e44d0e600>{
                    msg: "failed to list services: Unauthorized",
                    err: <*errors.StatusError | 0xb2e455a0500>{
                        ErrStatus: {
                            TypeMeta: {Kind: "", APIVersion: ""},
                            ListMeta: {
                                SelfLink: "",
                                ResourceVersion: "",
                                Continue: "",
                                RemainingItemCount: nil,
                                ShardInfo: nil,
                            },
                            Status: "Failure",
                            Message: "Unauthorized",
                            Reason: "Unauthorized",
                            Details: nil,
                            Code: 401,
                        },
                    },
                },
            },
        ],
    }
occurred
2h 28m 27s branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel
post-goodmerged PR
No additional run metadata captured
 westeurope n/a passed n/a 0 n/a
2h 30m 10s branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel
post-goodmerged PR
No additional run metadata captured
 brazilsouth n/a passed n/a 0 n/a
2h 28m 41s branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel
post-goodmerged PR
No additional run metadata captured
 eastus2 n/a passed n/a 0 n/a
2h 25m 37s branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel
post-goodmerged PR
No additional run metadata captured
 canadacentral n/a passed n/a 0 n/a
2h 18m 48s branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel
post-goodmerged PR
No additional run metadata captured
 australiaeast n/a passed n/a 0 n/a
2h 18m 54s branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel
post-goodmerged PR
No additional run metadata captured
 centralindia n/a passed n/a 0 n/a
2h 18m 39s branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel
post-goodmerged PR
No additional run metadata captured
 switzerlandnorth n/a passed n/a 0 n/a
2h 15m 36s branch-ci-Azure-ARO-HCP-main-e2e-prod-e2e-parallel
post-goodmerged PR
No additional run metadata captured
 uksouth n/a passed n/a 0 n/a
1h 58m 3s periodic-ci-Azure-ARO-HCP-main-periodic-prod-e2e-parallel
post-goodmerged PR
No additional run metadata captured
 australiaeast e2e failed n/a 4
e2e (4)
VerifyHyperShiftIngressCertificate timed out after 10m0
02:00:17 UTC · failed at e2e · test ARO-HCP should serve the default ingress certificate for 5.1 · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/default_cluster_ingress_cert.go:165]: HyperShift's ingress certificate secret must exist and be referenced by IngressController/default
Expected success, but got an error:
    <*fmt.wrapError | 0x2064b1710780>: 
    VerifyHyperShiftIngressCertificate timed out after 10m0.009s: getting HyperShift ingress secret openshift-ingress/default-ingress-cert: client rate limiter Wait returned an error: context deadline exceeded
    {
        msg: "VerifyHyperShiftIngressCertificate timed out after 10m0.009s: getting HyperShift ingress secret openshift-ingress/default-ingress-cert: client rate limiter Wait returned an error: context deadline exceeded",
        err: <*fmt.wrapError | 0x2064b1710700>{
            msg: "getting HyperShift ingress secret openshift-ingress/default-ingress-cert: client rate limiter Wait returned an error: context deadline exceeded",
            err: <*fmt.wrapError | 0x2064b17106a0>{
                msg: "client rate limiter Wait returned an error: context deadline exceeded",
                err: <context.deadlineExceededError>{},
            },
        },
    }
fail [github.com/Azure/ARO-HCP/test/e2e/default_cluster_ingress_cert.go:165]: HyperShift's ingress certificate secret must exist and be referenced by IngressController/default
Expected success, but got an error:
    <*fmt.wrapError | 0x2064b1710780>: 
    VerifyHyperShiftIngressCertificate timed out after 10m0.009s: getting HyperShift ingress secret openshift-ingress/default-ingress-cert: client rate limiter Wait returned an error: context deadline exceeded
    {
        msg: "VerifyHyperShiftIngressCertificate timed out after 10m0.009s: getting HyperShift ingress secret openshift-ingress/default-ingress-cert: client rate limiter Wait returned an error: context deadline exceeded",
        err: <*fmt.wrapError | 0x2064b1710700>{
            msg: "getting HyperShift ingress secret openshift-ingress/default-ingress-cert: client rate limiter Wait returned an error: context deadline exceeded",
            err: <*fmt.wrapError | 0x2064b17106a0>{
                msg: "client rate limiter Wait returned an error: context deadline exceeded",
                err: <context.deadlineExceededError>{},
            },
        },
    }
ERROR CODE: NoRegisteredProviderFound; detail message No registered resource provider found for location 'australiaeast' and API version '2026-10-01' for type 'HCPOpenShiftClusters'. The supported api-versions are '2026-09-01-preview, 2026-06-30-preview, 2025-12-23-preview, 2024-06-10-preview'. The supported locations are 'uksouth, switzerlandnorth, canadacentral, australiaeast, westeurope, centralindia, brazilsouth, eastus2, centraluseuap, eastus2euap'.; provider Microsoft.RedHatOpenShift
02:00:17 UTC · failed at e2e · test Nodepool OS Disk Encryption should create a nodepool with customer-managed disk encryption via DES · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_osdisk_encryption.go:135]: failed to create HCP cluster des-encrypt
Unexpected error:
    <*fmt.wrapError | 0x1976922010a0>: 
    failed to create HCP cluster des-encrypt: failed starting cluster creation "des-encrypt" in resourcegroup="des-encrypt-4n7f599ndw26": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/des-encrypt-4n7f599ndw26/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/des-encrypt
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: NoRegisteredProviderFound
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "NoRegisteredProviderFound",
        "message": "No registered resource provider found for location 'australiaeast' and API version '2026-10-01' for type 'HCPOpenShiftClusters'. The supported api-versions are '2026-09-01-preview, 2026-06-30-preview, 2025-12-23-preview, 2024-06-10-preview'. The supported locations are 'uksouth, switzerlandnorth, canadacentral, australiaeast, westeurope, centralindia, brazilsouth, eastus2, centraluseuap, eastus2euap'."
      }
    }
    --------------------------------------------------------------------------------
    
    <Azure error internals omitted>
        Azure request IDs:
          PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/des-encrypt-4n7f599ndw26/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/des-encrypt
            correlation ID: 90fe650f-dd4d-4539-aefc-1901e9671e38
            client request ID: unavailable
occurred
fail [github.com/Azure/ARO-HCP/test/e2e/nodepool_osdisk_encryption.go:135]: failed to create HCP cluster des-encrypt
Unexpected error:
    <*fmt.wrapError | 0x1976922010a0>: 
    failed to create HCP cluster des-encrypt: failed starting cluster creation "des-encrypt" in resourcegroup="des-encrypt-4n7f599ndw26": PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/des-encrypt-4n7f599ndw26/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/des-encrypt
    --------------------------------------------------------------------------------
    RESPONSE 400: 400 Bad Request
    ERROR CODE: NoRegisteredProviderFound
    --------------------------------------------------------------------------------
    {
      "error": {
        "code": "NoRegisteredProviderFound",
        "message": "No registered resource provider found for location 'australiaeast' and API version '2026-10-01' for type 'HCPOpenShiftClusters'. The supported api-versions are '2026-09-01-preview, 2026-06-30-preview, 2025-12-23-preview, 2024-06-10-preview'. The supported locations are 'uksouth, switzerlandnorth, canadacentral, australiaeast, westeurope, centralindia, brazilsouth, eastus2, centraluseuap, eastus2euap'."
      }
    }
    --------------------------------------------------------------------------------
    
    <Azure error internals omitted>
        Azure request IDs:
          PUT https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/resourceGroups/des-encrypt-4n7f599ndw26/providers/Microsoft.RedHatOpenShift/hcpOpenShiftClusters/des-encrypt
            correlation ID: 90fe650f-dd4d-4539-aefc-1901e9671e38
            client request ID: unavailable
occurred
ACM ingress secret openshift-ingress/cluster-ingress-cert exists; expected ACM certificate delivery to be suppressed
02:00:17 UTC · failed at e2e · test Customer should be able to issue the default ingress serving certificate from Let's Encrypt using cert-manager with ACME DNS-01 challenges for cluster version 5.1 · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_ingress_cert_letsencrypt.go:509]: ACM must not create its ingress certificate secret
Expected success, but got an error:
    <*errors.errorString | 0xf07f8c50bb0>: 
    ACM ingress secret openshift-ingress/cluster-ingress-cert exists; expected ACM certificate delivery to be suppressed
    {
        s: "ACM ingress secret openshift-ingress/cluster-ingress-cert exists; expected ACM certificate delivery to be suppressed",
    }
fail [github.com/Azure/ARO-HCP/test/e2e/cluster_ingress_cert_letsencrypt.go:509]: ACM must not create its ingress certificate secret
Expected success, but got an error:
    <*errors.errorString | 0xf07f8c50bb0>: 
    ACM ingress secret openshift-ingress/cluster-ingress-cert exists; expected ACM certificate delivery to be suppressed
    {
        s: "ACM ingress secret openshift-ingress/cluster-ingress-cert exists; expected ACM certificate delivery to be suppressed",
    }
external auth <external-auth>.Status was nil
02:00:17 UTC · failed at e2e · test Customer should be able to create a cluster with an external auth config and get the external auth config · suite prod/parallel
Show raw failure
fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:343]: Timed out after 300.003s.
Degraded condition did not converge to False
The function passed to Eventually failed at /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:323 with:
external auth Properties.Status was nil
Expected
    <*armredhatopenshifthcp.ResourceStatus | 0x0>: nil
not to be nil
fail [github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:343]: Timed out after 300.003s.
Degraded condition did not converge to False
The function passed to Eventually failed at /opt/app-root/src/github.com/Azure/ARO-HCP/test/e2e/external_auth_create.go:323 with:
external auth Properties.Status was nil
Expected
    <*armredhatopenshifthcp.ResourceStatus | 0x0>: nil
not to be nil